Short-form insights you can read in under 5 minutes.
It’s about people. I embrace generative AI in my work. But what makes my work useful is the skill and generosity of my coworkers. What makes the time joyful is learning and laughing with them. What makes the effort meaningful is the people we do it for.
Finding myself in the camp of preferring fixtures over mocks in unit tests. It’s a debate I have with myself periodically. Usually when I let myself get so rusty I forget how easy fixtures are to work with.
A powerful tool can still be the wrong tool for a particular problem.
Vector-based approximate nearest neighbor search is incredible for finding conceptually related content. Context built from this type of search is part of the “magic” in text based RAG implementations.
But sometimes a cigar is just a cigar. A more straightforward approach is better when we know what we’re looking for and the content is semantically rich or highly structured.
For example, use source code indexing to provide dependencies to LLM-based code refactoring.
Hit my patience limit of manual repetition of a task before I wrote code to automate (maybe 10 times?). When I was younger it was closer to two.
High tech is certainly in a novelty/ hype / r&d phase. The real threats and opportunities will leak out from the edges.
BTW There are robot lawn mowers that can be had for several grand. Kind of like a roomba with very sharp rotating teeth.
Meta released a new version of Llama. So, there’s a new open mega model. Currently 8B and 70B parameters. Access to an open model allows developers to decide for specific use cases the parameter size and precision at which to operate the model. I’ve had some success using a larger Mistral AI model quantized down to a smaller memory footprint for simple prompts. I can also confirm speed/cost optimized models can perform well at code completion. Specifically — Anthropic Claude3 Sonnet.
As to the new Meta Llama :
“Improvements in our post-training procedures substantially reduced false refusal rates, improved alignment, and increased diversity in model responses. We also saw greatly improved capabilities like reasoning, code generation, and instruction following making Llama 3 more steerable.”
General advice is to make prompts to mega models verbose. But in reiterating an instruction you can inadvertently create ambiguity in the difference in wording between the repetitions creating less predictable responses.
Learned this after spending a day iterating on prompt and instructions to get Claude 3 sonnet to make a consistent code refactor across a dozen variations in many files of code.
Spent several hours crafting a prompt and instructions to refactor code.
The goal was to make a prompt with the minimum of direction specific to the problem to get the desired code change while providing guiding instructions reusable across many such prompts.
The nuances of how wording affects output is fascinating…
When the agent makes a mistake, asking it how to reword the prompt and instructions to avoid the mistake in future works.
By 2026, the AI industry is expected to have grown exponentially to consume at least ten times its demand in 2023.”
“(From 2022 to 2026)… we expect global electricity consumption of data centres, cryptocurrencies and artificial intelligence to (consume) roughly equivalent to adding at least one Sweden or at most one Germany.”
As a participant (however small) in the adoption of generative AI, it’s my obligation to look for ways to reduce computes expended in development and ongoing operation. Even as I have to look to platform, data center, and energy providers, as well as governments and industry to evolve more efficient, transparent ways of delivering those computes.
To start with, I need to find ways to reduce wasted processing time, use lower resource consuming models where and when possible, and apply AI solutions only when they provide a meaningful value over less energy intensive traditional approaches.
#iea #electricity #ai
Much of what software developers are asked to do isn’t particularly creative problem solving. And just as we should rely on existing libraries to solve solved problems, we might be able to rely heavily on generic completions from Mega models on these kinds of problems. But…
Much of what we do is creative problem solving — not just on the leading edge of innovation but everyday — because the problem space includes the human and software systems that surround the work to be done. In fact leaders, including the people who seek to displace the workers with automation, create many of the obstacles that make delivering so difficult.
The attached article and referenced research finds that in order to move away from training bias and approach creativity, you need to guide an AI assistant with challenging follow up prompts. And that the responses are a tool for solving a creative problem, not the solution in and of itself.
AI will still help us. Possibly considerably. But it will need masterful guidance. Either at the level of the tool makers or the tool users. Probably both.
How Engineers feel when people talk about AI making engineering obsolete:
We’re told we’re the stone chipper who will be irrelevant with the advent of bronze. But we’re more likely the person who will need to tie the bit of sharp to the stick. So, that’s all good then.
There’s alot more to using generating AI effectively and responsibly then setting up a hello world RAG app and letting the mega model do magic. The irony being this reality means there are many really fascinating software engineering challenges to solve while the hype is about software engineers becoming obsolete.