You can’t tack production requirements onto rapidly produced code. The age of AI hasn’t changed this fact.

Agentic code agents can rapidly prototype applications in hours. This is amazing. These prototypes can quickly help people test the viability and value of a feature or implementation approach.

Production code is a complex mix of such details that require scalability, safety, security, deployability, test coverage, quality evaluation and stakeholder sign off, and optimization for performance, troubleshooting and maintainability. Production code takes orders of magnitude longer to create than prototypes.

The disposable prototypes agents generate usually can’t even be incorporated into such systems without a rewrite. Production software requires a more human centric and expert led approach from the start.

Finally, the speed of delivering software to users is not directly driven by the speed at which the code itself is produced. Code authoring is the middle of a value stream and is not the sole bottleneck in building effective software. AI can also speed up investigation and learning. It can facilitate communication and documentation. In the hands of qualified, experienced teams, AI tools can speed up actual delivery of quality production software by 10-30%.

This gain is worth shooting for but it is an iterative, evolutionary practice improvement. Not something you get out of the box as part of a software license.

Using Anthropic Claude models for coding:

Opus is strong on analysis. Industry benchmarks show it’s strong at coding, but I find it ambitious and rangy. It drifts beyond its mission when struggling, and will charge down supposition paths without grounding.

Sonnet is good at tasking out work and quite good at following through on well-defined tasks, but can be short-sighted in terms of pattern and method reuse.

The current version of Haiku falls short. It shows its age with anti-patterns like aggressively duplicating code and hardcoding values, particularly when spins on a task.

So, for now, I live in Sonnet. I use Opus mostly for analysis on big problems. I’ve stopped trying to use Haiku for now. Would love to hear from someone who is getting decent results in complex code with one of the smaller models.

These days, I do my code work in Cursor with Anthropic models in a Plan-Do-Check-Act cycle.

I tried experimenting with using Haiku for some of my implementation (Do) tasks after using Sonnet for planing. The hope was to bring down cost per token on my work. But I didn’t have much luck. The results are quite understandably very poor. Despite detailed tasking from higher models, Haiku is too likely to do things like duplicate code instead of re-use it, hard code values instead of use properties, etc. etc. Too many poor patterns to enumerate. Cleaning that up exceeds any savings I’d get. I’ll wait for a model update.

In fact, I’ve started doing some of the more complicated analysis and planning with Opus. So cost per token is going up. The hope is that better analysis and planning leads to a cleaner, shorter coding cycle but I won’t have enough data and experience to establish that for a bit.

Is anyone else having luck with lower cost models for some aspects of their coding process?