Articles and Posts

Wrapping 3rd-party dependencies in interfaces reveals leaky abstractions. I build an abstract syntax tree, then render it. In render, I allowed access to underlying compiler symbols. Now I can’t. I’m discovering where I unnecessarily couple render to compiler implementation.

My tool can generate an abstract syntax tree for .NET code using the rosalyn compiler. I’m persisting that tree in a neo4j graph db. Now I need to recreate that AST using queries to the db. Then we can batch a repo’s full graph and retrieve vertical slices at will.

Abstract syntax tree representing a vertical slice of a legacy codebase stored in a graph database for retrieval as context for AI completions. The full set of symbols and and filtered for methods and named types (e.g. classes)

Anchoring prompt for the code gen agent in my IDE before launching into work.

Me - “This is VERY IMPORTANT. You must red green test drive changes to the code. We need to make changes in small testable increments, first by introducing or modifying a test to the new expectation which will fail, then introducing the change to code, then verify with passing tests before moving on.”

Agent - “Absolutely right. Let’s follow a strict TDD approach with small, testable increments. I’ll start by examining the existing test structure and then make targeted changes

Not only do the models lack an ethical imperative to be truthful they may be influenced by perverse incentives to be untruthful.

Relevant, focused context is vital for effective code generation in existing codebases. Even in my own greenfield tracing tool, rushing ahead of my understanding creates problems. When relying on whatever code context the codegen assistant finds for itself, we end up co-creating redundant implementations rather than correctly leveraging existing patterns just outside the immediate context.

We have in the bank validation from two clients that the vertical slices of code and database context we can assemble for legacy .NET codebases does enable an LLM to provide useful insights and input into a rewrite project.

The next step is to place this context into a queryable form so human experts can explore and rewrite their existing codebases as they best see fit. We’re exploring using a graph db so that we can target how broad or narrow a swath of context to provide for different use cases at different levels of abstraction: domain mapping, story writing, and code suggestions.

Try telling your code generation agent it can’t suggest a production code change until it has produced a valid failing test. Assuming you can assess what a valid test is, the quality of the interaction (on its side - and yours) steps up immediately.

The cycle I find with code generation is that I move quickly - past comfort at or beyond the rim of my comprehension. That reaches a breaking point. Then comes reflection, I slow up, dive into the implementation, wrest control of it. Then I speed up again. Sustainable not steady.

Sometimes that new feature request disappears when we do a thorough clean up of the existing code. Turns out the “new” behavior should have always been there but for a logic flaw in the implementation. The tree no one heard falling in the forest.

Reasoning models appear to be designed to help people solve a complex problems they are not capable of solving for themselves. Unfortunately, at the moment, that appears to be as a tradeoff to the fidelity to instruction that would accelerate the work of people who can.