The guess and check pattern
Here's a software pattern that I've found new use for in our new world of agents. I'll motivate it with an example from build systems.
When using build systems you often write down some information that feels redundant. For example, from the source code you can see that module A imports from module B, but then the build system requires you to separately tell it to build module B before building A. Why do we need to write this out, when it is visible from the source? It is tempting to figure it out automatically, as tools like ekam do.
The easy answer for why we don't always do this is the normal software answer of
path dependence and how software tends to be clunky. But there are also two good
reasons: information like this can be costly to gather and also can be hard to
get completely right. It can be costly if your language design you might need to
parse the whole source tree to find where a given module actually lives, as
perhaps with C++ modules.
It can be hard to get correct if there is configuration complexity; for example
in a C file with #include "config.h", it is not clear which of potentially
multiple config.h candidates it means; a tool that guesses will be hard to
predict and sometimes wrong. (Just so I'm not only picking on C, I'll add that
finding where a given import ... from 'x' in JavaScript resolves to has
surprisingly complex semantics,
and can even depend on the path of the source file.)
However, in the common case it really is often possible to automate things, and it's not that costly. This leads us to what in my head I call the guess and check pattern: use a tool to automatically guess, edit and/or verify the result, and write that down as the definitive answer.
For example Gazelle updates
Bazel BUILD files from Go source. Bazel then gets to run quickly without
parsing Go source, and if Gazelle gets things wrong, you can still edit the
files. As an additional benefit, you check the result in to source control,
which means a human gets to see what actually changed, and it pins down the
result.
For another example of this pattern, consider "lock files" as used in
programming language package managers. When you npm add or cargo add etc.,
the tooling makes network requests, runs a
complex algorithm to
resolve what you meant, and writes down the result. Builds after this point can
use the lock files without looking around on the network or guessing about
versions.
I used this pattern in my Windows emulator projects. For any given Windows function I declare the emulator's version of the function in Rust. To automate writing out the function's matching argument list I can generate it from metadata published by Microsoft. But I only need that metadata when I run the generator, and the output doesn't need to be perfect! I can generate code that is most of the way there, because it's fine as a human to fix up the details after. The generator can even just make up type names that don't exist because the compiler will catch it afterwards.
Another way to look at this pattern is to see there's a chain of work to guarantee determinism atop a nondeterministc start: you can use a tool that makes a guess, and as long as you write down that guess and use deterministic steps from that written state onward you still have determinism in the result.
Which finally leads me to one way I've been tentatively experimenting with AI. AI behavior matches the shape I've been describing. An agent can do a long slow processing step and generate some output — not limited to code, even things like gathering data — as long as I write the output down as a result and have a human and/or tooling verification pass after. Keeping this pattern in mind has helped me find places where AI is suitable: problems where I know up front that a plausible guess is all I require, and that framing helps me keep in mind that the output is always just a guess.
(PS: just in case it wasn't obvious, my blog posts are always just me banging away on a keyboard, no clankers will keep me from my meandering!)