Quality Requires Visibility

Quality requires visibility. You cannot fix — or even trust — what you cannot see. That’s true in any project, but it becomes even more urgent when the team doing the work is a software factory and you’re not reading every line of code.

When I’m not micro-managing the implementation, the thing I’m actually managing is my own insight: the gap between what the system was intended to do and what it’s actually doing. So let’s go over a little bit of what I’ve done to ensure I’ve got the visibility I need to ship a robust experience while using unreliable coding agents to do the heavy lifting.

In Which I Intervene in the Code

(Image credit: Hang Xu)

Three issues emerged that warranted some manual intervention this week. All three have prompted me to refine the process a bit more.

  1. After auditing the codebase for needless defensive guard code, neither Claude nor Codex managed to find any.
  2. The overhaul to how sources / destinations are specified silently broke the Fleeing mechanic for burning orcs.
  3. Low usage of Haiku and Sonnet compared to Opus.

So here’s what happened, and what I did about it.

The Meta-Game Begins

The original Hordes of Orcs games were pretty simple things. Everything was available straight away — every map, every tower, every upgrade, every spell. That’s not a bad thing, and I don’t regret it. But it does mean people burned through all of the content pretty quickly.

For this game, I want the experience to last longer. That means two things: there needs to be a meta-game, and it needs to be a lot easier to author new maps. The last six days were largely about both — and about the unanticipated consequences of getting there.

Progress, 37 Days in

Three posts in, this series has been almost entirely about process — the factory, the sub-agents, the invisible work that keeps a codebase from strangling itself. Every one of them ended with some variation of “the game continues to come along nicely,” and asked you to take my word for it.

This post is me finally showing my work. Here is roughly a minute of Hordes of Orcs 3 as it stood on day 34.

The Invisible Work Matters

A lot of engineering is invisible work. Work that never surfaces to a manager, and that either doesn’t show up in a PR diff or isn’t explicitly explained for what it is. It’s often work that the engineer doesn’t consciously register as work at all. Because it doesn’t get talked about, very little of it ends up written down. Because very little of it ends up written down, very little of it ended up in the training data for the models I’m now handing my codebase to.

That absence is a quiet, compounding risk to the long-term health of a project. By its nature, it tends to shorten what “long-term” even means.