Someone asks me to build a data engineering agent. I end up building a platform to create and deploy agents in general. We need charts with a specific brand style. I end up building a charting library wrapper and a YAML-based config system so anyone on the team can generate them without touching Python.

3x the effort on day one. Then 10 minutes every time after.

This is not a flex. It’s a pattern I’ve noticed in myself, and honestly it’s a problem sometimes. But when it works, it really works.

Where this instinct comes from

I think this started with Ruby on Rails. Convention over configuration was the first framework I fell in love with. The idea that you could encode decisions into the framework itself, so individual developers didn’t have to make those decisions every time, was genuinely mind-expanding to me. Even when I barely knew how to code I was already trying to do meta-programming.

Rails taught me that the best abstractions aren’t about writing less code. They’re about making the right behavior the default and making the wrong behavior hard. You don’t tell every developer to name their database table the plural of the model name. You make that the convention, and only force them to think about it when they need to deviate.

That principle generalized. Every time I look at a task now, some part of my brain is asking: is this an instance or a pattern? Am I solving something, or am I going to be solving this same kind of thing again next month?

The agent platform example

The data engineering agent story is a real one. We needed an agent that would monitor a set of pipelines, detect failures, diagnose root causes, and either fix the issue automatically or escalate with context. Straightforward enough.

But as I started building it, I kept running into questions that had nothing to do with the specific use case. How do we deploy this? How do we monitor the agent itself? How do we version its prompts? How do we test changes without breaking production? How do we give it access to tools safely?

These aren’t pipeline-monitoring questions. They’re agent infrastructure questions. And they’d come up again the next time someone wanted to build a different agent.

So instead of building a pipeline-monitoring agent, I built a platform for defining, deploying, and managing agents. The pipeline monitor became the first agent deployed on that platform. The second agent took a fraction of the time to build because all the infrastructure was already in place.

Was this the right call? In that case, yes, because we knew more agents were coming. But I’ve made the wrong call before. I’ve built platforms for things that only ever had one instance. I’ve built configuration systems that only ever had one configuration. The upfront cost was real and the payoff never came.

The charting library example

The charting one is more nuanced. We had a brand style guide and needed a set of charts for a report. The obvious move is to fire up matplotlib, style each chart manually, and move on. Takes a few hours, looks good, done.

But I’d been through this before. The report would come back next quarter. Someone would want a similar chart for a different dataset. A new team member would try to match the style and get it slightly wrong. Small variations would accumulate until the charts looked like they came from five different companies.

So I built a thin wrapper around the charting library that encoded the brand defaults: colors, fonts, grid style, spacing. Then I added a YAML config layer so you could describe a chart declaratively: here’s the data, here’s the chart type, here’s the title. The wrapper handled everything else.

First chart: maybe 4x the effort of just writing it directly. Every chart after that: a YAML file and a one-line command. New team members could produce on-brand charts without knowing Python. Changes to the brand style happened in one place and propagated everywhere.

The trick is that the wrapper didn’t try to do everything. It didn’t cover every possible chart type or every conceivable customization. It covered the 80% case well and got out of the way for the remaining 20%. That restraint is what made it actually useful instead of being yet another abstraction that’s harder to work with than the thing it abstracts.

When systems beat one-offs

The pattern that makes a system worth building:

Repetition. If you’re going to do something once, just do it. The threshold for me is usually three. If I’ve done something twice and I can see it happening again, that’s when the system starts making sense. Not before.

Variation with structure. The task keeps coming back, but with different inputs or slightly different requirements. This is the sweet spot for systems. The structure stays the same, the parameters change. A config file, a template, a factory function: these all exploit this pattern.

Multiple people. When it’s just you, you can hold the context in your head. You remember why the chart is styled that way, where the pipeline config lives, how to deploy the thing. But the moment someone else needs to do it, all that implicit knowledge becomes a bottleneck. A system externalizes the knowledge.

Error-prone manual steps. Some tasks are not hard, they’re just finicky. Lots of steps, easy to miss one, and the consequences of getting it wrong are annoying. Deploy scripts, data validation, environment setup: these are better as systems not because they’re complex but because they’re tedious and easy to mess up.

When one-offs beat systems

This is the part I had to learn the hard way.

Exploration. When you’re still figuring out what you need, building a system is premature. You don’t know enough about the problem to encode the right abstractions. Build the thing three times badly, then build the system. Not the other way around.

Genuinely unique tasks. Some things really are one-offs. A one-time data migration, a specific analysis for a board meeting, a prototype to test an idea. If you catch yourself building a framework for a one-time migration, stop.

When the cost of abstraction exceeds the cost of repetition. Some tasks are so simple that systemizing them adds more complexity than it removes. If the “system” is harder to understand and maintain than just doing the thing manually each time, you’ve made things worse.

When you’re the only user and you can’t predict others. Building for hypothetical future users is a trap. If no one else is going to use your system, and you’re not sure they ever will, the system is for your own satisfaction, not for actual leverage.

AI changed the economics

Something interesting happened recently. Tools like Claude Code made building systems dramatically faster. The upfront cost that used to be 3-4x is now much closer to 1.5-2x, sometimes less. The calculus that used to require three expected repetitions to justify a system now tips in favor of the system much sooner.

But the more surprising effect is what happens after the system exists. It’s not just humans who benefit from it. The AI models themselves become significantly better when they can work within a system rather than starting from scratch.

I’ve been noticing this a lot lately. When I ask an AI to generate a one-off chart, the result is fine but generic. When I point it at my charting system and say “use this config format to create a chart for this dataset,” the output is consistently on-brand, follows our conventions, and needs minimal adjustment. The system constrains the AI in the same way it constrains a new team member: it encodes the decisions that have already been made so the AI doesn’t have to guess at them.

The same thing happened with the agent platform. Building a new agent from scratch with AI assistance is possible but messy. There are too many open decisions, too many ways it can go. But building a new agent on top of the platform, where the deployment, monitoring, and tooling patterns are already defined, is a much more tractable problem. The AI has clear boundaries, clear interfaces, and clear conventions to follow.

This creates a compounding loop. AI helps you build the system faster. Then the system makes AI more effective at solving instances of the problem. Which frees you up to build more systems. I don’t think we’ve fully internalized what this means yet, but the people who are building good systems right now are going to have a serious advantage as these tools keep getting better.

The meta-instinct

Don’t solve the instance, solve the class of problems. That’s the instinct. But the skill isn’t having the instinct. The skill is knowing when to act on it and when to override it.

The best version of this pattern isn’t “always build systems.” It’s recognizing the moment when a task stops being a task and starts being a pattern. It’s doing the manual thing the first time, noticing the repetition the second time, and building the system the third time. It’s having the restraint to wait for that signal instead of jumping straight to the abstraction.

Not every task deserves a system. But the ones that keep coming back? A system will always beat a one-off.