Agents Audit What Humans Assume

I was telling an AI agent how happy I was with my new Linux desktop. Everything just worked, and the ultrawide monitor was finally running at its full resolution and refresh rate, something I’d never managed on Windows.

The agent didn’t just agree. It compared what I’d said with the machine. It read the display state and the small script I use to switch monitor layouts, and found that the script had the monitor hard-coded at a lower resolution and half the refresh rate. Every time the layout switched (including whenever I streamed the desktop to another device), the monitor quietly dropped to the lower mode. I’d never noticed, because it still worked.

A compliment turned into a bug report. That’s worth thinking about.

Systems drift away from what people believe about them

Any system that several people touch carries several mental models of how it runs. The models start out accurate and drift: a default changes, a workaround becomes permanent, someone copies a config from an old project. The system keeps working well enough that nobody looks, so it’s running a way, just not the way anyone thinks, and often not by the principles it was designed around.

Humans are bad at catching this. We check what we expect to be wrong, and we stop looking at things that seem fine.

Config as code turns beliefs into testable claims

What made my case easy is that the whole setup was code: dotfiles, a shell script, a config file, and a command that reports the live display state. Once a system is described like that, every “it does X” is a claim that can be checked: read the config, query the live state, run a test, compare.

An agent in the loop is good at exactly that. It isn’t smarter than the people involved. It just takes each stated belief literally and goes and looks, without assuming that things which seem fine are fine.

The loop is simple:

1. Say what you believe. Thinking out loud (“we always…”, “it’s set to…”, “this runs at…”) is the trigger. The most useful beliefs are the casual ones, because nobody thought to test them.
2. The agent checks it against the code, the config and the live system.
3. A human decides which side is right. Sometimes the config is wrong; sometimes the belief is, and the docs need fixing instead.
4. Make the check permanent. Turn the finding into a test, an alert, an eval or a fitness function, so the belief keeps being verified after everyone has forgotten it.

A fitness function for expectations

This is the same shape as an architecture fitness function: an automated check of a property you care about, so drift fails loudly instead of accumulating quietly. The difference is what gets checked. A fitness function in CI checks the properties someone remembered to encode. An agent listening to you checks the properties you only assumed, the ones still living in your head.

My monitor setup is a small but honest example of drift. I’d set it up slightly wrong, believed I’d set it up right, and the gap would have surfaced months later as confusion (“why does this look worse after streaming?”) with no obvious cause. Because everything was code, and an agent treated my expectation as a test to run, the drift was caught the day I stated it. The next step is to keep it caught: a one-line check that the monitor’s configured mode matches its best mode turns that conversation into a permanent fitness function.

Where this goes beyond a desktop

• Platform teams: does the golden path actually do what the docs promise? Does every new service really get observability by default, or only the ones created since the template changed?
• Security: are secrets really off disk, or did one tool write a token to a config file? (See Protecting SSH: Keys Off Disk, in a Password Manager.)
• Operations: are the alerts we rely on configured to fire for the thing we think they cover?
• AI features: evals are this idea at scale. They check what a model-backed feature does, not what its prompt was meant to make it do.

The limits

• Only what’s observable can be checked. Behaviour that lives in someone’s head, or in a console nobody exported, stays invisible. That’s another argument for Infrastructure as Code for People, Not Just Servers.
• Access is a trust decision. To check reality, an agent needs to read it. Give it read access to state and config, keep secrets out of reach, and approve anything that changes the system.
• It doesn’t replace judgement. The agent finds the gap between belief and behaviour; deciding which one is right is still a human call.

The takeaway I keep coming back to: write your assumptions down where something can check them. Notes and conversations used to be where assumptions went to be forgotten. With an agent that can read the system, they’re where assumptions go to be tested.