A familiar scene: two companies launch an AI agent on the same model. For one, it processes inbound requests reliably for months; for the other, it answers brilliantly today and produces nonsense tomorrow. The owner concludes "AI isn't ready yet." In reality, the difference isn't in the model but in three layers around it: the system of constraints, the way instructions are written, and access to fresh documentation. Let's take them one by one — with a checklist of questions for your contractor at the end.
The model is the engine, the harness is the car
Princeton studied 14 models and found an uncomfortable pattern: accuracy improves from generation to generation, reliability doesn't. Models get smarter without getting more predictable. They're different properties, and one doesn't automatically pull the other along.
That's why teams serious about building agents invest not in "a better model" but in the harness — the system of constraints around it. Manus rewrote their harness five times in six months: same models, five different constraint architectures — and every iteration improved the results. Vercel removed 80% of their agent's tools — and the agent got better: fewer options, less room for error. I run 17 agents in my daily work, and they're stable not thanks to the hottest model or "magic prompts" but thanks to strict constraints that make stepping out of line impossible.
In practice, a harness is a skill: a precise instruction with five elements. What to do (one paragraph, no filler). How to do it (three steps at most). Acceptance criteria (not met — the result doesn't count). An exact output format. And prohibitions (what never to do). One skill — one task; if the task is too broad, split it in two.
It sounds counterintuitive, but more constraints mean more stable output: narrowing the task and setting clear criteria push the share of successful runs to roughly 90%. A mediocre model in a strict harness is more reliable than the smartest one in chaos. And this isn't just about AI, by the way: think of your best employee working without a clear spec — the result is different every time.
Why "you are a seasoned senior engineer" doesn't work
The second layer is the instructions themselves. The popular recipe — "give the model a role and it gets smarter" — doesn't work. Anthropic described this reasoning error and named it the "Claude Mythos Problem."
A persona doesn't add knowledge to the model — it activates a stereotype from the training data. In the dataset, an "experienced senior engineer" isn't any particular engineer's experience; it's LinkedIn posts about work-life balance, cautious corporate wikis, and average forum answers. That's exactly how the agent will sound: averaged, predictable, dull.
What works instead is a functional role:
❌ "You are the best code reviewer with 20 years of experience"
✅ "You are a code reviewer. You receive code changes. You return at most 5 comments labeled: critical / warning / nitpick"
The first is an identity: zero new information for the model. The second is a goal, instructions, and acceptance criteria. The model doesn't need to know "who it is." It needs to know what it receives as input, what it returns as output, and where the quality bar sits.
The agent that lives in the past
The third layer shows up most clearly in a development example. Claude wrote getServerSideProps — an approach the project abandoned three years ago. The agent reads the code carefully, understands the architecture, remembers the context — and is simultaneously stuck in 2023 when it comes to documentation.
Every model has a cutoff date — the point after which it knows nothing. Next.js 14 ships — the model is still on 13. React 19 brings new tools — the model suggests the old ones. And this isn't "occasionally wrong," it's systematic behavior: with every popular library you end up checking whether a method is still current. A developer used to trusting the agent's suggestions loses an hour a day to those checks.
The fix is simple. Context7 by Upstash is an MCP server that plugs into Claude Code (Cursor and Codex too) and feeds the agent fresh library documentation in real time — straight from the source, not from the model's memory. Setup takes five minutes: one command, and from then on adding "use context7" to a request is enough. A real example: fresh library docs in 30 seconds, a module rewritten to the new API — not a single invented function, and the code worked on the first run.
The checklist: what to ask your contractor
None of these three layers are visible in a demo — in a presentation, the agent always works. But they're easy to probe with questions:
Question | The answer you want to hear |
|---|---|
How do you constrain the agent? | Each one has a skill: goal, steps, acceptance criteria, format, prohibitions |
How many tasks does one agent handle? | One. Broad tasks get split across several agents |
What happens to a wrong result? | If it fails the acceptance criteria, it doesn't count |
Do your prompts say "you are the best expert"? | No, roles are functional: input, output, quality bar |
Where does the agent get its documentation? | Live documentation in real time (MCP), not just the model's memory |
Results are unstable — what do you do? | Narrow the task and tighten the constraints, not swap the model |
If the answer to that last question is "we'll switch to a more powerful model," that's a red flag: swapping the engine when what's missing is steering and brakes. Instability is a sign of a weak harness, not a weak model.
Want to understand why your agent runs unreliably? The Grow2.ai AI audit breaks it down on a specific process. The foundational guide: AI agents for SMB. If the agent is going to live inside your processes and CRM, that's the domain of our sister brand Auspex; for the strategic view on AI — Andrew Maryasov.