How to Verify AI Agent Work: Gates for Code and Everything Else

Why can't I trust an AI agent's own report?

Hosted autonomous agent platforms have a recurring complaint against them: agents mark tasks complete when they are not. One published review of a hosted platform recorded a user's own tally of 47 tasks, of which 41 were reported completed while 24 had gone out with the wrong name, giving a real success rate around 21%.

The agent is not lying. It genuinely believes it finished. The problem is structural: nothing independent checked.

The instinct in the developer community is already correct. An r/ClaudeAI thread with 390+ upvotes and 180+ comments argues that anyone running Claude Code on a larger project should move to a CI pipeline immediately. Verification is exactly that instinct, applied per task rather than per merge.

What gates should I use for code?

Use the project's real commands. They are stronger evidence than anything generic and they are already the standard the rest of your work is held to.

• npm test, or go test ./..., or pytest
• The linter, because a passing test suite with broken style still needs a human
• A type check, which catches a category of damage tests often miss
• The build, for anything that compiles

How do I verify work that has no test suite?

This is the part almost nobody writes about, and it is where most non-engineering agent work fails silently. A content or research role still needs proof. The shape changes: instead of a test suite, prove the output exists, has substance, and is not half-finished.

• Existence and non-emptiness: test -s content/post.md
• Substance, so a two-line stub does not pass: test $(wc -w < content/post.md) -ge 400
• Freshness, so a role cannot pass by leaving last week's file alone: grep -q "$(date +%Y-%m-%d)" research/log.md
• No placeholders left behind: ! grep -rniE "lorem ipsum|TBD|TODO" content/
• Landed in the right place: test -n "$(ls -A outbox)"

What makes a verification gate useless?

This is the most common mistake and the most dangerous, because the result looks verified.

Commands like true, echo done, git status and git diff --stat all exit zero whether or not any work happened. A role with one of those has a gate in name only, and you will trust its output more than you would have trusted no gate at all.

The test is simple: can you describe a realistic failure that would make this command exit non-zero? If not, it is not a gate.

Should a failed gate trigger a retry or a repair?

A blind retry asks the agent to do the same thing again with the same information. It usually produces the same result.

A repair attempt hands back the failing command, why it failed, and the actual output. That is a materially different prompt and it is why repair works where retry does not. Keep the tail of long output rather than the beginning, because compiler and test output puts the real failure last.

Bound the attempts. Without a ceiling, a permanently broken gate will loop until morning, spending money to fail repeatedly.

Should I tell the agent what its gates are?

Disclosing the gate commands in the agent's instructions changes its behaviour for the better. It runs them itself and fixes problems before finishing, which is cheaper than a repair round.

We observed this directly. Given a role whose gate was go test ./... in a repository with a deliberately broken pre-existing assertion the prompt never mentioned, the agent ran the tests, found the failure, and corrected the wrong assertion before the gate ever executed. It fixed the assertion rather than deleting the test, which is the behaviour that matters.

Pair the disclosure with an explicit instruction not to weaken, skip or disable a check to make it pass. Gates are a safety net, not a guarantee, which is why verified work should still land somewhere reviewable such as a branch.

Method and limitations

The 21% success rate is one user's self-reported tally published in a third-party review of a hosted platform, not a controlled measurement.

Reddit engagement counts are from a reddit-mcp query against r/ClaudeAI over the past year, rounded, and are a snapshot. Quotes are attributed to the subreddit only.

The self-correction example is from our own instrumented test run, a single observation rather than a benchmark.

FAQ