Verification gates

Why gates exist

The most common complaint about autonomous agent platforms is that they mark work complete when it is not. An agent reports success, the task moves to done, and nobody notices until a customer does. Gates are the answer: the work is only accepted if something independent of the agent confirms it.

Gates for code

gates:
  - run: npm test
  - run: npm run lint
  - run: npx tsc --noEmit

Prefer the project's real commands. They are stronger evidence than anything generic, and they are already the standard the rest of your work is held to.

Gates for content and non-code work

A non-code role still needs a gate. The shape differs: instead of a test suite, prove that output exists, has substance, and is not full of placeholders.

# The file exists and is not empty
- run: test -s .nightshift/content/post.md

# It has real substance
- run: test $(wc -w < .nightshift/content/post.md) -ge 400

# Nothing was left half-written
- run: '! grep -rniE "lorem ipsum|TBD|TODO" .nightshift/content'

# Research was appended today, not just repeated
- run: grep -rq "$(date +%Y-%m-%d)" .nightshift/research

# A draft landed where the human will look for it
- run: test -n "$(ls -A .nightshift/outbox)"

Gate options

A gate can be a bare string, or a mapping when you need more control.

gates:
  # Terse form
  - "npm test"

  # A gate that must FAIL for the work to be accepted, for example
  # "there must be no TODOs left"
  - run: "grep -rn TODO src/"
    expect_exit: 1

  # Exit zero is not enough; the output has to prove it
  - run: "curl -s localhost:3000/health"
    expect_stdout: "ok"

  # A slow gate with its own budget
  - run: "npm run e2e"
    timeout: 15m

• expect_exit sets the required exit code. Defaults to 0.
• expect_stdout requires that text to appear in the output.
• timeout bounds the gate. It inherits the role's timeout when unset.
• Gates run in order and stop at the first failure, so a later gate never wastes time after an earlier one fails.
• A timeout is always a failure: a gate that cannot finish proves nothing.

The repair loop

1. 1

   Gates run after the agent finishes

   They execute in the role's working directory, which is its isolated checkout when isolation is on.

2. 2

   A failure is turned into feedback

   The failing command, the reason it failed and the tail of its output are handed back to the agent in the same session.

3. 3

   The agent gets a bounded number of attempts

   max_repair_attempts caps this, so a permanently broken gate cannot loop all night.

4. 4

   Gates re-run after each repair

   If they pass, the cycle succeeds. If the attempts are exhausted, the cycle fails and the work does not land.

Gates that do not work

# Always passes. Proves nothing, but looks like proof.
true
echo done
git status
git diff --stat        # exits 0 whether or not anything changed

# Better: prove something actually happened
test -n "$(git status --porcelain)"