looptape logo, a project by Chris Zemmel: a cassette whose reels are a pair of quotation marks

Free and open source (MIT)

Live

looptape

Agents fail in ways a green build never catches. looptape records a real session once, replays it into your own harness in place of the model, with no model and no network, and checks whether your code let a lie through.

At a glance

checks for the ways agent loops lie
37
runtime dependencies
0
APIs it records: Anthropic and OpenAI
2
free, published on npm
MIT

Previews

zsh: npx looptape check tapes
looptape, Checking the seed tapes: phone view
Checking the seed tapes. npx looptape check over the 24 seed tapes in the package: every lie a tape holds, named by the check that caught it.

The real output of looptape 0.1.0 from npm, run over the seed tapes that ship with it, set in a terminal window.

Overview

Agents fail in ways a green build never catches: a final answer that promises work nothing will ever do, an invented "I can't access that" with no tool tried, "saved a memory" when no memory was written, a count quoted from memory instead of from the listing that was just made, a list cut short without anyone being told.

None of those are model bugs; they are bugs in the loop around the model. You cannot test that against a live model, because it will not say the same thing twice. You can test it against what the model already said, and that is what a tape holds.

Process

  1. Step 1: Record

    Point your SDK at the recording proxy for the Anthropic or OpenAI API and run your agent as usual. Every completion, tool call and result is written to a tape, exactly as the model saw it.

  2. Step 2: Check

    37 checks read what actually happened: a promise with nothing behind it, an offer instead of an action, an invented inability, a claim with no evidence, a result shortened without saying so.

  3. Step 3: Replay

    A deterministic player feeds the tape into your real harness in place of the model, with no model and no network. When the harness asks for something the tape never recorded, the test fails instead of improvising.

  4. Step 4: Prove the fix

    Fixture first, then the fix, then the fixture proves the fix. The predicates behind the checks are exported, so a loop can reject a bad reply while it runs, with the same definition the tape check uses afterwards.

By design

  • Every check from a real incident. The origin is kept on the check itself, because a check whose origin nobody remembers gets deleted the first time it is inconvenient.

  • Tapes say where they came from. Every tape is marked as recorded (a byte-exact capture), reconstructed (rebuilt from a real incident) or synthetic (it never happened).

  • No key, no network. Zero dependencies, and nothing in it needs an API key: a replay runs with no model and no network.

Honest status

Live

Feature-complete, free and MIT-licensed, and published on npm: npm install -D looptape.

Try it on your agent

Run npm install -D looptape, then npx looptape init writes a starter tape so you can see what a finding looks like.

Open on GitHub(opens in a new tab)

Live at github.com/chriszemmel/looptape. Questions go to chris@zyric.de.

looptape · Zyric