TurnScope

Evidence for your next improvement

Improve your agent.
Check the outcome.

Record the calls, cache charges, and task results behind a change. Compare the whole study, including failed and missing trials.

Compare two runs

Local trace import. Structured experiments. No model connection.

The view across turnsSYNTHETIC
Synthetic input rises through turn four, then falls. This is an illustration, not a benchmark. TURN 01TURN 06
Repetition is a signal.

Its purpose, cache status, and cost need context.

01Import a runOTLP or your recorded counts

02Test one changeEvidence and required checks

03Compare outcomesKeep failures in view

Start with a trace

Bring your run into focus.

JSON · version 1
Paste a trace or choose a file2 MiB max
Add your token rates optional

Use your own rates per million tokens. Pricing already in the JSON is preserved unless you override it here.

Manual pricing

Your trace is sent to this service only when you choose Analyze. It is processed in memory, with no persistent storage, content logging, or external model calls.

Before you adopt the change

Did the whole run improve?

Comparison schema ↗

Compare the same tasks and trials, with the same evaluator and accounting scope. Fewer tokens alone cannot tell you whether the agent still did its job.

Try a published result

Three task pairs from the authors’ SWE-agent study. These are external observations, not TurnScope benchmark results.

Source data & limitations ↗

Use the local client’s measure command to create records after running your evaluator. Outcomes are supplied by you. Files stay local until you choose Compare; nothing is stored by this service.

Complete the evidence · new in 0.3

What did the whole run cost?

Recorder & study guide ↗

Cache writes, helper models, and retries belong in the bill. Missing trials belong in the study. Inspect both here using metadata recorded by your agent.

Try a small example

These fixtures use invented counts and prices to demonstrate the checks. Loading one stays in your browser until you choose Inspect evidence.

JSON Schema ↗

Only submit prepared metadata. The local recorder hashes content and excludes raw prompts and tool arguments. This editor sends your JSON when you choose Inspect evidence; it does not redact it for you. Requests are processed in memory. 2 MiB maximum.

For the agent doing the work

Record. Account.
Compare.

Normalize telemetry locally or record events with the Python helper. Declare expected trials before the experiment, retain failures, and submit your evaluator’s results.

Base CLI ↓Evidence CLI ↓Evidence guide ↗Agent guide ↗OpenAPI ↗API manifest ↗Research ↗
python3 turnscope.py normalize trace.json --format genai --output imported.json
python3 turnscope.py analyze imported.json --server "$TURNSCOPE_BASE_URL" --output report.json
POST /api/v1/analyze
curl "$TURNSCOPE_BASE_URL/api/v1/analyze" \
  -H 'Content-Type: application/json' \
  --data-binary @run.json

Fingerprints and reported counts are enough.
Raw prompts and documents are optional.