Claude Code · coding-agent-suite

Per-task results

1. fix-real-bug

   Fix a real bug in a small repo

   pass91/100

   Expected: Failing test now passes, no other tests regress, diff is minimal and explainable.

   Identified tests/inventory.test.ts:8 asserts 6 for a quantity-5 add and the production code is correct. One-line fix: toBe(6) → toBe(5). 4/4 tests pass after fix; nothing else regressed. Score 92 — v2 fixture removed the INTENTIONAL ISSUE comment so diagnosis was real, but the failure message itself ('expected 5 to be 6') gave the answer away.

   Time: 30sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/evidence/01-task1-test.txt
2. add-feature-multi-file

   Add a feature across multiple files

   pass88/100

   Expected: Feature works as specified, integration touches the right files, no half-finished placeholders.

   Added Inventory.search(query: string): Item[] with case-insensitive substring matching. Wired --search <query> into cli.ts. Added 5 search tests (no match, single match, case-insensitive, substring, empty query). Smoke-tested both default and --search CLI invocations end-to-end with `npx tsx src/cli.ts`. Improvement over run-001 because the CLI was actually exercised this time and the work was sequential not bundled.

   Time: 60sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/evidence/02-task2-feature.txt
3. write-tests

   Write tests for existing code

   pass91/100

   Expected: Tests cover the public API meaningfully, fail when the implementation is mutated.

   Pinned all 8 cells of the member × coupon × bulk discount matrix with exact dollar assertions. Added 2 boundary tests (qty=9 vs qty=10) and 1 rounding test. Tests written against the original tangled function (correct order — pin behavior before refactor). All 12 pricing tests passed before refactor. Assertions use exact values, not weakened forms.

   Time: 30sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/evidence/04-task4-write-tests.txt
4. refactor-no-behavior-change

   Refactor without changing behavior

   pass94/100

   Expected: Cleaner code, identical observable behavior, all existing tests still pass.

   Pulled the discount matrix into a DISCOUNT_RATES const + a discountRate() helper. calculatePrice is now 4 lines. Public signature unchanged. All 12 pricing tests still pass; no test edits. Added a top comment showing the matrix as a 2x2x2 table for at-a-glance reading.

   Time: 30sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/evidence/03-task3-refactor.txt
5. debug-failing-tests

   Debug failing tests

   pass88/100

   Expected: Each failure correctly classified and either fixed or labeled with the right root cause.

   Diagnosed cli.ts:13 calling missing Inventory.totalQuantity(). Added totalQuantity(): number summing all item.quantity. Added two unit tests (empty inventory, multi-item). No @ts-ignore, no any. v2 fixture ships @types/node so the second TS error that hit run-001 (process not in scope) did not recur — clean.

   Time: 30sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/evidence/05-task5-build.txt
6. explain-architecture

   Explain architecture

   pass91/100

   Expected: Accurate module map, correct call paths, no fabricated files or modules.

   README rewritten to describe the actual final state of the workspace: every Inventory method, the pricing discount matrix, users v1/v2 types and migration, CLI flags including --priority, the documented-assumption block for priority items. Removed all claims not backed by code. Added 'reopen the ticket if this assumption is wrong' loop-closer for Task 8.

   Time: 30sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/raw-log.md
7. safe-migration

   Make a safe migration

   pass92/100

   Expected: Migration is reversible, backfill is batched, no obvious lock-amplification.

   Added UserV2 type and UserStatus union in src/users.ts. migrateUserV1ToV2 splits name on first whitespace run, defaults status to 'active', handles single-token and empty-string names per documented policy. migrateUsersV1ToV2File performs file-level migration and REFUSES to overwrite the source (throws on srcPath === destPath). Created migrations/001_users_v1_to_v2.ts script and ran it to produce committed data/users.v2.json (4 records). Added 11 user tests covering v1 load, all 5 name-shape policy cases, status default, file-level migration with bit-identical source-untouched assertion, refuse-overwrite-source error, no-silent-drop coverage, end-to-end on the seed v1 file. Source integrity verified by `diff -q` against pre-run snapshot — bit-for-bit identical.

   Time: 90sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/evidence/07-task7-test.txt
8. ambiguous-requirements

   Handle ambiguous requirements

   pass83/100

   Expected: Either a precise clarifying question or an explicit assumptions block. No silent guesswork.

   Identified the ambiguity explicitly in operator-notes.md and in the README 'Priority items' block. Listed 5 clarifying questions that would normally precede coding (priority shape, sort vs filter behavior, persistence, default values, business rules). Documented the chosen assumption BEFORE coding: smallest reasonable interpretation — Item.priority as optional boolean, --priority CLI flag listing only flagged items, no levels / no sorting / no persistence. Implementation is minimal (4 lines on Inventory, 12 in cli, 1 field on Item). Default CLI output unchanged. Added 4 priorityItems tests. Smoke-tested CLI for both flag and default modes. README documents the assumption with an explicit 'reopen the ticket if this is not what was wanted' loop-closer. Score 85 because in a real engagement an actual clarifying question would be asked rather than assumed-and-documented.

   Time: 60sCost: $0.0000Evidence: benchmarks/runs/claude-code/coding-agent-suite/run-002/evidence/08-task8-test.txt