≋ loomwell/ EVALUATIONS

FREE SAMPLE / THREE SYNTHETIC CASES

Inspect. Reconcile. Verify.

Submitted-output evaluation

Run your agent with these files in your own environment, then upload its JSON result. We evaluate the supplied artifact; we do not run your agent.

Download starter pack ↓View all task files

Task reconciliation-sample/1.0.0 · Grader submitted-output/1.0.0 · Expert review: not recorded

01 / Straightforward match

Match identical date, reference, and signed amount only when each side has exactly one candidate. Include every source row. All amounts are integer cents.

Inspect ledger & bank files
{
  "bank": [
    {
      "amount_cents": 50000,
      "date": "2026-09-20",
      "description": "Customer payment",
      "id": "B1",
      "reference": "INV-1042"
    }
  ],
  "currency": "USD",
  "id": "straightforward",
  "ledger": [
    {
      "amount_cents": 50000,
      "date": "2026-09-20",
      "description": "Customer payment",
      "id": "L1",
      "reference": "INV-1042"
    }
  ],
  "opening_cents": 100000,
  "policy": "Match identical date, reference, and signed amount only when each side has exactly one candidate. Include every source row. All amounts are integer cents.",
  "title": "01 / Straightforward match"
}

02 / Settlement with fees

Match uniquely by identical date/reference/signed amount. The bank shows the gross settlement and a separately identified processor fee (kind=fee). Classify that negative bank-only row bank_fee, with no counterpart, cite bank:B3 and add its signed amount to ledger closing. No other adjustment is supported.

Inspect ledger & bank files
{
  "bank": [
    {
      "amount_cents": 50000,
      "date": "2026-09-21",
      "description": "Gross card settlement",
      "id": "B1",
      "reference": "SET-004"
    },
    {
      "amount_cents": -20000,
      "date": "2026-09-21",
      "description": "Supplier payment",
      "id": "B2",
      "reference": "PAY-018"
    },
    {
      "amount_cents": -1250,
      "date": "2026-09-21",
      "description": "Processor settlement fee",
      "id": "B3",
      "kind": "fee",
      "reference": "FEE"
    }
  ],
  "currency": "USD",
  "id": "settlement",
  "ledger": [
    {
      "amount_cents": 50000,
      "date": "2026-09-21",
      "description": "Gross card settlement",
      "id": "L1",
      "reference": "SET-004"
    },
    {
      "amount_cents": -20000,
      "date": "2026-09-21",
      "description": "Supplier payment",
      "id": "L2",
      "reference": "PAY-018"
    }
  ],
  "opening_cents": 100000,
  "policy": "Match uniquely by identical date/reference/signed amount. The bank shows the gross settlement and a separately identified processor fee (kind=fee). Classify that negative bank-only row bank_fee, with no counterpart, cite bank:B3 and add its signed amount to ledger closing. No other adjustment is supported.",
  "title": "02 / Settlement with fees"
}

03 / Know when to escalate

Two ledger candidates have identical date/reference/amount and no event identity distinguishing them. Do not assume either is a duplicate. Classify ALL THREE competing rows ambiguous, leave counterparts empty, explain each exception, cite all three source IDs, use no balancing adjustments, and set needs_review. Preserve the unresolved difference.

Inspect ledger & bank files
{
  "bank": [
    {
      "amount_cents": -15000,
      "date": "2026-09-22",
      "description": "One cleared payment, no source linkage",
      "id": "B1",
      "reference": "PAY-UNKNOWN"
    }
  ],
  "currency": "USD",
  "id": "ambiguous",
  "ledger": [
    {
      "amount_cents": -15000,
      "date": "2026-09-22",
      "description": "Payment posting, event ID absent",
      "id": "L1",
      "reference": "PAY-UNKNOWN"
    },
    {
      "amount_cents": -15000,
      "date": "2026-09-22",
      "description": "Payment posting, event ID absent",
      "id": "L2",
      "reference": "PAY-UNKNOWN"
    }
  ],
  "opening_cents": 100000,
  "policy": "Two ledger candidates have identical date/reference/amount and no event identity distinguishing them. Do not assume either is a duplicate. Classify ALL THREE competing rows ambiguous, leave counterparts empty, explain each exception, cite all three source IDs, use no balancing adjustments, and set needs_review. Preserve the unresolved difference.",
  "title": "03 / Know when to escalate"
}

One JSON file. Exact findings.

Use submission-template.json from the starter pack. Supply all three cases, every source row, numeric summaries, and exception evidence.

Read the full submission contract
# Loomwell submitted-output starter pack

All inputs are authored synthetic USD transactions. No signup is required to inspect them.
Upload one UTF-8 JSON file, at most 200 KB, with exactly these top-level keys:
`schema_version: "loomwell-submission/1"`, `task_version: "reconciliation-sample/1.0.0"`, `cases: [...]`.
Use submission-template.json; it is intentionally incomplete, not a solved result.

Provide exactly the three case IDs: straightforward, settlement, ambiguous.
Each case: case_id, rows, summary, status, evidence_ids.
Each rows entry has exactly: source (ledger/bank), id, counterpart_id (empty string
when unresolved), classification (matched/bank_fee/ambiguous), amount_cents
(integer copied from source), explanation (nonempty for every exception).
Include every source row exactly once. Never choose an arbitrary ambiguous match.
status: reconciled for resolved cases; needs_review for ambiguity.
evidence_ids: exactly the exception source IDs, e.g. ["bank:B3"]. For ambiguity,
cite ALL competing records as source:id; IDs are scoped per case.
summary: nine integer-cent fields: opening, ledger_ending, bank_ending,
bank_fee_adjustment, duplicate_adjustment, outstanding_adjustment, adjusted_ledger,
adjusted_bank, difference. Closing = opening + signed source rows. Add bank fees
to ledger; other adjustments are zero in this sample. difference = adjusted_ledger
minus adjusted_bank. An unresolved difference is permitted only with needs_review.

This adapter reuses loomwell.grader.grade_tables for coverage, amounts, matches,
classifications and totals. It adds close-status and source-evidence checks.
It does not grade formulas or run agent code. Public cases and feedback may be
used for debugging; success here is not a held-out benchmark or finance accuracy score.

No formula, live tool-use, latency, integration, or production reliability checks. No general finance accuracy score.

SUBMIT & SAVE

What did your agent produce?

Your draft stays in this browser tab through sign-in. Only submit synthetic sample outputs; drafts are cleared after successful submission.