{
  "video": {
    "id": "gdzsJZJ6DDs",
    "title": "Your Financial Agent Needs to Show Its Work - Anirudh Veeraragavan, Fieldguide",
    "duration": 808,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "gdzsJZJ6DDs",
    "title": "Your Financial Agent Needs to Show Its Work - Anirudh Veeraragavan, Fieldguide",
    "one_liner": "In audit, the opinion is the product and is only as good as its basis, so a financial agent has to be defensible to 'an experienced auditor new to the work'. That takes four controls together: state-aware authorization, recorded execution steps, citations as content-hashed identities, and an audited eval set. Escalations should be easy to verify but not easy to approve.",
    "summary": "Anirudh Veeraragavan, lead security engineer at Fieldguide, argues that what matters for a financial agent is less whether it is right and more whether it can show the decision-making process behind each opinion. When an audit is picked for inspection and someone asks 'why was this sufficient', evals, metrics and happy users are no answer. He works through a bank reconciliation example (a fictional freight-lines client: $2.9M per the bank, $2.7M per the books, 41 reconciling items) to show four controls: authorization, execution control, evidence provenance and evaluation integrity. All four rest on knowing when the agent should stop and a human should decide. He calls defensibility 'an and', meaning you need every control, and says the same pattern carries over to credit decisions and AML alerts.",
    "key_points": [
      "The standard is borrowed from the audit industry: an experienced auditor who hasn't done the work should be able to understand the test, the evidence and the decision. Adapted for agents, 'an experienced auditor new to the work can follow the agent.'",
      "Peregrine Financial Group is the motivating case. For over 20 years it claimed more than $200M on paper while the bank held only $5M, and when the fraud came to light about $200M vanished. Being wrong in finance is costly.",
      "Authorization: a reviewer flags a check, later context triggers an agent run, and the agent rewrites the row to 'no exception'. Its conclusion was correct, but what the human signed off no longer matches what they reviewed. Authorization has to be 'is this tool call allowed for this object given this state at this point in time'. The agent may draft unreviewed rows as itself but must never overwrite human-reviewed work.",
      "Execution control: rerunning a year or two later can give a different result, and ordinary logs can't reconstruct why. A run should be a chain of recorded steps, each with an ID, inputs and outputs, and each marked as deterministic (code fetches or matches) or agentic (extracting a PDF field, reaching the conclusion). Then individual steps can be rerun and runs compared to find the exact step that diverged.",
      "Evidence provenance: a citation like 'January bank statement page two' is useless, because many documents can share that name. A citation should be an identity: a specific document ID, a version and a hash of the contents. That makes rules enforceable, such as accepting only evidence sourced directly from the bank, not client uploads that could be tampered with.",
      "Evaluation integrity: gold data built from last year's reviewed audits can be wrong, through superseded decisions, decisions made on bad evidence, or methodology changes. In the example the score is 97%, but four of nine misses were the eval being wrong, while 12 'no exception' labels are now wrong because of business-rule or methodology changes. The headline number hides whether the agent is doing better or worse than expected.",
      "Escalate to a human when the agent would change human-reviewed work, when a new run disagrees with a previous one, or when evidence breaks the evidence rules. Because of automation bias, flag only exceptional cases and state the discrepancy clearly, for example a deposit that took 11 business days to clear. He frames it as the agent handling about 80% of cases and humans the 20% where it counts.",
      "Defensibility is 'an and'. Skip authorization and answers change under reviewers. Skip execution control and you can't show how you got the right answer ('may as well attribute it to luck'). Skip provenance and you can't trace a wrong answer. Skip eval integrity and 97% tells you nothing."
    ],
    "takeaways": [
      "Make authorization depend on object state and time, not just on the tool. Let agents draft unreviewed items as suggestions under their own identity, and block them from overwriting anything a human has reviewed and signed.",
      "Record every agent run as a chain of step records with IDs, inputs and outputs, with deterministic and LLM steps marked apart, so you can rerun single steps and diff old runs against new ones.",
      "Store citations as identities (document ID, version, content hash, uploader and source) rather than strings, and enforce provenance rules such as bank-sourced evidence only.",
      "Audit your gold eval set for superseded labels, bad evidence and methodology changes, and break down headline scores by why each miss happened.",
      "Escalate only real exceptions (overwrites of reviewed work, disagreement between runs, evidence that breaks the rules), each with a clear statement of what is wrong, so humans don't rubber-stamp out of automation bias."
    ],
    "topics": [
      "agents",
      "auditability",
      "financial audit",
      "evals",
      "provenance",
      "human-in-the-loop",
      "authorization",
      "compliance"
    ],
    "tools": [
      "Fieldguide",
      "Peregrine Financial Group"
    ],
    "quotes": [
      {
        "text": "So in audit the opinion is the product and opinion is only as good as the basis behind it.",
        "at": "00:18",
        "url": "https://www.youtube.com/watch?v=gdzsJZJ6DDs&t=18s"
      },
      {
        "text": "an experienced auditor new to the work can follow the agent.",
        "at": "03:27",
        "url": "https://www.youtube.com/watch?v=gdzsJZJ6DDs&t=207s"
      },
      {
        "text": "these escalations, they should be easy to verify, but not necessarily easy to approve.",
        "at": "10:34",
        "url": "https://www.youtube.com/watch?v=gdzsJZJ6DDs&t=634s"
      },
      {
        "text": "And if your if your defense is the model said so that's not really going to cut it in today's day and world.",
        "at": "12:58",
        "url": "https://www.youtube.com/watch?v=gdzsJZJ6DDs&t=778s"
      }
    ],
    "words": 2953
  },
  "summary_url": "/#gdzsJZJ6DDs",
  "transcript": {
    "html": "/transcripts/gdzsJZJ6DDs.html",
    "txt": "/transcripts/gdzsJZJ6DDs.txt",
    "vtt": "/transcripts/gdzsJZJ6DDs.vtt"
  }
}