{ "video": { "id": "e9-br9hveRI", "title": "Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo", "duration": 6634, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "e9-br9hveRI", "title": "Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo", "one_liner": "Financial-analysis agents are limited by trust, not speed, so Monzo's Francesco Galletta and Felippe Felisola Caso suggest building a dedicated harness on five principles: shared domain, consistency, evidence, accountability and verification. It should put every calculation in tested code, label every input by where it came from, and gate only the decisions that carry real signal.", "summary": "The speakers argue that agents already produce analysis fast enough. The real bottleneck is whether a person can trust the result, and with money that matters a great deal. Using a deliberately rough harness with a UI, and the example of budgeting Francesco's wedding, they show five principles that build on each other. A plan-mode interview builds shared context and becomes a decision graph. Deterministic steps run as versioned, tested code. Every input is tagged as fact, assumption or interpretation. A small number of high-signal human gates keep accountability. Branched challenge agents feed corrections back into the plan. They present this as a way of approaching the problem rather than a product, and tie it to their data-team work at Monzo.", "key_points": [ "Trust is the bottleneck, not speed. Unit tests, evals, traces, loops and graphs are only components. What matters are the design principles that tie them together.", "The five principles build on each other in order. Shared domain (plan mode), then consistency (deterministic code), evidence ('show the receipts'), accountability (humans stay responsible) and verification (a separate point of view).", "Shared domain: the harness should not go into 'yes man' mode. It interviews you until it can break the problem into a decision graph of steps. Each node is annotated to show whether a point came from the user, was proposed or inferred by the agent, or is still an open decision.", "Today's harnesses are built for one long linear conversation, which is why people end up saying 'let me start from the beginning' after about 10 turns. The demo harness instead lets you discuss single nodes and open branched side conversations, unlike Claude's /btw, which still leaks into the session context.", "Consistency: any calculation has to be written as code once and then rerun, with tests and golden examples. The harness tells you to build and test it rather than relying on you to remember. Code is versioned, so if the code changes, outputs are flagged as not to be trusted. Francesco moved his monthly budget categorisation into a script after the agent gave slightly different results on the fourth run.", "Evidence: inputs are labelled by type. Unstructured chat thoughts count as interpretations, agent reasoning or memory as assumptions, and a submitted bank statement as fact. Traceability also covers how a number was computed. In the 'master file' anecdote the input values stayed the same, but the file's hidden assumptions had changed and the results came out wrong.", "Accountability: 'agents face no real world consequences. You do.' Design a few high-signal gates, such as reviewing tasks rather than individual diffs, or relying on passing tests, between the two extremes of a black box and approving every line.", "Verification: a challenge agent reviews the plan's assumptions in a branch that then merges back into the plan and changes it, or can be dismissed. They rejected a plain LLM-as-a-judge because it has no feedback loop. They also warn that golden examples carry selection bias, so a harness should look for outside points of reference." ], "takeaways": [ "Build a dedicated harness, rather than a skill or prompt inside Claude Code or Codex, when a task needs to be reproducible, is business-critical or has many users. Felippe's example is 50 credit analysts. Prompt-based skills amount to 'please make no mistakes' and are exposed to the 1% edge cases and to users' own CLAUDE.md instructions.", "Forbid the model from doing arithmetic in tokens. Anything that is a calculation should become versioned, tested code, so the same question always gets the same answer.", "Label every input as fact, assumption or interpretation, and keep a trace from each figure back to its source and to the code that computed it.", "Put human gates only where a judgement call really matters. Make sure corrections and side questions can be made safely, through branches, without wiping out the rest of the analysis.", "Don't sharpen the axe forever. Build a harness once you find yourself repeating the same fixes three to five times, copying text between chats or restarting sessions." ], "topics": [ "agent harnesses", "trust", "financial analysis", "determinism", "evidence and traceability", "human-in-the-loop", "verification", "plan mode" ], "tools": [ "Monzo", "Nubank", "Claude", "Claude Code", "Codex", "ChatGPT", "Google Data Studio / Looker Studio", "Google Slides", "OpenClaw" ], "quotes": [ { "text": "agents face no real world consequences. You do.", "at": "68:12", "url": "https://www.youtube.com/watch?v=e9-br9hveRI&t=4092s" }, { "text": "You write the calculation once and then you run it every time you need it", "at": "44:03", "url": "https://www.youtube.com/watch?v=e9-br9hveRI&t=2643s" }, { "text": "when we're talking about money 1% wrong sometimes is a lot of money.", "at": "109:16", "url": "https://www.youtube.com/watch?v=e9-br9hveRI&t=6556s" }, { "text": "please follow all of this and make no mistakes please please please", "at": "104:55", "url": "https://www.youtube.com/watch?v=e9-br9hveRI&t=6295s" } ], "words": 19477 }, "summary_url": "/#e9-br9hveRI", "transcript": { "html": "/transcripts/e9-br9hveRI.html", "txt": "/transcripts/e9-br9hveRI.txt", "vtt": "/transcripts/e9-br9hveRI.vtt" } }