{ "video": { "id": "8g0g1ncwzG0", "title": "Reliable, Smart Workflows for Finance, Wealth and Insurance — Karthic Rao, Perpetual", "duration": 921, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "8g0g1ncwzG0", "title": "Reliable, Smart Workflows for Finance, Wealth and Insurance — Karthic Rao, Perpetual", "one_liner": "In regulated finance work, don't ask for more agency or less. Ask where the boundary sits: determinism governs control flow, evidence and side effects, and the LLM only answers small typed judgment questions. Built that way, about 95% of model calls can run on a small flash-tier model, possibly on private hardware.", "summary": "Karthik, CTO of the wealth-management rollup Perpetual, argues that reliability and intelligence are not opposites. The design work is deciding where the boundary between them sits. Using the one-sentence request \"Fill out the IRS W9 for Mr. Harper\", he shows how it becomes a process that runs for days, with ambiguity, missing data, client waits, follow-ups, e-signature and an evidence trail. He then walks through Perpetual's engine: an LLM planner that only translates the request into a typed semantic graph; a deterministic 11-stage compiler that checks it against 42 typed action contracts and fails closed; and a state machine that runs the approved, hash-pinned plan. That state machine makes fixed-schema model calls only at judgment points, so most calls can go to a small model and eventually to privately hosted ones.", "key_points": [ "Four facts of finance/wealth workflows: they are complex and need runtime strategies; the data is sensitive, so you need a path to fully privately hosted models; a policy violation has severe consequences, not just a bug report; so execution must be highly reliable and deterministic. Use intelligence only where runtime judgment is genuinely required.", "\"Token maxing is the wrong goal\" in regulated industries. These tasks are close to a closed domain, so you can lay them out as a massive state machine that needs only \"a sprinkle of intelligence\". Most steps just follow the next edge, and the model picks a branch only at a few judgment points.", "The W9 example: there are three Harper households, so a human has to be asked which one. Every form is \"a small program\": W8 instead of W9 for foreign persons, exactly one box on line 3a, SSN or EIN but never both and taken from a verified document, no exemption codes for S corps, a signature under penalty of perjury. Missing values trigger a client email chase with a 2-day follow-up, partial replies get chased again, and values are written back to the system of record along with how they were obtained.", "Both common approaches fail. Fixed-flowchart automation breaks at the first 'which Harper', and a free-improvising agent is something a compliance team can't sign off on. 'More agency' and 'more rules' are both the wrong axis.", "Architecture: an LLM planner emits a typed semantic graph (intents, entities, facts, artifacts, events, constraints), with every node grounded in the words of the request. A compiler that never calls a model runs 11 deterministic stages to produce an execution DAG, with only three verdicts: valid, needs clarification, or refused. There are about 40 fail-closed conditions, and the plan is immutable and pinned by its hash.", "A grammar of 42 typed action contracts (forms, documents, email, e-signature, calendar, human steps), with the contract digest pinned identically in the engine, back end and UI. The grammar inserts required steps the planner never asked for, such as rendering the PDF, attaching it and judging the reply.", "Gates and waits are nodes in the plan, not prompts. A reply wait defaults to 72 hours, and explicit deadlines can run from 1 minute to 30 days. When a deadline passes, a rule decides what happens, not a model. One approval covers the plan, and each external action has its own gate, approved against a hash of the exact payload and re-checked right before sending. A partial reply judgment is provisional until a person confirms it. Follow-ups run as child plans and never modify the approved parent, with at most 5 rounds. Side effects go through prepare, dispatching, then succeeded, or outcome unknown leading to manual review. There are no automatic retries. Chat can approve a step but can't change an approved plan.", "In an illustrative run of 20 decisions, 19 went to the flash tier and 1 to a frontier model, which is why they designed for about 95% of decisions never needing a frontier model. Frontier decisions that a person confirms or rejects become eval-grade training data for post-training the flash tier. Logging and evals exist today; the training loop is next. Because every call is small and schema-checked, moving to a privately hosted model is a one-line configuration change." ], "takeaways": [ "Design the boundary explicitly: keep control flow, evidence and side effects deterministic, and limit the LLM to interpretation and judgment through fixed-schema, typed calls whose answers the state machine acts on.", "Make the LLM planner a translator only. Validate its output with a deterministic compiler against a closed grammar of typed action contracts that fails closed, so the model can only ask for things the firm already knows how to do safely.", "Turn waits, deadlines, approvals and follow-ups into plan nodes governed by rules. Gate every external side effect behind human approval tied to a hash of the exact payload, and send unclear outcomes to manual review instead of retrying automatically.", "Route small typed judgments to a cheap flash-tier model and log every judgment along with the human confirmations, so frontier decisions become training data that shrinks the frontier share release over release.", "Record provenance (how a value was obtained) alongside the value itself. In regulated work, how you know a value matters as much as the value." ], "topics": [ "agents", "workflow orchestration", "regulated industries", "determinism", "human-in-the-loop", "fintech", "local models", "model routing" ], "tools": [ "Perpetual", "IRS Form W-9", "IRS Form W-8" ], "quotes": [ { "text": "But in a regulated industry, token maxing is the wrong goal.", "at": "01:53", "url": "https://www.youtube.com/watch?v=8g0g1ncwzG0&t=113s" }, { "text": "In this industry, how you know a value matters as much as the value itself.", "at": "04:55", "url": "https://www.youtube.com/watch?v=8g0g1ncwzG0&t=295s" }, { "text": "The model steers. It never drives.", "at": "12:26", "url": "https://www.youtube.com/watch?v=8g0g1ncwzG0&t=746s" }, { "text": "don't ask whether your system should have more agency or less. Ask where the boundary is.", "at": "14:45", "url": "https://www.youtube.com/watch?v=8g0g1ncwzG0&t=885s" } ], "words": 2584 }, "summary_url": "/#8g0g1ncwzG0", "transcript": { "html": "/transcripts/8g0g1ncwzG0.html", "txt": "/transcripts/8g0g1ncwzG0.txt", "vtt": "/transcripts/8g0g1ncwzG0.vtt" } }