Reliable, Smart Workflows for Finance, Wealth and Insurance — Karthic Rao, Perpetual
https://www.youtube.com/watch?v=8g0g1ncwzG0
AI Engineer · 15 min · 187 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:00] Hello, I'm Karthik, CTO at Perpetual.
[00:03] In this presentation, I'll talk about building workflows that are both reliable and intelligent for finance, wealth, and insurance.
[00:10] In regulated domains, workflows must be reliable enough to meet regulatory standards while still being intelligent enough to handle the real world.
[00:17] In fields such as finance and wealth management, this presents both a technical and a compliance issue.
[00:22] Reliability and intelligence are not opposites.
[00:25] It is up to you where you draw the boundary between them and that is the point of this talk deciding where that boundary lies.
[00:32] For us, it's not merely a thought experiment but a question that surfaces serious consequences.
[00:37] Perpetual is a wealth management rollup we acquire running traditional wealth management firms and we maximize their throughput and revenue with the help of AI.
[00:47] Wealth management is a regulated industry and that is where we build with AI in production.
[00:53] Everything in this talk is grounded in building for those constraints.
[00:57] Workflows in finance and wealth management come with four facts and each of them pushes the architecture in the same direction.
[01:04] First, they're complex and they need runtime strategies as the process unfolds.
[01:09] You can't script every path in advance.
[01:11] Second, the data is sensitive, so you need a path to move these workflows entirely onto privately hosted models.
[01:18] Third, in a highly regulated industry, a policy violation isn't a bug report.
[01:23] It has severe consequences.
[01:24] So, fourth, execution has to be highly reliable and deterministic.
[01:29] Inject intelligence only where runtime judgment is genuinely required and minimize reliance on LLM everywhere else.
[01:38] AI is spreading through regulated industries, but there's still no consensus on an architecture that gives you high reliability with the least use of LLM intelligence.
[01:48] Meanwhile, most of what developers hear from the big labs is how to burn more tokens.
[01:53] But in a regulated industry, token maxing is the wrong goal.
[01:56] These tasks are close to main.
[01:58] You can lay them out in advance and build them into a massive state machine.
[02:02] And that machine needs only a sprinkle of intelligence to navigate it.
[02:06] Most steps just follow the next edge.
[02:08] Only at a few judgment points does the model pick a branch.
[02:12] So the intelligent way to build is to use intelligence minimally and on purpose.
[02:19] Let's make this concrete with a real wealth management workflow.
[02:23] A wealth adviser turns to their assistant and says one sentence.
[02:27] Fill out the IRS W9 for Mr. Harper.
[02:30] To a person, that's a simple instruction.
[02:33] Watch what it actually triggers when a human being executes it.
[02:36] The task has complexities that go beyond what meets the eye.
[02:40] First, the assistant finds form W9 from the firm's records or from the IRS website.
[02:45] Then, they look for Mr. Harper in the system of record and find three Harper households.
[02:50] So, they call the adviser, which Harper do you mean?
[02:53] Only once that ambiguity is resolved do they move on to the firm.
[02:57] A lookup, an ambiguity, and a human in the loop all in the first minute of a oneline task.
[03:08] Filling a form sounds like data entry.
[03:10] It isn't.
[03:11] Every form is a small program.
[03:14] A W9 first decides who the taxpayer is, the person or the owner if it's a disregarded entity.
[03:21] A foreign person stops right here because they need a W8 instead.
[03:25] Then how they're taxed.
[03:27] Exactly one box on line 3 A and an LLC adds its letter.
[03:32] That decides which name goes on line one and which on line two.
[03:36] Then whose number?
[03:37] An SSN or an EIN?
[03:39] Never both.
[03:40] Taken from a verified document, never from memory.
[03:43] Exemption codes apply only to some entities and never to an S corporation.
[03:48] And finally, who signs under penalties of perjury?
[03:51] Each answer changes the next question.
[03:54] And the people involved only ever see three kinds of gates.
[03:57] A question, a document to collect, or a signature.
[04:04] Now they start filling the form and realize the system of record is missing three of the values it needs.
[04:11] They check the documents on file and the system of record again still missing.
[04:15] So they start an email thread with a client asking for exactly those three values.
[04:20] The task now depends on someone outside the firm on their timeline and now the workflow waits.
[04:31] The request is with the client and all three values are still missing.
[04:35] If the client doesn't reply within 2 days, the assistant follows up and brings it to the adviser at the daily standup.
[04:41] If the client replies with only two of the three values, they follow up again for the missing one.
[04:46] When all three arrive, they complete the form and write the new information back to the system of record, including the trail of how they obtained those values.
[04:54] In this industry, how you know a value matters as much as the value itself.
[05:07] Once the form is filled, it goes out for e- signature.
[05:10] If the client doesn't sign within a day or two, the assistant nudges them again.
[05:15] Once it's signed, they file it for the client, send the acknowledgement and update the adviser at the next day standup.
[05:22] One sentence from the adviser became a process that runs for days, crosses several systems, waits on two people, and loops back on itself more than once.
[05:44] Look at what's inside that one sentence.
[05:47] Ambiguity to resolve, data to gather, approvals, weights measured in days, follow-up loops, and an evidence trail at the end.
[06:03] Workflow automation tools exist, but they rarely handle this real world complexity and ambiguity.
[06:09] A fixed flowchart breaks at the first witch harper and the lack of a native system of record with an integration and an agent that improvises its way through is exactly what a compliance team can't sign off on.
[06:21] One breaks on ambiguity, the other can't be signed off.
[06:24] Neither handles what's inside that sentence.
[06:27] So what do we do about it?
[06:29] The fashionable framing is more agency and better models.
[06:33] Give agent the whole task and wait for the next model to make it reliable.
[06:38] The reflexive answer, especially inside regulated firms, is the opposite.
[06:43] Less agency, more rules, lock it down and hardcode every path.
[06:48] Both are the wrong axis.
[06:51] In regulated verticals, reliability comes from where you place the boundary between the two, between the parts that must be deterministic and the parts that genuinely need judgment.
[07:01] Here's the principle.
[07:03] Determinism governs control flow, evidence, and side effects.
[07:07] what runs next, what we know and how we know it.
[07:10] And anything that changes the world, like a record written, an email sent, or a form filed, intelligence provides interpretation and judgment, reading and ambiguous instruction, deciding which Harper, understanding a client's reply, the model answers a type question,
[07:33] and the deterministic system decides what to do with the answer.
[07:36] And the interplay is recursive.
[07:38] A judgment can open a new smaller plan that's compiled, gated, and deterministic.
[07:44] Again, you'll see that in a moment with follow-ups.
[07:47] The boundary between the two is the design.
[07:49] Here's how that works in the workflow engine we've built.
[08:01] The request goes to a planner that is a model call and its only job is translation.
[08:07] It emits a type semantic graph of intents, entities, facts, artifacts, events, and constraints.
[08:14] Each grounded in the words of the request with any ambiguities it found.
[08:18] Every node points back to the words it came from.
[08:21] That graph goes to a compiler that never calls a model in 11 deterministic stages.
[08:26] It turns the graph into an execution tag or it fails closed.
[08:31] There are only three verdicts.
[08:32] Valid, needs clarification or refused.
[08:38] Which Harper comes back as a question, not a guess.
[08:42] The plan is compiled immutable and pinned by its hash.
[08:56] The model translates.
[08:58] The plan is compiled.
[08:59] The compiler checks the plan against the grammar, the vocabulary of what the firm is allowed to do.
[09:05] 42 typed action contracts across forms, documents, email, e signature, calendar, and human steps.
[09:13] Each action the planner asks for has to match a contract.
[09:17] Here, form fill finds its forms contract and locks in.
[09:21] If the planner asks for an action that isn't in the grammar or wires one up wrong, the compiler refuses.
[09:28] There are about 40 ways to fail closed from a cycle in the graph to an unsupported action.
[09:34] And the same contract digest is pinned in the engine, the back end, and the UI.
[09:39] So, all three agree on what an action means.
[09:42] That is what a closed domain buys you.
[09:44] The model can only ask for things the firm already knows how to do safely.
[09:48] Here is a compiled plan for a W9 chase, send the client the form, wait for it to come back, check it, and file it.
[09:56] Before anything runs, a person approves the plan itself, find the form, prepare it for the client to fill, render the PDF, and draft the email.
[10:05] Drafting is a model call, so it is violet.
[10:12] Sending is an action gate.
[10:14] Nothing leaves the firm until a person approves that exact email.
[10:18] Then the plan waits for a reply and a judge steps in to read it.
[10:24] The returned form is verified.
[10:26] If it matches, it's archived behind another gate.
[10:30] If it doesn't, the plan follows up.
[10:32] Now look at the dashed steps.
[10:35] Rendering the PDF, attaching it, judging the reply.
[10:39] The planner never asked for them.
[10:42] The grammar added them because the contracts require them.
[10:45] In the story, waiting, approvals, and follow-ups were most of the work.
[10:50] Here, there are nodes in the plan, not prompts.
[10:53] A reply weight defaults to 72 hours, and an explicit deadline can be anything from a minute to 30 days.
[11:01] When it passes, a rule decides what happens, not a model.
[11:05] One approval covers the plan and every external action like sending an email or a signature request has its own gate.
[11:13] The approval is tied to a hash of the exact payload and it's checked again right before sending.
[11:19] When a client replies with only part of what we asked, the reply judge marks it partial.
[11:24] That judgment is provisional.
[11:27] The step stays blocked until a person confirms it.
[11:30] A person confirms it and the step unblocks.
[11:32] Then the follow-up runs as a new child plan.
[11:35] The approved parent is never changed and there are at most five rounds.
[11:46] Once a plan is approved, a state machine runs it.
[11:49] It picks every step and records every result.
[11:52] When a step needs judgment, it makes one fixed schema model call.
[11:57] Compose an email, judge a reply, spot missing values, extract fields, verify a document, the answer comes back typed, and the state machine decides what to do with it.
[12:08] Side effects get a life cycle of their own.
[12:10] Prepare, dispatching, succeeded, or when the world doesn't answer cleanly.
[12:15] Outcome unknown, and manual review.
[12:18] There are no automatic retries.
[12:20] A chat message can approve a step or answer a question.
[12:24] It can't change an already approved plan.
[12:26] The model steers.
[12:28] It never drives.
[12:35] Now look at what is left for the model to do.
[12:37] Outside the planner, the judgments are small and typed.
[12:41] Is this reply complete?
[12:43] Does this document match?
[12:44] Which field is which?
[12:46] They don't need frontier intelligence.
[12:48] A fast small model, the flash tier, can handle them.
[12:52] In this illustrative run of 20 decisions, 19 go to the flash tier and one goes to the frontier.
[12:58] That is why we designed for about 95% of model decisions in a workflow, never needing a frontier model.
[13:05] And the flash tier can run on hardware you control.
[13:09] Only the genuinely hard cases go to a frontier model.
[13:14] And the 5% is not a permanent tax because every judgment is typed and logged and the provisional ones are confirmed or rejected by a person.
[13:24] Each frontier decision produces eval grade distribution real training data by construction.
[13:31] That data can postrain the flash tier.
[13:34] Logging and evals exist today.
[13:36] The training loop is next.
[13:38] So the frontier share shrinks release over release until even the hard cases run locally.
[13:51] Remember the requirement we started with because data is sensitive.
[13:55] You need a path to move workflows entirely to privately hosted LLMs.
[14:00] So can this design really get there?
[14:02] Take one type call from the Harper Berby 9 plan.
[14:06] Every model call is small, typed, and checked against a schema.
[14:10] So pointing it at a privately hosted model is a configuration change, not a rewrite.
[14:16] One line changes and the same answer comes back.
[14:19] Every other call has that same small typed shape.
[14:22] So the same change applies to almost all of them.
[14:26] That is how about 95% local inference becomes viable as an architectural consequence, not a policy promise.
[14:33] That is the path that sensitive data demands and compliance risk goes down as an engineering road map release by release.
[14:42] So if you remember one thing, don't ask whether your system should have more agency or less.
[14:48] Ask where the boundary is.
[14:50] Let determinism govern control flow, evidence, and side effects.
[14:55] Let intelligence handle interpretation and judgment.
[14:58] At typed points, you can inspect and use every judgment you log to need less of it.
[15:04] Next time, ask where the boundary is.
[15:12] Thank you.
[15:13] I'm Karthik, CTO at Perpetual.
[15:16] If you're building AI into regulated work, I'd love to compare notes.