Reliable, Smart Workflows for Finance, Wealth and Insurance — Karthic Rao, Perpetual

AI Engineer · 15 min · 187 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:00Hello, I'm Karthik, CTO at Perpetual.
00:03In this presentation, I'll talk about building workflows that are both reliable and intelligent for finance, wealth, and insurance.
00:10In regulated domains, workflows must be reliable enough to meet regulatory standards while still being intelligent enough to handle the real world.
00:17In fields such as finance and wealth management, this presents both a technical and a compliance issue.
00:22Reliability and intelligence are not opposites.
00:25It is up to you where you draw the boundary between them and that is the point of this talk deciding where that boundary lies.
00:32For us, it's not merely a thought experiment but a question that surfaces serious consequences.
00:37Perpetual is a wealth management rollup we acquire running traditional wealth management firms and we maximize their throughput and revenue with the help of AI.
00:47Wealth management is a regulated industry and that is where we build with AI in production.
00:53Everything in this talk is grounded in building for those constraints.
00:57Workflows in finance and wealth management come with four facts and each of them pushes the architecture in the same direction.
01:04First, they're complex and they need runtime strategies as the process unfolds.
01:09You can't script every path in advance.
01:11Second, the data is sensitive, so you need a path to move these workflows entirely onto privately hosted models.
01:18Third, in a highly regulated industry, a policy violation isn't a bug report.
01:23It has severe consequences.
01:24So, fourth, execution has to be highly reliable and deterministic.
01:29Inject intelligence only where runtime judgment is genuinely required and minimize reliance on LLM everywhere else.
01:38AI is spreading through regulated industries, but there's still no consensus on an architecture that gives you high reliability with the least use of LLM intelligence.
01:48Meanwhile, most of what developers hear from the big labs is how to burn more tokens.
01:53But in a regulated industry, token maxing is the wrong goal.
01:56These tasks are close to main.
01:58You can lay them out in advance and build them into a massive state machine.
02:02And that machine needs only a sprinkle of intelligence to navigate it.
02:06Most steps just follow the next edge.
02:08Only at a few judgment points does the model pick a branch.
02:12So the intelligent way to build is to use intelligence minimally and on purpose.
02:19Let's make this concrete with a real wealth management workflow.
02:23A wealth adviser turns to their assistant and says one sentence.
02:27Fill out the IRS W9 for Mr. Harper.
02:30To a person, that's a simple instruction.
02:33Watch what it actually triggers when a human being executes it.
02:36The task has complexities that go beyond what meets the eye.
02:40First, the assistant finds form W9 from the firm's records or from the IRS website.
02:45Then, they look for Mr. Harper in the system of record and find three Harper households.
02:50So, they call the adviser, which Harper do you mean?
02:53Only once that ambiguity is resolved do they move on to the firm.
02:57A lookup, an ambiguity, and a human in the loop all in the first minute of a oneline task.
03:08Filling a form sounds like data entry.
03:10It isn't.
03:11Every form is a small program.
03:14A W9 first decides who the taxpayer is, the person or the owner if it's a disregarded entity.
03:21A foreign person stops right here because they need a W8 instead.
03:25Then how they're taxed.
03:27Exactly one box on line 3 A and an LLC adds its letter.
03:32That decides which name goes on line one and which on line two.
03:36Then whose number?
03:37An SSN or an EIN?
03:39Never both.
03:40Taken from a verified document, never from memory.
03:43Exemption codes apply only to some entities and never to an S corporation.
03:48And finally, who signs under penalties of perjury?
03:51Each answer changes the next question.
03:54And the people involved only ever see three kinds of gates.
03:57A question, a document to collect, or a signature.
04:04Now they start filling the form and realize the system of record is missing three of the values it needs.
04:11They check the documents on file and the system of record again still missing.
04:15So they start an email thread with a client asking for exactly those three values.
04:20The task now depends on someone outside the firm on their timeline and now the workflow waits.
04:31The request is with the client and all three values are still missing.
04:35If the client doesn't reply within 2 days, the assistant follows up and brings it to the adviser at the daily standup.
04:41If the client replies with only two of the three values, they follow up again for the missing one.
04:46When all three arrive, they complete the form and write the new information back to the system of record, including the trail of how they obtained those values.
04:54In this industry, how you know a value matters as much as the value itself.
05:07Once the form is filled, it goes out for e- signature.
05:10If the client doesn't sign within a day or two, the assistant nudges them again.
05:15Once it's signed, they file it for the client, send the acknowledgement and update the adviser at the next day standup.
05:22One sentence from the adviser became a process that runs for days, crosses several systems, waits on two people, and loops back on itself more than once.
05:44Look at what's inside that one sentence.
05:47Ambiguity to resolve, data to gather, approvals, weights measured in days, follow-up loops, and an evidence trail at the end.
06:03Workflow automation tools exist, but they rarely handle this real world complexity and ambiguity.
06:09A fixed flowchart breaks at the first witch harper and the lack of a native system of record with an integration and an agent that improvises its way through is exactly what a compliance team can't sign off on.
06:21One breaks on ambiguity, the other can't be signed off.
06:24Neither handles what's inside that sentence.
06:27So what do we do about it?
06:29The fashionable framing is more agency and better models.
06:33Give agent the whole task and wait for the next model to make it reliable.
06:38The reflexive answer, especially inside regulated firms, is the opposite.
06:43Less agency, more rules, lock it down and hardcode every path.
06:48Both are the wrong axis.
06:51In regulated verticals, reliability comes from where you place the boundary between the two, between the parts that must be deterministic and the parts that genuinely need judgment.
07:01Here's the principle.
07:03Determinism governs control flow, evidence, and side effects.
07:07what runs next, what we know and how we know it.
07:10And anything that changes the world, like a record written, an email sent, or a form filed, intelligence provides interpretation and judgment, reading and ambiguous instruction, deciding which Harper, understanding a client's reply, the model answers a type question,
07:33and the deterministic system decides what to do with the answer.
07:36And the interplay is recursive.
07:38A judgment can open a new smaller plan that's compiled, gated, and deterministic.
07:44Again, you'll see that in a moment with follow-ups.
07:47The boundary between the two is the design.
07:49Here's how that works in the workflow engine we've built.
08:01The request goes to a planner that is a model call and its only job is translation.
08:07It emits a type semantic graph of intents, entities, facts, artifacts, events, and constraints.
08:14Each grounded in the words of the request with any ambiguities it found.
08:18Every node points back to the words it came from.
08:21That graph goes to a compiler that never calls a model in 11 deterministic stages.
08:26It turns the graph into an execution tag or it fails closed.
08:31There are only three verdicts.
08:32Valid, needs clarification or refused.
08:38Which Harper comes back as a question, not a guess.
08:42The plan is compiled immutable and pinned by its hash.
08:56The model translates.
08:58The plan is compiled.
08:59The compiler checks the plan against the grammar, the vocabulary of what the firm is allowed to do.
09:0542 typed action contracts across forms, documents, email, e signature, calendar, and human steps.
09:13Each action the planner asks for has to match a contract.
09:17Here, form fill finds its forms contract and locks in.
09:21If the planner asks for an action that isn't in the grammar or wires one up wrong, the compiler refuses.
09:28There are about 40 ways to fail closed from a cycle in the graph to an unsupported action.
09:34And the same contract digest is pinned in the engine, the back end, and the UI.
09:39So, all three agree on what an action means.
09:42That is what a closed domain buys you.
09:44The model can only ask for things the firm already knows how to do safely.
09:48Here is a compiled plan for a W9 chase, send the client the form, wait for it to come back, check it, and file it.
09:56Before anything runs, a person approves the plan itself, find the form, prepare it for the client to fill, render the PDF, and draft the email.
10:05Drafting is a model call, so it is violet.
10:12Sending is an action gate.
10:14Nothing leaves the firm until a person approves that exact email.
10:18Then the plan waits for a reply and a judge steps in to read it.
10:24The returned form is verified.
10:26If it matches, it's archived behind another gate.
10:30If it doesn't, the plan follows up.
10:32Now look at the dashed steps.
10:35Rendering the PDF, attaching it, judging the reply.
10:39The planner never asked for them.
10:42The grammar added them because the contracts require them.
10:45In the story, waiting, approvals, and follow-ups were most of the work.
10:50Here, there are nodes in the plan, not prompts.
10:53A reply weight defaults to 72 hours, and an explicit deadline can be anything from a minute to 30 days.
11:01When it passes, a rule decides what happens, not a model.
11:05One approval covers the plan and every external action like sending an email or a signature request has its own gate.
11:13The approval is tied to a hash of the exact payload and it's checked again right before sending.
11:19When a client replies with only part of what we asked, the reply judge marks it partial.
11:24That judgment is provisional.
11:27The step stays blocked until a person confirms it.
11:30A person confirms it and the step unblocks.
11:32Then the follow-up runs as a new child plan.
11:35The approved parent is never changed and there are at most five rounds.
11:46Once a plan is approved, a state machine runs it.
11:49It picks every step and records every result.
11:52When a step needs judgment, it makes one fixed schema model call.
11:57Compose an email, judge a reply, spot missing values, extract fields, verify a document, the answer comes back typed, and the state machine decides what to do with it.
12:08Side effects get a life cycle of their own.
12:10Prepare, dispatching, succeeded, or when the world doesn't answer cleanly.
12:15Outcome unknown, and manual review.
12:18There are no automatic retries.
12:20A chat message can approve a step or answer a question.
12:24It can't change an already approved plan.
12:26The model steers.
12:28It never drives.
12:35Now look at what is left for the model to do.
12:37Outside the planner, the judgments are small and typed.
12:41Is this reply complete?
12:43Does this document match?
12:44Which field is which?
12:46They don't need frontier intelligence.
12:48A fast small model, the flash tier, can handle them.
12:52In this illustrative run of 20 decisions, 19 go to the flash tier and one goes to the frontier.
12:58That is why we designed for about 95% of model decisions in a workflow, never needing a frontier model.
13:05And the flash tier can run on hardware you control.
13:09Only the genuinely hard cases go to a frontier model.
13:14And the 5% is not a permanent tax because every judgment is typed and logged and the provisional ones are confirmed or rejected by a person.
13:24Each frontier decision produces eval grade distribution real training data by construction.
13:31That data can postrain the flash tier.
13:34Logging and evals exist today.
13:36The training loop is next.
13:38So the frontier share shrinks release over release until even the hard cases run locally.
13:51Remember the requirement we started with because data is sensitive.
13:55You need a path to move workflows entirely to privately hosted LLMs.
14:00So can this design really get there?
14:02Take one type call from the Harper Berby 9 plan.
14:06Every model call is small, typed, and checked against a schema.
14:10So pointing it at a privately hosted model is a configuration change, not a rewrite.
14:16One line changes and the same answer comes back.
14:19Every other call has that same small typed shape.
14:22So the same change applies to almost all of them.
14:26That is how about 95% local inference becomes viable as an architectural consequence, not a policy promise.
14:33That is the path that sensitive data demands and compliance risk goes down as an engineering road map release by release.
14:42So if you remember one thing, don't ask whether your system should have more agency or less.
14:48Ask where the boundary is.
14:50Let determinism govern control flow, evidence, and side effects.
14:55Let intelligence handle interpretation and judgment.
14:58At typed points, you can inspect and use every judgment you log to need less of it.
15:04Next time, ask where the boundary is.
15:12Thank you.
15:13I'm Karthik, CTO at Perpetual.
15:16If you're building AI into regulated work, I'd love to compare notes.