Your Financial Agent Needs to Show Its Work - Anirudh Veeraragavan, Fieldguide

AI Engineer · 13 min · 159 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:12Uh my name is Anerud and I'm lead security engineer at field guide where we're defining what a gentic audit means.
00:19So in audit the opinion is the product and opinion is only as good as the basis behind it.
00:26So it's not so much whether you're correct or wrong.
00:28It's really about the decision-making process behind what you're putting forward.
00:33And so this raises the question, what does an agentic decision-making process look like?
00:40So to kind of dive into this, let's build a financial agent.
00:43So we'll take a financial audit use case and we'll take this specific audit procedure where we're testing a bank reconciliation.
00:51uh to any auditors in the room, please bear with me on the simplification.
00:55But bank reconciliation is where you take the client's book and the cash balance that they have on that those books and you compare it to the cash balance that their bank has.
01:04And any differences should be explained by things like deposits in transit, outstanding checks makes.
01:10Basically, you want to make sure that whatever the client is saying is their financial reality actually matches the truth.
01:17So, how would you kind of build this out if you wanted to build an agent for this?
01:20because this can be kind of tedious high volume work.
01:23So what you would do is you would take your client's book.
01:25You would take the list of reconciling items.
01:27You would take evidence.
01:28You would give it to your agent which has some audit methodology for doing the the matching.
01:33You can even have some prior year audits as examples.
01:36You can even build an eval previously reviewed audits and then you kind of surface everything to a human reviewer to finally sign off.
01:42This is totally reasonable.
01:44Great great approach.
01:45So you know we build this out.
01:47We have great evals, great metrics, users are happy, everyone wins, we ship this to GA and the story ends there until your audit is selected for inspection and someone asks you why was this sufficient
02:01that is I don't care what your eval say.
02:03I don't care what your metrics say.
02:05I don't even care what your users say for this specific audit, for this specific test procedure, for this specific piece of evidence.
02:13How did you arrive at this decision?
02:16And you may be wondering, this seems like a little overkill.
02:18You know, this is maybe not going to happen that much in practice.
02:22But the thing is, financial fraud is very much real.
02:25Paragrin Financial Group is, or I guess I should say, was a brokerage firm that was uh based in Iowa.
02:32And for over 20 years, they committed bank account fraud.
02:35On paper, they claimed they had over $200 million.
02:38And in reality, at the bank, there was only $5 million.
02:41And when this fraud came to light, $200 million basically just vanished.
02:46And I'm not here to litigate who or what or why this happened.
02:50But what this shows us is that in the financial industry, the consequences of being wrong are quite severe.
02:57So we need to have confidence behind the decisions that we make and we need to be able to stand behind them in case things do go ary in the future.
03:05So in terms of how to do this, I think the audit industry has like a nice definition we can take some inspiration from.
03:11uh they say that an experienced auditor who hasn't done the work before should be able to understand the test you performed, the evidence you collected, the decision that was reached.
03:21In other words, an experienced auditor new to the work can follow it.
03:25And we can adopt this to the agent world where we say an experienced auditor new to the work can follow the agent.
03:31And in order to follow the agent, you need these different controls in place around authorization, execution control, evidence, provenence, and evaluation integrity.
03:41And ultimately, all of this rests upon knowing when the agent should stop and the human should decide instead.
03:48So this is a pattern for decision assurance.
03:51And I'll kind of walk through this financial audit example again, but it really applies to many more domains than just this one.
03:58So let's go back to our audit.
04:00You know, we're going back to bank reconciliation.
04:02We have this fictional client called their freight lines and we're closing the books for 2026.
04:07So, as per their bank, they have around $2.9 million.
04:11On their books, they have $2.7 million.
04:13And the difference is explained by 41 reconciling items.
04:17So, we'll take these four, two checks, two deposits, and let's kind of see what our agent does and let's walk through these different controls to ensure we're making defensible decision-m.
04:27So, the first one is around authorization.
04:29So you can imagine that a human reviewer is kind of going through these reconciling items.
04:34They flag one of these checks as hey I think there may be something wrong here and a few days later we get some additional context.
04:40We get some more additional information around why that is not an exception which triggers the agent run and then it rewrites that to no exception.
04:49Now the thing here here is that the human already a human already reviewed it and they already signed it but what they reviewed the content behind it has now changed.
04:58The agent didn't do anything wrong.
04:59It had the permission.
05:00It had the evidence.
05:01And it arrived at the right conclusion.
05:04But in this specific instance, it shouldn't have overwritten anything because now what the reviewer has claimed to review doesn't match what they actually looked at.
05:12So what we're seeing here is that authorization isn't just about is this tool call allowed.
05:17It's is this tool call allowed for this object given this state at this point in time.
05:22The agent should be allowed to draft unreviewed rows as itself, but it cannot overwrite what a human reviewed.
05:28It cannot replace what a human did.
05:30It has to kind of exist as its own suggestion as opposed to its own authoritative truth.
05:37The next control is around execution control.
05:39Uh so you can imagine you know in 2027 we kind of go through this exercise, we have no exception and then maybe a year or two later we're selected for review.
05:48We rerun it and we get a completely different result.
05:50So now the question arises like what happened?
05:53How did this how could this have happened?
05:55So we have logs.
05:56We know that the agent fetched evidence.
05:58It did some reasoning.
05:59It outputed uh its final answer.
06:02But that's not enough to reconstruct what happened in 2027 against what happened in 2028 and identify the gap.
06:09So really what we need is a record.
06:10We need a record of these were the steps that happened.
06:14This is the culmination of the run and you can interrogate each point along the way.
06:19So this run is a chain of recorded steps where each step has a ID that kind of lets you understand what is being done here.
06:27What are the inputs?
06:28What are the outputs?
06:29And you can clearly delineate between deterministic steps where you're using code to fetch an item or to do a match versus where you're using the agent to extract, you know, a field from a PDF or to arrive at the final conclusion.
06:43And given this chain of recorded steps, you can then rerun individual steps as opposed to the whole thing and seeing what happens.
06:50And this gives you much stronger assurance around being able to uh being able to compare runs against different points in time because now we can kind of compare our new run against our old run,
07:01identify the exact step where something went wrong.
07:04And in this example, we're seeing that our original run was actually correct, even though the run of this newer run is wrong.
07:13But again, it's not so much about whether you're correct or not.
07:15It's about being able to demonstrate with a high level of conviction that you know you were right.
07:20You didn't just get lucky that the model picked the right answer.
07:22You can clearly show that you had these steps and when you rerun this step, this is why there's now a difference compared to when we originally ran it, the model did arrive at the right answer.
07:33Then you next we have evidence providence.
07:36So in this example, the model arrived at the wrong conclusion.
07:39It said that this was valid, but it actually pointed to the wrong account.
07:43Um, and if you look at the citation, the model claims that it found this piece of evidence from January bank statement page two.
07:49And the thing is this citation isn't useful because you can imagine if you have a big engagement, you could have many such documents with the same name.
07:58Even if this is a unique document name, what version did the model look at?
08:01Who uploaded this?
08:02Like where did this even come from?
08:05And this isn't even related to hallucination.
08:07It can be a real document but we need to understand more about the context behind that document.
08:13So what we arrive at is a citation is identity.
08:15It's not just a string.
08:17It's have a very specific ID, a very specific version of the document, a hash of the contents behind it.
08:22And being able to concretely tie where an agent picks something from allows you to then assert rules around the evidence that's allowed or any kind of um quality constraints you want around it.
08:35For example, we may only want evidence that's sourced directly from the bank as opposed to something that the client uploaded where they may be able to tamper with it in some sort of way.
08:44And finally, we arrive at evaluation integrity.
08:48So, we claimed we have this eval, this gold data that was based on last year's reviewed audits.
08:54But just because the audit was reviewed doesn't necessarily mean it's correct.
08:57So it's important to audit that data set as well because mistakes in it could punish a right agent or it could reward a wrong agent.
09:06So there's some examples here of how in the reviewed audit you could have examples that were then superseded.
09:13You could have uh you could have decisions that were made based on out of bad evidence or you could even have changes in methodology.
09:20And all of these are examples of how your underlying goal data may not be as gold as you thought it was.
09:26And what this can kind of look like in practice is that this kind of headline great overall score can hide a lot of underlying failures.
09:34So in this case we have 97% but out of nine of the misses four was because the eval itself was just wrong.
09:41So our agent is actually doing better than expected.
09:44But there are 12 examples where our evil said this should be no exception.
09:50But because of some business rule changes or because of some audit methodology changes these are now wrong.
09:54So these are examples where the agent actually missed and was incorrect.
09:58So what we see is we don't know if our agent is doing better than expected, it's doing worse than expected.
10:03The score is hiding a lot of important context behind how effective our agent actually is.
10:09And you know ultimately there is a point where the agent has to stop and ask a human.
10:14And all of this ties back to kind of examples we've already covered through these past few controls.
10:18If it's trying to change work a human has already reviewed.
10:21If a new run is disagreeing with a previous run, if evidence is not conforming with these different rules, all of these are examples where something has gone ary and a human needs to step in to determine what's happening.
10:34And the key thing here is that these escalations, they should be easy to verify, but not necessarily easy to approve.
10:40We know automation bias is real.
10:42If you can if you continually ping the human, they'll just start hitting yes to everything.
10:46So, we only want to flag exceptional cases.
10:49And those exceptional cases should very clearly call out like what is wrong and where does someone need to step in.
10:55So the example here is we have this deposit.
10:58It was cleared in 11 business days but it's calling out what is the discrepancy and what needs to be investigated.
11:05It may take some time for the human to do this review and to eventually arrive at an approval decision.
11:09But that's okay because we know 80% of the time the agent will do what it needs to do.
11:13And for those 20% of cases when we want the human to step in, we want to make sure they're stepping in where it counts and they have the context they need to arrive at the right decision.
11:23So if we kind of take a step back, we've covered these four controls and what we arrive at is defensibility is an and in the sense that you need all of them otherwise things can only can always slip through the cracks.
11:36If you skip authorization, then you could have uh a scenario where someone reviewed something, but then the answer changes out underneath them.
11:45If you skip execution control, you can arrive at the right answer, but you can't show how.
11:49You may as well attribute it to luck at that point.
11:52If you skip evidence providence, you can have the wrong answer, but you don't know where the wrong answer came from.
11:58Came from some citation that you can't tie back to a concrete source.
12:02And if you skip evaluation integrity, you can arrive at a great flagship number of 97%.
12:07But you have no idea what [clears throat] the reality is.
12:09Are you doing better than that?
12:10Are you doing worse than that?
12:11All you know is you have this number to go off of.
12:15And this pattern for assurance, you know, I've kind of been talking about audit, but it really scales beyond audit into many other domains.
12:21In credit, if you're approving or declining a loan, you can interrogate that.
12:25You know, why did you make that decision?
12:27in AML if you're clo as closing or escalating alert again what was the evidence behind that so these same four questions can apply to different domains where you may change the nouns behind them but the principles
12:39uh are still the same so you know I think uh what this really comes down to is anytime you're making a financial decision where the impact could be quite
12:51uh tremendous you need to have some defensibility behind how you're arriving at that decision And if your if your defense is the model said so that's not really going to cut it in today's day and world.
13:06So thank you all for your time.
13:07Uh these are some of the things we're working on at Field Guide.
13:10So if you're interested we are hiring and if you want to learn talk more about this feel free to find me at the conference.
13:16But yeah thank you all for your time and hope you enjoy the rest of today.
13:20[applause]