WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=gdzsJZJ6DDs

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/gdzsJZJ6DDs.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.719 --> 00:00:19.359
Uh my name is Anerud and I'm lead security engineer at field guide where we're defining what a gentic audit means.

s2
00:00:19.359 --> 00:00:26.000
So in audit the opinion is the product and opinion is only as good as the basis behind it.

s3
00:00:26.000 --> 00:00:28.800
So it's not so much whether you're correct or wrong.

s4
00:00:28.800 --> 00:00:33.760
It's really about the decision-making process behind what you're putting forward.

s5
00:00:33.760 --> 00:00:40.079
And so this raises the question, what does an agentic decision-making process look like?

s6
00:00:40.079 --> 00:00:43.360
So to kind of dive into this, let's build a financial agent.

s7
00:00:43.360 --> 00:00:51.440
So we'll take a financial audit use case and we'll take this specific audit procedure where we're testing a bank reconciliation.

s8
00:00:51.680 --> 00:00:55.280
uh to any auditors in the room, please bear with me on the simplification.

s9
00:00:55.280 --> 00:01:04.479
But bank reconciliation is where you take the client's book and the cash balance that they have on that those books and you compare it to the cash balance that their bank has.

s10
00:01:04.479 --> 00:01:10.400
And any differences should be explained by things like deposits in transit, outstanding checks makes.

s11
00:01:10.400 --> 00:01:17.360
Basically, you want to make sure that whatever the client is saying is their financial reality actually matches the truth.

s12
00:01:17.360 --> 00:01:20.320
So, how would you kind of build this out if you wanted to build an agent for this?

s13
00:01:20.320 --> 00:01:23.119
because this can be kind of tedious high volume work.

s14
00:01:23.119 --> 00:01:25.439
So what you would do is you would take your client's book.

s15
00:01:25.439 --> 00:01:27.680
You would take the list of reconciling items.

s16
00:01:27.680 --> 00:01:28.880
You would take evidence.

s17
00:01:28.880 --> 00:01:33.600
You would give it to your agent which has some audit methodology for doing the the matching.

s18
00:01:33.600 --> 00:01:36.560
You can even have some prior year audits as examples.

s19
00:01:36.560 --> 00:01:42.799
You can even build an eval previously reviewed audits and then you kind of surface everything to a human reviewer to finally sign off.

s20
00:01:42.799 --> 00:01:44.240
This is totally reasonable.

s21
00:01:44.240 --> 00:01:45.840
Great great approach.

s22
00:01:45.840 --> 00:01:47.360
So you know we build this out.

s23
00:01:47.360 --> 00:02:01.759
We have great evals, great metrics, users are happy, everyone wins, we ship this to GA and the story ends there until your audit is selected for inspection and someone asks you why was this sufficient

s24
00:02:01.759 --> 00:02:03.920
that is I don't care what your eval say.

s25
00:02:03.920 --> 00:02:05.680
I don't care what your metrics say.

s26
00:02:05.680 --> 00:02:13.680
I don't even care what your users say for this specific audit, for this specific test procedure, for this specific piece of evidence.

s27
00:02:13.680 --> 00:02:16.400
How did you arrive at this decision?

s28
00:02:16.400 --> 00:02:18.800
And you may be wondering, this seems like a little overkill.

s29
00:02:18.800 --> 00:02:22.239
You know, this is maybe not going to happen that much in practice.

s30
00:02:22.239 --> 00:02:25.280
But the thing is, financial fraud is very much real.

s31
00:02:25.280 --> 00:02:32.239
Paragrin Financial Group is, or I guess I should say, was a brokerage firm that was uh based in Iowa.

s32
00:02:32.239 --> 00:02:35.599
And for over 20 years, they committed bank account fraud.

s33
00:02:35.599 --> 00:02:38.640
On paper, they claimed they had over $200 million.

s34
00:02:38.640 --> 00:02:41.920
And in reality, at the bank, there was only $5 million.

s35
00:02:41.920 --> 00:02:46.720
And when this fraud came to light, $200 million basically just vanished.

s36
00:02:46.720 --> 00:02:50.640
And I'm not here to litigate who or what or why this happened.

s37
00:02:50.640 --> 00:02:57.280
But what this shows us is that in the financial industry, the consequences of being wrong are quite severe.

s38
00:02:57.280 --> 00:03:05.680
So we need to have confidence behind the decisions that we make and we need to be able to stand behind them in case things do go ary in the future.

s39
00:03:05.680 --> 00:03:11.440
So in terms of how to do this, I think the audit industry has like a nice definition we can take some inspiration from.

s40
00:03:11.440 --> 00:03:21.200
uh they say that an experienced auditor who hasn't done the work before should be able to understand the test you performed, the evidence you collected, the decision that was reached.

s41
00:03:21.200 --> 00:03:25.360
In other words, an experienced auditor new to the work can follow it.

s42
00:03:25.360 --> 00:03:31.920
And we can adopt this to the agent world where we say an experienced auditor new to the work can follow the agent.

s43
00:03:31.920 --> 00:03:41.680
And in order to follow the agent, you need these different controls in place around authorization, execution control, evidence, provenence, and evaluation integrity.

s44
00:03:41.680 --> 00:03:48.000
And ultimately, all of this rests upon knowing when the agent should stop and the human should decide instead.

s45
00:03:48.000 --> 00:03:51.120
So this is a pattern for decision assurance.

s46
00:03:51.120 --> 00:03:58.640
And I'll kind of walk through this financial audit example again, but it really applies to many more domains than just this one.

s47
00:03:58.640 --> 00:04:00.319
So let's go back to our audit.

s48
00:04:00.319 --> 00:04:02.879
You know, we're going back to bank reconciliation.

s49
00:04:02.879 --> 00:04:07.519
We have this fictional client called their freight lines and we're closing the books for 2026.

s50
00:04:07.519 --> 00:04:11.120
So, as per their bank, they have around $2.9 million.

s51
00:04:11.120 --> 00:04:13.920
On their books, they have $2.7 million.

s52
00:04:13.920 --> 00:04:17.280
And the difference is explained by 41 reconciling items.

s53
00:04:17.280 --> 00:04:27.440
So, we'll take these four, two checks, two deposits, and let's kind of see what our agent does and let's walk through these different controls to ensure we're making defensible decision-m.

s54
00:04:27.440 --> 00:04:29.759
So, the first one is around authorization.

s55
00:04:29.759 --> 00:04:34.000
So you can imagine that a human reviewer is kind of going through these reconciling items.

s56
00:04:34.000 --> 00:04:40.320
They flag one of these checks as hey I think there may be something wrong here and a few days later we get some additional context.

s57
00:04:40.320 --> 00:04:49.120
We get some more additional information around why that is not an exception which triggers the agent run and then it rewrites that to no exception.

s58
00:04:49.120 --> 00:04:58.160
Now the thing here here is that the human already a human already reviewed it and they already signed it but what they reviewed the content behind it has now changed.

s59
00:04:58.160 --> 00:04:59.600
The agent didn't do anything wrong.

s60
00:04:59.600 --> 00:05:00.639
It had the permission.

s61
00:05:00.639 --> 00:05:01.600
It had the evidence.

s62
00:05:01.600 --> 00:05:04.000
And it arrived at the right conclusion.

s63
00:05:04.000 --> 00:05:12.720
But in this specific instance, it shouldn't have overwritten anything because now what the reviewer has claimed to review doesn't match what they actually looked at.

s64
00:05:12.720 --> 00:05:17.600
So what we're seeing here is that authorization isn't just about is this tool call allowed.

s65
00:05:17.600 --> 00:05:22.560
It's is this tool call allowed for this object given this state at this point in time.

s66
00:05:22.560 --> 00:05:28.240
The agent should be allowed to draft unreviewed rows as itself, but it cannot overwrite what a human reviewed.

s67
00:05:28.240 --> 00:05:30.240
It cannot replace what a human did.

s68
00:05:30.240 --> 00:05:37.440
It has to kind of exist as its own suggestion as opposed to its own authoritative truth.

s69
00:05:37.440 --> 00:05:39.919
The next control is around execution control.

s70
00:05:39.919 --> 00:05:48.240
Uh so you can imagine you know in 2027 we kind of go through this exercise, we have no exception and then maybe a year or two later we're selected for review.

s71
00:05:48.240 --> 00:05:50.639
We rerun it and we get a completely different result.

s72
00:05:50.639 --> 00:05:53.120
So now the question arises like what happened?

s73
00:05:53.120 --> 00:05:55.199
How did this how could this have happened?

s74
00:05:55.199 --> 00:05:56.240
So we have logs.

s75
00:05:56.240 --> 00:05:58.080
We know that the agent fetched evidence.

s76
00:05:58.080 --> 00:05:59.120
It did some reasoning.

s77
00:05:59.120 --> 00:06:02.000
It outputed uh its final answer.

s78
00:06:02.000 --> 00:06:09.199
But that's not enough to reconstruct what happened in 2027 against what happened in 2028 and identify the gap.

s79
00:06:09.199 --> 00:06:10.880
So really what we need is a record.

s80
00:06:10.880 --> 00:06:14.160
We need a record of these were the steps that happened.

s81
00:06:14.160 --> 00:06:19.759
This is the culmination of the run and you can interrogate each point along the way.

s82
00:06:19.759 --> 00:06:27.520
So this run is a chain of recorded steps where each step has a ID that kind of lets you understand what is being done here.

s83
00:06:27.520 --> 00:06:28.479
What are the inputs?

s84
00:06:28.479 --> 00:06:29.759
What are the outputs?

s85
00:06:29.759 --> 00:06:43.919
And you can clearly delineate between deterministic steps where you're using code to fetch an item or to do a match versus where you're using the agent to extract, you know, a field from a PDF or to arrive at the final conclusion.

s86
00:06:43.919 --> 00:06:50.880
And given this chain of recorded steps, you can then rerun individual steps as opposed to the whole thing and seeing what happens.

s87
00:06:50.880 --> 00:07:01.360
And this gives you much stronger assurance around being able to uh being able to compare runs against different points in time because now we can kind of compare our new run against our old run,

s88
00:07:01.360 --> 00:07:04.639
identify the exact step where something went wrong.

s89
00:07:04.639 --> 00:07:13.360
And in this example, we're seeing that our original run was actually correct, even though the run of this newer run is wrong.

s90
00:07:13.360 --> 00:07:15.919
But again, it's not so much about whether you're correct or not.

s91
00:07:15.919 --> 00:07:20.400
It's about being able to demonstrate with a high level of conviction that you know you were right.

s92
00:07:20.400 --> 00:07:22.560
You didn't just get lucky that the model picked the right answer.

s93
00:07:22.560 --> 00:07:33.039
You can clearly show that you had these steps and when you rerun this step, this is why there's now a difference compared to when we originally ran it, the model did arrive at the right answer.

s94
00:07:33.280 --> 00:07:36.160
Then you next we have evidence providence.

s95
00:07:36.160 --> 00:07:39.440
So in this example, the model arrived at the wrong conclusion.

s96
00:07:39.440 --> 00:07:43.360
It said that this was valid, but it actually pointed to the wrong account.

s97
00:07:43.360 --> 00:07:49.919
Um, and if you look at the citation, the model claims that it found this piece of evidence from January bank statement page two.

s98
00:07:49.919 --> 00:07:58.160
And the thing is this citation isn't useful because you can imagine if you have a big engagement, you could have many such documents with the same name.

s99
00:07:58.160 --> 00:08:01.759
Even if this is a unique document name, what version did the model look at?

s100
00:08:01.759 --> 00:08:02.879
Who uploaded this?

s101
00:08:02.879 --> 00:08:05.039
Like where did this even come from?

s102
00:08:05.039 --> 00:08:07.199
And this isn't even related to hallucination.

s103
00:08:07.199 --> 00:08:13.120
It can be a real document but we need to understand more about the context behind that document.

s104
00:08:13.120 --> 00:08:15.759
So what we arrive at is a citation is identity.

s105
00:08:15.759 --> 00:08:17.199
It's not just a string.

s106
00:08:17.199 --> 00:08:22.479
It's have a very specific ID, a very specific version of the document, a hash of the contents behind it.

s107
00:08:22.479 --> 00:08:35.200
And being able to concretely tie where an agent picks something from allows you to then assert rules around the evidence that's allowed or any kind of um quality constraints you want around it.

s108
00:08:35.200 --> 00:08:44.959
For example, we may only want evidence that's sourced directly from the bank as opposed to something that the client uploaded where they may be able to tamper with it in some sort of way.

s109
00:08:44.959 --> 00:08:48.000
And finally, we arrive at evaluation integrity.

s110
00:08:48.000 --> 00:08:54.000
So, we claimed we have this eval, this gold data that was based on last year's reviewed audits.

s111
00:08:54.000 --> 00:08:57.760
But just because the audit was reviewed doesn't necessarily mean it's correct.

s112
00:08:57.760 --> 00:09:06.640
So it's important to audit that data set as well because mistakes in it could punish a right agent or it could reward a wrong agent.

s113
00:09:06.640 --> 00:09:13.040
So there's some examples here of how in the reviewed audit you could have examples that were then superseded.

s114
00:09:13.040 --> 00:09:20.959
You could have uh you could have decisions that were made based on out of bad evidence or you could even have changes in methodology.

s115
00:09:20.959 --> 00:09:26.880
And all of these are examples of how your underlying goal data may not be as gold as you thought it was.

s116
00:09:26.880 --> 00:09:34.640
And what this can kind of look like in practice is that this kind of headline great overall score can hide a lot of underlying failures.

s117
00:09:34.640 --> 00:09:41.600
So in this case we have 97% but out of nine of the misses four was because the eval itself was just wrong.

s118
00:09:41.600 --> 00:09:44.880
So our agent is actually doing better than expected.

s119
00:09:44.880 --> 00:09:50.160
But there are 12 examples where our evil said this should be no exception.

s120
00:09:50.160 --> 00:09:54.959
But because of some business rule changes or because of some audit methodology changes these are now wrong.

s121
00:09:54.959 --> 00:09:58.880
So these are examples where the agent actually missed and was incorrect.

s122
00:09:58.880 --> 00:10:03.279
So what we see is we don't know if our agent is doing better than expected, it's doing worse than expected.

s123
00:10:03.279 --> 00:10:09.440
The score is hiding a lot of important context behind how effective our agent actually is.

s124
00:10:09.760 --> 00:10:14.399
And you know ultimately there is a point where the agent has to stop and ask a human.

s125
00:10:14.399 --> 00:10:18.720
And all of this ties back to kind of examples we've already covered through these past few controls.

s126
00:10:18.720 --> 00:10:21.680
If it's trying to change work a human has already reviewed.

s127
00:10:21.680 --> 00:10:34.000
If a new run is disagreeing with a previous run, if evidence is not conforming with these different rules, all of these are examples where something has gone ary and a human needs to step in to determine what's happening.

s128
00:10:34.000 --> 00:10:40.800
And the key thing here is that these escalations, they should be easy to verify, but not necessarily easy to approve.

s129
00:10:40.800 --> 00:10:42.640
We know automation bias is real.

s130
00:10:42.640 --> 00:10:46.720
If you can if you continually ping the human, they'll just start hitting yes to everything.

s131
00:10:46.720 --> 00:10:49.600
So, we only want to flag exceptional cases.

s132
00:10:49.600 --> 00:10:55.839
And those exceptional cases should very clearly call out like what is wrong and where does someone need to step in.

s133
00:10:55.839 --> 00:10:58.640
So the example here is we have this deposit.

s134
00:10:58.640 --> 00:11:05.040
It was cleared in 11 business days but it's calling out what is the discrepancy and what needs to be investigated.

s135
00:11:05.040 --> 00:11:09.680
It may take some time for the human to do this review and to eventually arrive at an approval decision.

s136
00:11:09.680 --> 00:11:13.760
But that's okay because we know 80% of the time the agent will do what it needs to do.

s137
00:11:13.760 --> 00:11:23.360
And for those 20% of cases when we want the human to step in, we want to make sure they're stepping in where it counts and they have the context they need to arrive at the right decision.

s138
00:11:23.519 --> 00:11:36.480
So if we kind of take a step back, we've covered these four controls and what we arrive at is defensibility is an and in the sense that you need all of them otherwise things can only can always slip through the cracks.

s139
00:11:36.480 --> 00:11:45.279
If you skip authorization, then you could have uh a scenario where someone reviewed something, but then the answer changes out underneath them.

s140
00:11:45.279 --> 00:11:49.760
If you skip execution control, you can arrive at the right answer, but you can't show how.

s141
00:11:49.760 --> 00:11:52.800
You may as well attribute it to luck at that point.

s142
00:11:52.800 --> 00:11:58.079
If you skip evidence providence, you can have the wrong answer, but you don't know where the wrong answer came from.

s143
00:11:58.079 --> 00:12:02.560
Came from some citation that you can't tie back to a concrete source.

s144
00:12:02.560 --> 00:12:07.680
And if you skip evaluation integrity, you can arrive at a great flagship number of 97%.

s145
00:12:07.680 --> 00:12:09.200
But you have no idea what [clears throat] the reality is.

s146
00:12:09.200 --> 00:12:10.079
Are you doing better than that?

s147
00:12:10.079 --> 00:12:11.360
Are you doing worse than that?

s148
00:12:11.360 --> 00:12:15.040
All you know is you have this number to go off of.

s149
00:12:15.040 --> 00:12:21.760
And this pattern for assurance, you know, I've kind of been talking about audit, but it really scales beyond audit into many other domains.

s150
00:12:21.760 --> 00:12:25.519
In credit, if you're approving or declining a loan, you can interrogate that.

s151
00:12:25.519 --> 00:12:27.839
You know, why did you make that decision?

s152
00:12:27.839 --> 00:12:39.360
in AML if you're clo as closing or escalating alert again what was the evidence behind that so these same four questions can apply to different domains where you may change the nouns behind them but the principles

s153
00:12:39.360 --> 00:12:51.760
uh are still the same so you know I think uh what this really comes down to is anytime you're making a financial decision where the impact could be quite

s154
00:12:51.760 --> 00:13:06.079
uh tremendous you need to have some defensibility behind how you're arriving at that decision And if your if your defense is the model said so that's not really going to cut it in today's day and world.

s155
00:13:06.079 --> 00:13:07.839
So thank you all for your time.

s156
00:13:07.839 --> 00:13:10.639
Uh these are some of the things we're working on at Field Guide.

s157
00:13:10.639 --> 00:13:16.800
So if you're interested we are hiring and if you want to learn talk more about this feel free to find me at the conference.

s158
00:13:16.800 --> 00:13:20.419
But yeah thank you all for your time and hope you enjoy the rest of today.

s159
00:13:20.419 --> 00:13:22.419
[applause]