WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=3osCbSufv8g

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/3osCbSufv8g.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.680 --> 00:00:13.880
Um so I'm Brooke.

s3
00:00:13.880 --> 00:00:18.480
I'm the founder of Koval and we build evaluations for voice agents.

s4
00:00:18.480 --> 00:00:28.200
My background's from Waymo, so I led our evaluation infrastructure team at Waymo and my team was responsible for all of our developer tools for launching and running simulations.

s5
00:00:28.200 --> 00:00:32.720
You might be like, what does that have to do with with voice agents?

s6
00:00:32.720 --> 00:00:36.760
Well, autonomous vehicles are actually very similar to autonomous voice agents.

s7
00:00:36.760 --> 00:00:45.040
How do you make sure that something reliably gets from point A to point B, all the possible paths in between, and also how do you architect lots of models working together

s8
00:00:45.040 --> 00:00:50.000
in order to build an autonomous system that reacts and responds to the world reliably?

s9
00:00:50.000 --> 00:01:02.360
Um so we work with enterprises, everyone from really fast-growing AI companies like Perplexity to Fortune 500s across the board, um to make sure that their voice agents are working well for customer support, logistics, etc.

s10
00:01:02.360 --> 00:01:03.960
Cool, and I'm Shiv.

s11
00:01:03.960 --> 00:01:06.440
Uh I lead our voice AI at Upstart.

s12
00:01:06.440 --> 00:01:13.000
Uh for those not familiar with Upstart, we are a fintech company and we primarily do lending.

s13
00:01:15.680 --> 00:01:16.480
Okay, cool.

s14
00:01:16.480 --> 00:01:19.120
So, let me talk about voice a little bit.

s15
00:01:19.120 --> 00:01:25.360
So, Upstart in general does uh thousands of calls, well, every day.

s16
00:01:25.360 --> 00:01:30.040
Um I think um we talked about domain, which is lending.

s17
00:01:30.040 --> 00:01:35.360
So, what is unique about lending is obviously it's a highly regulated space, um you know.

s18
00:01:35.360 --> 00:01:47.840
And when when we have voice agents doing these calls with with applicants or borrowers, um if things go wrong, it's not just a uh you know, customer experience issue.

s19
00:01:47.840 --> 00:01:51.600
It is It could be a potential regulatory issue, which is worse.

s20
00:01:51.600 --> 00:01:51.880
Right?

s21
00:01:51.880 --> 00:02:03.560
So, um whenever we we launch these agents, we want to make sure that uh you know, we are we are first and foremost compliant with all the rules and regulations.

s22
00:02:03.560 --> 00:02:11.040
So for this one, I'm going to talk about a use case that you know, we we launched.

s23
00:02:11.040 --> 00:02:16.160
The process the the agent that we launched was automating verification calls.

s24
00:02:16.160 --> 00:02:23.280
So what happens is about 90% of people who apply to Upstart are instantly decisioned.

s25
00:02:23.280 --> 00:02:28.880
The 10% that are not, they have to call in and they have to verify some information.

s26
00:02:28.880 --> 00:02:31.080
So use case is fairly simple.

s27
00:02:31.080 --> 00:02:32.520
We ask three questions.

s28
00:02:32.520 --> 00:02:35.400
Will your income sources continue for the next three months?

s29
00:02:35.400 --> 00:02:36.720
How do you plan to use the funds?

s30
00:02:36.720 --> 00:02:51.440
And do you have any outstanding obligations such as you know, alimony, wage garnishments, etc. Okay, so those three questions aside, you know, we we first need to verify that the person calling in is

s31
00:02:51.440 --> 00:02:53.240
actually the person.

s32
00:02:53.240 --> 00:02:57.560
So they don't ask like, "Hey, can you tell me about some random application that they don't own?"

s33
00:02:57.560 --> 00:03:04.560
Um then sort of you may think that this is a very simple use case, but this is where sort of regulatory things come into the picture.

s34
00:03:04.560 --> 00:03:06.840
So I'll give a couple of examples.

s35
00:03:06.840 --> 00:03:12.800
If someone says that I plan to use the funds for medical purposes, we can't ask the amount.

s36
00:03:12.800 --> 00:03:14.280
We just prevented from doing that.

s37
00:03:14.280 --> 00:03:18.320
If we do ask it, it creates a big issue for us.

s38
00:03:18.320 --> 00:03:23.760
Um on the other hand, if someone says, "I plan to use the funds to travel to graduation."

s39
00:03:23.760 --> 00:03:24.200
Right?

s40
00:03:24.200 --> 00:03:25.760
Well, whose graduation is it?

s41
00:03:25.760 --> 00:03:26.640
Your graduation?

s42
00:03:26.640 --> 00:03:28.160
Your kids' graduation?

s43
00:03:28.160 --> 00:03:29.280
Someone else's?

s44
00:03:29.280 --> 00:03:31.600
That really determines the use of funds, right?

s45
00:03:31.600 --> 00:03:39.880
So depending on on sort of what the applicant or the human at the other end says, we have to do specific things.

s46
00:03:39.880 --> 00:03:43.080
We can ask or cannot ask things.

s47
00:03:43.080 --> 00:03:51.680
And lastly, I think the it was very important for my compliance team that when we handle this use case, we just handle this use case, right?

s48
00:03:51.680 --> 00:04:01.040
A lot of applicants are very eager to know what their status is or when they'll get the money because we can't just say like it'll be up to three business days.

s49
00:04:01.040 --> 00:04:04.400
We just sort of sidestep the question in a nice and polite way.

s50
00:04:04.400 --> 00:04:09.440
Um so that's that's sort of the use case that we launched.

s51
00:04:10.320 --> 00:04:10.760
Cool.

s52
00:04:10.760 --> 00:04:14.280
So um So that's the call.

s53
00:04:14.280 --> 00:04:21.440
Now obviously when we launched this we want to make sure that the agent was doing well what it set out to do.

s54
00:04:21.440 --> 00:04:25.560
Uh you know, so when we start audit auditing was very important.

s55
00:04:25.560 --> 00:04:28.880
So every call had to be reviewed by a human.

s56
00:04:28.880 --> 00:04:35.800
Um now obviously when this started out well how do you review all these calls?

s57
00:04:35.800 --> 00:04:38.760
So we started with the humble Google sheet.

s58
00:04:38.760 --> 00:04:40.600
You know, we start maintaining it.

s59
00:04:40.600 --> 00:04:46.360
Our operations team that were reviewing these calls started putting in data there.

s60
00:04:46.520 --> 00:04:59.880
In a few weeks five columns went to 10 then like 15 people started adding columns randomly putting stuff in columns creating things that you know, just just sort of messed up the whole process, right?

s61
00:04:59.880 --> 00:05:11.280
So ultimately like what we need is data to improve both our agent as well as you know, the uh the things that we look in terms of performance, right?

s62
00:05:11.280 --> 00:05:23.000
So that's where we sort of uh needed to have a better and more structured process for human review, right?

s63
00:05:23.000 --> 00:05:27.880
And that's where Kowal came in and helped us build this human review process.

s64
00:05:27.880 --> 00:05:31.760
Um but before I go there let me just talk a little bit about metrics, right?

s65
00:05:31.760 --> 00:05:39.400
So when I talk about metrics here think about LM as a judge running once a call is complete.

s66
00:05:39.400 --> 00:05:39.640
Right?

s67
00:05:39.640 --> 00:05:44.960
So one important like criteria is were the compliance requirements met?

s68
00:05:44.960 --> 00:05:48.960
So I it's a simple like yes no question with an explanation, right?

s69
00:05:48.960 --> 00:05:57.280
So, uh one of the things we did is post post run, we run this judge and it produces an output with an explanation.

s70
00:05:57.280 --> 00:06:00.960
Now, we want to make sure that the judge is working.

s71
00:06:00.960 --> 00:06:09.440
So, we we put it and and we get a human to review the output of the LLM judge.

s72
00:06:09.440 --> 00:06:20.560
Yeah, so when we started off, you know, we had metrics and we were tracking like a whole bunch of things like call time, you know, obviously accuracy, clarity, etc.

s73
00:06:20.560 --> 00:06:25.960
I think over time we sort of whittled down the list to the metrics that were sort of the most important, right?

s74
00:06:25.960 --> 00:06:35.600
And you can see some of the ones here like accuracy, clarity, completeness, policy alignment, whether it was escalated correctly, and ultimately CSAT score.

s75
00:06:35.600 --> 00:06:41.640
One important thing I'd say is don't be too attached to your metrics.

s76
00:06:41.640 --> 00:06:44.280
Start off with the ones that seem logical.

s77
00:06:44.280 --> 00:06:52.520
If things change over time, you know, discard those metrics and track the other ones that really like impact your use case.

s78
00:06:52.520 --> 00:06:54.600
Um Cool.

s79
00:06:54.600 --> 00:06:57.000
So, we talked about LLM as judge and metrics a little bit.

s80
00:06:57.000 --> 00:07:03.600
So, again, all these are run automatically once a call is complete.

s81
00:07:03.600 --> 00:07:09.000
Now, I'll talk about one specific one here which was, you know, adherence to compliance, right?

s82
00:07:09.000 --> 00:07:10.720
Or compliance violation.

s83
00:07:10.720 --> 00:07:14.360
So, basically we we score the call.

s84
00:07:14.360 --> 00:07:16.400
It goes into a queue.

s85
00:07:16.400 --> 00:07:21.760
A human reviews the output of the of the metric.

s86
00:07:21.760 --> 00:07:23.080
And then puts in their feedback, right?

s87
00:07:23.080 --> 00:07:32.560
Why this is important is because we want to make sure that whatever the judge is outputting aligns with humans or experts in that particular area, right?

s88
00:07:32.560 --> 00:07:39.919
And as you can see here like when we launched a while back, the compliance sort of agreement metric was 33%.

s89
00:07:39.919 --> 00:07:48.600
That means a human reviewer and a LLM agent or a LLM judge agreed only 33% of the time.

s90
00:07:48.600 --> 00:07:54.000
Now, well, we don't we we can't get to 100% but we can get like very close, right?

s91
00:07:54.000 --> 00:08:02.600
So, we really start looking into like what is the gap there and that's where this process of providing feedback by experts sort of really helped, right?

s92
00:08:02.600 --> 00:08:05.200
And ultimately it came down to a couple of things, right?

s93
00:08:05.200 --> 00:08:10.160
There was a lot of tribal knowledge when we launched the agent that was not coded as part of the agent.

s94
00:08:10.160 --> 00:08:15.919
So, we changed that and over time the the metric seems number went up.

s95
00:08:15.919 --> 00:08:22.040
Then we realized that amongst humans in who are reviewing this call there was disagreement as well, right?

s96
00:08:22.040 --> 00:08:26.919
Because there was a lot of stuff that was not written or or like tribal knowledge.

s97
00:08:26.919 --> 00:08:29.240
So, we tried to like encode that.

s98
00:08:29.240 --> 00:08:34.479
And then like sort of the next big bump came as we started to like look at more use cases.

s99
00:08:34.479 --> 00:08:38.960
We started to see that well, the judge itself needed to be improved, right?

s100
00:08:38.960 --> 00:08:45.971
So, ultimately we went up from like a 33% agreement rate to now today it's at 92%.

s101
00:08:45.971 --> 00:08:46.160
[clears throat]

s102
00:08:46.160 --> 00:08:50.600
Um we'll talk a little bit about how this process works in detail.

s103
00:08:50.600 --> 00:08:53.000
So, I'll hand it over to Brooke.

s104
00:08:53.000 --> 00:08:53.600
Amazing.

s105
00:08:53.600 --> 00:09:02.360
So, from there what we really implemented was this continuous this complete loop of how you continuously improve the agents.

s106
00:09:02.360 --> 00:09:10.760
So, I think this is something that Upstart has done really well is actually setting up a process that doesn't allow you to just launch once but allows you Um amazing.

s107
00:09:10.760 --> 00:09:12.480
I'm super happy this is useful.

s108
00:09:12.480 --> 00:09:14.400
Um so, everyone taking pictures.

s109
00:09:14.400 --> 00:09:17.960
Um but it really allows you to like continuously improve your agent.

s110
00:09:17.960 --> 00:09:28.600
I think this is something that enterprises get wrong is that they set up how whether the release gate so that we can launch one time but then it makes it very hard to change a model or change a prompt or anything underneath.

s111
00:09:28.600 --> 00:09:42.680
And so, we go from like um I'll step through all of these processes of how you go from monitoring conversations, defining what your metrics look like, flagging calls, and then using that to constantly improve your agent.

s112
00:09:42.680 --> 00:09:54.560
Um and so we also use this human review that Shiv just talked about and really believe that human review is a super important part of the evaluation process and where you should be spending most of your time is thinking about

s113
00:09:54.560 --> 00:09:57.480
the product like evals are really the product definition.

s114
00:09:57.480 --> 00:10:04.080
So where should you be spending your time um is what should my product be doing?

s115
00:10:04.160 --> 00:10:05.960
Um so the first step is defining your metrics.

s116
00:10:05.960 --> 00:10:07.880
So there are different types of evaluations.

s117
00:10:07.880 --> 00:10:11.600
Like I think about it in three categories of did it get to the end?

s118
00:10:11.600 --> 00:10:14.080
Did it adhere to the steps in order to get there?

s119
00:10:14.080 --> 00:10:17.440
And then what was the conversation quality or the experience?

s120
00:10:17.440 --> 00:10:21.840
Um and so in Cobal, and I'll be showing you how this actually works in Cobal.

s121
00:10:21.840 --> 00:10:29.640
So you can build out these LLM judge or metrics that align with this original criteria that human reviewers were going through and labeling.

s122
00:10:29.640 --> 00:10:38.240
That's also a really important step though is to have these human reviewers labeling what do they care about and then you can take that and turn it into automated evals

s123
00:10:38.240 --> 00:10:44.360
versus the reverse is starting with like we care about word error rate but that's not actually what we're actioning on.

s124
00:10:44.360 --> 00:10:50.400
Um and then the types of metrics are you know LLM as a judge, heuristics, audio processing.

s125
00:10:50.560 --> 00:10:55.720
Um and so what we do is we monitor then take these metrics to monitor live calls.

s126
00:10:55.720 --> 00:10:58.880
And so we can detect known failure patterns at scale.

s127
00:10:58.880 --> 00:11:09.000
So being able to show exactly where for example we didn't verify um the correct pieces for the loan um or maybe where it didn't have the wording correct.

s128
00:11:09.000 --> 00:11:14.640
Um and this is really important because no system works uh 100% of the time at scale.

s129
00:11:14.640 --> 00:11:18.920
Uh even Google you know six nines of reliability there will will be failures.

s130
00:11:18.920 --> 00:11:21.160
Um and then we also detect unknown unknowns.

s131
00:11:21.160 --> 00:11:28.560
So things that we didn't know were going to happen doing by doing like clustering analysis and anomaly detection.

s132
00:11:29.480 --> 00:11:33.320
Um and then we can route each one of these failures for human review.

s133
00:11:33.320 --> 00:11:44.040
So you can configure review queues where they're actioning on any one of these metrics that we defined and then having these amazing human reviewers look through those failures.

s134
00:11:44.040 --> 00:11:50.080
So now instead of looking at every call, they're only looking at the calls that matter most.

s135
00:11:51.040 --> 00:11:55.880
And so from there we route all of these different failures to the right teams.

s136
00:11:55.880 --> 00:12:04.560
This is a really critical step because you want to set up an operational practice so that every failure that's coming in you're doing something with that.

s137
00:12:04.560 --> 00:12:09.720
So either you're routing it to you're routing for human review to detect real failures.

s138
00:12:09.720 --> 00:12:14.120
So let's say like you know off script or wrong answer is 34% of the time.

s139
00:12:14.120 --> 00:12:15.760
These are just made up numbers.

s140
00:12:15.760 --> 00:12:30.840
Um then like maybe we send those to the agent team versus compliance violations we send that to a compliance team so that they can file a process or you know take some like tactical action on it versus sending to an engineering team etc.

s141
00:12:31.640 --> 00:12:42.480
Um and so now once we have a set of real failures, we can then re-simulate or we then for example Steven who's here can figure out what is the right fix to the agent.

s142
00:12:42.480 --> 00:12:49.200
Um and then we can re-simulate that that fix to see what exactly the result is.

s143
00:12:49.200 --> 00:12:52.280
Like did it actually fix this issue?

s144
00:12:52.520 --> 00:12:56.080
Um and so our simulations do voice to voice evaluation.

s145
00:12:56.080 --> 00:13:13.520
We will call your agent and actually run a simulation to um you know test different background noises, different accents, um the actual voice quality, interruptions, etc. And then we can verify that that improvement actually fixed things before it's impacting your users.

s146
00:13:13.520 --> 00:13:25.160
So once you know that there's an issue like a compliance violation or maybe like a poor user experience, you don't want to launch that again to production to see if it fixed it because you're now risking impact to your users.

s147
00:13:25.160 --> 00:13:28.560
So we want to shift left to use a devops term.

s148
00:13:28.560 --> 00:13:33.680
Um we want to detect as many failures as we can in simulation instead of doing that in production.

s149
00:13:33.680 --> 00:13:39.480
So, here you can see that, you know, we improved um those cases in the report.

s150
00:13:39.480 --> 00:13:47.600
And so, this finally completes the loop of um we monitor we define what our metrics are based on our product definition.

s151
00:13:47.600 --> 00:13:53.320
This is honestly where you should spend the most of your time is what does my agent do and what should it not do.

s152
00:13:53.320 --> 00:14:04.240
Um and then we can monitor and score every live call, um detect those failures, send them for human review, find failure patterns across all of these to prioritize engineering work.

s153
00:14:04.240 --> 00:14:14.120
And one of the most exciting things is now you can enable the self-healing loop where we can actually have agents that look at all of these evaluations and identify fixes,

s154
00:14:14.120 --> 00:14:23.320
but we can also use RL pipelines or fine-tuned LLMs in order to make our system even better over time automatically.

s155
00:14:23.320 --> 00:14:26.560
Um and then actually I have to go back.

s156
00:14:26.560 --> 00:14:27.280
Okay.

s157
00:14:27.280 --> 00:14:36.920
Um and then from there we can automate fixes and then prove that this actually is improving things in production through simulation.

s158
00:14:36.920 --> 00:14:42.839
And all of these things obviously are gated by human review and integrating human review in the loop.

s159
00:14:42.839 --> 00:14:48.960
Um and so, these defined setups can uh both make sure our metrics are really accurate.

s160
00:14:48.960 --> 00:14:59.839
So, as I should have talked through, every time you label a feedback every time you label a metric as it was being as it was incorrect, now you can align your metrics to human judgment

s161
00:14:59.839 --> 00:15:06.560
or it it actually is a true failure, in which case you can use that to automatically improve your agents.

s162
00:15:06.560 --> 00:15:08.560
Um

s163
00:15:09.640 --> 00:15:10.520
Cool.

s164
00:15:10.520 --> 00:15:12.760
Um so, yeah, in conclusion, right?

s165
00:15:12.760 --> 00:15:13.839
Evals are important.

s166
00:15:13.839 --> 00:15:15.600
Don't don't skip them.

s167
00:15:15.600 --> 00:15:28.480
Uh evals uh sort of help us uh sort of monitor and score the calls, but also determine which calls need attention or which calls need to be like reviewed by a human because humans can't review like thousands of calls a day.

s168
00:15:28.480 --> 00:15:35.160
Um, the human reviewer annotations are important because they help improve the LM judge.

s169
00:15:35.160 --> 00:15:39.840
Um, all this data that we collect ultimately helps improve the agent as well.

s170
00:15:39.840 --> 00:15:44.200
Um, changes are always gated behind a feature flag, right?

s171
00:15:44.200 --> 00:15:50.040
So, we when we say self-healing or self-improving, it doesn't mean like humans are completely out of the loop.

s172
00:15:50.040 --> 00:15:58.360
Uh, with uh, fintech, uh, we we need these, uh, changes to be sort of monitored and and signed on by like someone, right?

s173
00:15:58.360 --> 00:16:01.160
So, judges usually the engineers do.

s174
00:16:01.160 --> 00:16:07.480
Uh, but if uh, there was a compliance change, then compliance is involved in approving like, uh, the behavior of the agent.

s175
00:16:07.480 --> 00:16:16.760
Um, and lastly, like this has served like as a blueprint for us to launch agents safely meeting regulatory requirements and also improving their performance at the same time.

s176
00:16:16.760 --> 00:16:24.360
And I think that's why like actually being able to set up really robust evals defines a really high-quality product.

s177
00:16:24.360 --> 00:16:34.240
Um, in all cases, but I think especially in regulated industries, you just can't, um, deploy the applications that you want to without having really robust evals that give you confidence

s178
00:16:34.240 --> 00:16:43.720
that you're doing it like deploying these autonomous systems with precision and robust, reliable, um, you know, great products that can actually scale.

s179
00:16:43.720 --> 00:16:44.680
Cool.

s180
00:16:44.680 --> 00:16:45.880
Uh, we are almost at time.

s181
00:16:45.880 --> 00:16:53.800
So, yeah, feel free to like, uh, you know, connect with us on LinkedIn or you know, just reach out if you, uh, see us, uh, anywhere, uh, today.

s182
00:16:53.800 --> 00:16:57.600
So, but that's, uh, that's our presentation.