WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=qStB9GbppMU

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/qStB9GbppMU.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.800 --> 00:00:17.600
My name is Suman Yu and I'm the founder and CEO of Hamming.

s3
00:00:17.600 --> 00:00:25.920
And before working on voice agent reliability and safety, I worked at a company called Citizen out of New York.

s4
00:00:25.920 --> 00:00:28.720
Anybody here use Citizen app?

s5
00:00:28.720 --> 00:00:30.080
Awesome.

s6
00:00:30.080 --> 00:00:31.199
Thank you.

s7
00:00:31.199 --> 00:00:50.399
Uh, and at Citizen, we listened to crime, thousands of hours of police radio station data and sent millions of alerts to users in San Francisco, New York, LA, uh, Chicago, Baltimore, and so on.

s8
00:00:50.399 --> 00:00:53.760
Some obviously gory and pretty sad.

s9
00:00:53.760 --> 00:01:06.559
Uh, but others more funny like a person stealing bags of ice cream from Safeway or report of a man hanging off the side of the house after a woman stole his ladder.

s10
00:01:07.680 --> 00:01:16.080
If I actually take a look at the citizen app right now for those who are customers or users, I can see that there is a man yelling at person.

s11
00:01:16.080 --> 00:01:17.520
There's indecent exposure.

s12
00:01:17.520 --> 00:01:18.159
This is real.

s13
00:01:18.159 --> 00:01:19.040
This is real time.

s14
00:01:19.040 --> 00:01:22.240
This is, you know, couple hours ago.

s15
00:01:22.240 --> 00:01:25.920
These are real-time alerts that we're sending.

s16
00:01:27.439 --> 00:01:33.119
Now, voice agents scare me more because they're finally graduating from demos and PC's to production.

s17
00:01:33.119 --> 00:01:34.960
We should be super excited, but I'm nervous.

s18
00:01:34.960 --> 00:01:36.560
I'm personally nervous.

s19
00:01:36.560 --> 00:01:42.640
Uh, they're talking to users at a scale that would make Gary Tan and Polygram proud.

s20
00:01:43.759 --> 00:01:49.920
When I got started in voice agent reliability in early 2024, voice was just starting to work.

s21
00:01:49.920 --> 00:01:53.040
It was not quite good yet, but it was just starting to work.

s22
00:01:53.040 --> 00:01:59.680
You would have to pay me a lot of money for me to stop using, you know, Aqua voice, Super Whisper, uh, Whisper Flow, and so on.

s23
00:01:59.680 --> 00:02:01.759
These products are just getting super, super good.

s24
00:02:01.759 --> 00:02:08.319
And a big reason is because the underlying infrastructure is getting better, and the orchestration layer is getting meaningfully better.

s25
00:02:08.319 --> 00:02:18.160
It's getting much faster to build products and voice experiences that maybe are 60% good in a pretty short period of time, but the long tail is still Hey, Gorov.

s26
00:02:18.160 --> 00:02:21.520
the long tail is still uh wise away.

s27
00:02:21.520 --> 00:02:23.840
I think speech speech models are getting better.

s28
00:02:23.840 --> 00:02:39.360
Um teams are experiment experimenting with hybrid architectures of combining more voicetovoice modalities and also cascading stacks to make the experience reliable but still pretty low latency.

s29
00:02:39.599 --> 00:02:41.200
Things are obviously getting better.

s30
00:02:41.200 --> 00:02:46.400
Agents are being connected to calendars, CRM, HRs, reservation systems, and so on.

s31
00:02:46.400 --> 00:02:49.599
Voice agents can now take actions.

s32
00:02:50.160 --> 00:02:55.440
However, reliability is still the number one problem holding back most voice agent deployments at scale.

s33
00:02:55.440 --> 00:02:57.519
This is still the number one problem.

s34
00:02:57.519 --> 00:03:09.200
This is an example I found on Twitter pretty randomly, you know, two weeks ago and a person is trying to get information for a tradein and gets absolutely confused with the information that they're receiving.

s35
00:03:09.200 --> 00:03:17.599
Alex now has to correct for this loss of trust but trying to, you know, call the person and see what see what happened and fix the situation.

s36
00:03:17.599 --> 00:03:20.640
Let me see if audio works here.

s37
00:03:20.640 --> 00:03:22.560
Screwed up with another customer.

s38
00:03:22.560 --> 00:03:25.440
We're getting it fixed, but I got to call him and see if I can work it out.

s39
00:03:25.440 --> 00:03:28.640
I'm like, dude, half the time I'm like, I don't know if I'm talking to AI.

s40
00:03:28.640 --> 00:03:29.920
I don't know if I'm talking to a person.

s41
00:03:29.920 --> 00:03:32.000
It was just confusing, but we got there.

s42
00:03:32.000 --> 00:03:34.480
It probably is AI and human.

s43
00:03:34.480 --> 00:03:36.480
And

s44
00:03:37.040 --> 00:03:39.120
so I think voices sound very confident.

s45
00:03:39.120 --> 00:03:43.280
They sound very natural, but the information provided is often, you know, not correct.

s46
00:03:43.280 --> 00:03:46.000
That's the biggest problem here.

s47
00:03:46.000 --> 00:03:47.519
This example is more personal.

s48
00:03:47.519 --> 00:03:52.640
I had booked an appointment with a physician a couple weeks ago or I thought I did.

s49
00:03:52.640 --> 00:03:57.040
I showed up to the appointment and turns out I was not actually on the schedule.

s50
00:03:57.040 --> 00:03:59.280
So the front desk, you me turned me away.

s51
00:03:59.280 --> 00:04:01.280
I wasted 2 hours.

s52
00:04:01.280 --> 00:04:02.959
For me, this was a waste of time.

s53
00:04:02.959 --> 00:04:06.159
But what if this was actually your parent?

s54
00:04:06.159 --> 00:04:08.799
What if this was your grandparent?

s55
00:04:08.799 --> 00:04:13.120
What if this appointment was for a procedure instead of a regular checkup?

s56
00:04:13.120 --> 00:04:20.000
The costs for these different permutations of the same failure mode can actually be super super high.

s57
00:04:20.079 --> 00:04:22.960
Now, let's compare crime to voice agents.

s58
00:04:22.960 --> 00:04:27.919
Um, I think observation number one is crime is actually decreasing over time.

s59
00:04:27.919 --> 00:04:34.960
This is a good thing and I hope it crosses the x- axis at some point you know in the future.

s60
00:04:35.280 --> 00:04:41.919
Voice on the other hand is generally taking off right we're seeing a pretty fast takeoff of voice agents being deployed in production.

s61
00:04:41.919 --> 00:04:50.240
There's at least a trillion calls that are done every single year and majority of these will be done by conversational voice agents over the next you know five years.

s62
00:04:50.240 --> 00:04:54.960
If you assume a 1% error rate that is still 10 billion incidents per year.

s63
00:04:54.960 --> 00:04:57.440
That's a lot.

s64
00:04:58.080 --> 00:05:05.520
In practice, we currently monitor 10,000 agents and the error rate is closer to 10% in practice.

s65
00:05:05.520 --> 00:05:18.000
These range from agents saying they found the right policy when they actually skipped the eligibility or verification steps or applying discounts when they were not really supposed to, misharing what the person said, providing incorrect information,

s66
00:05:18.000 --> 00:05:23.440
or claiming they booked an appointment when they actually did not, just like it happened for me.

s67
00:05:23.440 --> 00:05:28.240
Now, not every single call has an equally, you know, bad cost.

s68
00:05:28.240 --> 00:05:36.880
Uh, some range, you know, in the crime land, some range from trash fires, which are kind of funny, annoying, not really hurting somebody.

s69
00:05:36.880 --> 00:05:44.000
For a voice equivalent, that would be annoyances like repetition, um, or just sort of not quite understanding what the user is saying.

s70
00:05:44.000 --> 00:05:55.600
all the way to safety risks like mass shootings or in the voice agent equivalent, it would be um a drive-thru that's deploying um voice agents at scale like a Taco Bell or McDonald's

s71
00:05:55.600 --> 00:05:59.919
and a person orders a vegan burger with peanut allergies.

s72
00:05:59.919 --> 00:06:07.120
If one of those two situations are not handled correctly, that is definitely a safety concern at scale.

s73
00:06:07.759 --> 00:06:19.759
The other big difference between crime and and voice agent deployments is is crime generally tends to be pretty hyper local, tends to be very decentralized, right?

s74
00:06:19.759 --> 00:06:24.560
Things like robbery or motor vehicle theft or lararseny.

s75
00:06:24.560 --> 00:06:30.880
They're impacting a finite set of individuals that are involved in that um situation.

s76
00:06:32.639 --> 00:06:35.919
On the other hand, voice agents are much more centralized.

s77
00:06:35.919 --> 00:06:46.639
a single prompt change or an architecture change can have pretty massive implications downstream for all of the millions of you know users that are um in the crossfire.

s78
00:06:46.639 --> 00:06:49.840
So the blast radius is is quite quite massive.

s79
00:06:49.840 --> 00:06:53.680
So the natural question is how do you make these incidents much more visible and obvious?

s80
00:06:53.680 --> 00:06:57.039
That's the kind of obvious question here.

s81
00:06:57.039 --> 00:07:02.319
I'll borrow a framework from a couple of my friends who were OG growth folks at Facebook.

s82
00:07:02.319 --> 00:07:09.759
So step one is to identify okay what are all the challenges and problems that um exist in your conversation experience.

s83
00:07:09.759 --> 00:07:11.840
Step two is to prioritize an impact size.

s84
00:07:11.840 --> 00:07:15.759
There's a frequency and severity analysis that's pretty important.

s85
00:07:15.759 --> 00:07:18.880
Step three is to understand okay how do we actually fix this?

s86
00:07:18.880 --> 00:07:20.160
Step four execute.

s87
00:07:20.160 --> 00:07:25.759
Step five okay did my change actually work and did it cause any regressions somewhere else.

s88
00:07:25.759 --> 00:07:29.680
And lastly we continue to monitor in production.

s89
00:07:30.080 --> 00:07:36.080
On the y- axis, I think it's important to highlight there are known problems that already exist.

s90
00:07:36.080 --> 00:07:41.759
Things like turnover latency, interruptions, um maybe some ASR problems you're, you know, aware of.

s91
00:07:41.759 --> 00:07:46.880
And these are known problems that exist that the team should track over time.

s92
00:07:46.880 --> 00:07:54.240
On the other axis is actually emerging behavior or patterns that are only obvious across lots of conversations.

s93
00:07:54.240 --> 00:07:58.800
Um on the x- axis, you have coverage just like insurance.

s94
00:07:58.800 --> 00:08:01.039
Are you analyzing few conversations?

s95
00:08:01.039 --> 00:08:03.840
Are you analyzing many, many conversations?

s96
00:08:03.840 --> 00:08:08.319
Most teams will typically start by listening to calls manually.

s97
00:08:08.400 --> 00:08:10.319
And I think that's the best place to start.

s98
00:08:10.319 --> 00:08:12.240
I don't think you should skip that step.

s99
00:08:12.240 --> 00:08:20.479
There's a lot of depth and insights to get by actually listening to specific conversations and building that texture that that comes from that intuition.

s100
00:08:20.479 --> 00:08:22.800
However, it's obviously not scalable.

s101
00:08:22.800 --> 00:08:34.479
So most teams end up having a spreadsheet of I don't know five or 10 different rubrics around greetings, closing, validation, um, core logic and so on.

s102
00:08:34.479 --> 00:08:39.440
To scale that up even further, you then end up investing in some eval product, right?

s103
00:08:39.440 --> 00:08:48.160
You might run some element as a judge and compute classic metrics and also more more deterministic and stoastic scoring logic.

s104
00:08:48.160 --> 00:08:58.000
Um, but there you're still stuck with checking for consistency of known problems, but you're not really discovering novel insights that are actually happening across conversations.

s105
00:08:58.000 --> 00:09:05.760
We're spending a ton of time on performing cross conversation analysis, not a pattern on a single call, but across conversations.

s106
00:09:05.760 --> 00:09:10.399
And some of the best teams that we work with are are doing the same.

s107
00:09:10.800 --> 00:09:15.839
Now, to prioritize an impact size, I think there's problems that are one-off that are low impact.

s108
00:09:15.839 --> 00:09:17.440
I mean, who cares?

s109
00:09:17.440 --> 00:09:26.880
uh even low impact and systematic problems in the crime world that would be a trash fire in a voice aation world it could be some repetitions the team is experiencing

s110
00:09:26.880 --> 00:09:37.120
they're still annoying at scale and if you are doing a bake off it's still worth solving for them I would not ignore these class of problems oneoff and high impact well hope it doesn't chronic

s111
00:09:37.120 --> 00:09:48.399
and I think systematic and high impact are obviously the P 0 you know target areas um for the team to solve an example of that would be in a fins serve capacity There's a voice agent

s112
00:09:48.399 --> 00:09:51.519
that um helps users freeze their credit cards.

s113
00:09:51.519 --> 00:09:55.440
And if it doesn't do that, well, that's a massive fail.

s114
00:09:56.560 --> 00:09:56.880
All right.

s115
00:09:56.880 --> 00:09:58.399
So, understand and execute.

s116
00:09:58.399 --> 00:10:00.000
I'm pretty sure everyone's doing this.

s117
00:10:00.000 --> 00:10:01.360
Please fix my agent.

s118
00:10:01.360 --> 00:10:12.160
Uh I think fixing or rather attempting to make a fix is the simplest and the lowest effort component of this debugging pipeline and loop.

s119
00:10:12.160 --> 00:10:16.000
Um the next step is all right, I made a change to my system.

s120
00:10:16.000 --> 00:10:20.000
How do I actually know this thing works um for real?

s121
00:10:20.000 --> 00:10:30.480
A great way that's naive is to take a real call, for example, in my case, I booked an appointment and it didn't get scheduled and replay that exact conversation

s122
00:10:30.480 --> 00:10:38.720
and run that maybe 5, 10, 20, 50 times and see, okay, what is my probability of passing this type of issue?

s123
00:10:38.720 --> 00:10:48.959
A better way is to keep the same intent but change the wordings, change the patterns, change the accents, change the style, add one more intent to the mix.

s124
00:10:48.959 --> 00:11:00.720
And that gives teams much more, you know, better coverage to feel confident that yes, I actually made a change and my changes are net positive instead of net negative.

s125
00:11:01.600 --> 00:11:10.320
There are certain fixes and I guess hypothesis that are very difficult to test in a pre-eployment synthetic setting.

s126
00:11:10.320 --> 00:11:14.720
And so AB testing ends up being, you know, pretty pretty critical for those circumstances.

s127
00:11:14.720 --> 00:11:21.120
For example, if you have an outbound agent, the first 5 seconds of a conversation tends to be the most important.

s128
00:11:21.120 --> 00:11:27.519
And so the vocal quality um and the specific words you end up using, they matter the most.

s129
00:11:27.519 --> 00:11:32.880
And so AB testing that is the only way in in kind of real life setting to to get results.

s130
00:11:32.880 --> 00:11:36.640
You can't really do it through simulations alone.

s131
00:11:37.200 --> 00:11:38.959
And so there we have the loop.

s132
00:11:38.959 --> 00:11:49.360
Identify, prioritize, impact size, understand the fix, execute, check, make sure it didn't break anything, and then continue monitoring.

s133
00:11:51.920 --> 00:11:57.360
So I think making voice agents useful is already hard as it is.

s134
00:11:57.360 --> 00:12:01.200
even when dealing with earnest users on the other line, right?

s135
00:12:01.200 --> 00:12:03.360
These are people who who just want their problem solved.

s136
00:12:03.360 --> 00:12:04.399
They're not trying to mess with you.

s137
00:12:04.399 --> 00:12:08.160
These are like legit normal people.

s138
00:12:08.639 --> 00:12:12.480
Now, what happens when mythos learns how to dial?

s139
00:12:13.600 --> 00:12:26.079
So, if it can extract trade secrets and uh you know, from the NSA, it can certainly, you know, seduce you into revealing PHI and PII data as well.

s140
00:12:26.959 --> 00:12:31.120
And I think both voice agents and humans will [clears throat] be targeted here.

s141
00:12:31.120 --> 00:12:36.800
Voice agents because there's a pressure to make these more capable.

s142
00:12:36.800 --> 00:12:38.399
Give them access to more data.

s143
00:12:38.399 --> 00:12:41.360
Give them access to more tools.

s144
00:12:41.440 --> 00:12:44.079
Deploy them quickly.

s145
00:12:45.279 --> 00:12:47.440
The more the capability, the bigger the surface area.

s146
00:12:47.440 --> 00:12:50.240
This is this is pretty pretty common sense.

s147
00:12:50.240 --> 00:13:00.880
And the more the voice agents become natural and human sounding, the more humans will be tricked along the way as well for those who are weaponizing.

s148
00:13:01.680 --> 00:13:11.519
Uh we ship a we shipped a red tipping product um back in April just to test out this hypothesis for how many agents can we actually break from a adversarial capacity

s149
00:13:11.519 --> 00:13:14.160
and we can probably break one in five agents at this point.

s150
00:13:14.160 --> 00:13:19.519
We've tested this across financial services, healthcare, um consumer and so on.

s151
00:13:19.519 --> 00:13:21.519
We've bypassed verification.

s152
00:13:21.519 --> 00:13:29.279
Uh we've definitely had agents, you know, we've been able to promject uh several agents and and gotten data we should not have.

s153
00:13:29.279 --> 00:13:30.800
So this is not theoretical.

s154
00:13:30.800 --> 00:13:34.560
This is actually a real a real concern.

s155
00:13:37.519 --> 00:13:45.360
I think the only real defense against the dark arts is step one to invest deeply in pre-eployment testing.

s156
00:13:45.360 --> 00:13:47.040
This could be textto text.

s157
00:13:47.040 --> 00:13:48.720
This could be voice to voice.

s158
00:13:48.720 --> 00:13:50.079
There's pros and cons to both.

s159
00:13:50.079 --> 00:13:53.040
Happy to chat offline if folks are interested.

s160
00:13:53.040 --> 00:13:59.600
And this is just making sure you're not self-owning, you know, when you're talking to real people who just want to get their problem solved.

s161
00:13:59.600 --> 00:14:03.519
Step two is to have a great monitoring system of all kinds.

s162
00:14:03.519 --> 00:14:13.440
And I've highlighted, you know, different flavors of monitoring per call scoring, manual kind of evals, you know, listening to conversations and cross call analysis.

s163
00:14:13.440 --> 00:14:19.600
And this is helpful both for monitoring what the agent is saying and behaving and how it's actually doing, but also the users.

s164
00:14:19.600 --> 00:14:21.040
Are the users being adversarial?

s165
00:14:21.040 --> 00:14:21.839
Are they being annoying?

s166
00:14:21.839 --> 00:14:26.639
Are are they trying to trick the agent into doing things it's not supposed to be doing?

s167
00:14:26.639 --> 00:14:38.399
And I think our new recommendation now is to run 24/7 red teaming um for your agents, especially if you believe the cost of bad interactions can be can be rather large.

s168
00:14:38.800 --> 00:14:52.079
So, I think voice agents um have this awesome potential of of making the world feel much more human compared to interacting with clunky IVR trees or chat bots or worse

s169
00:14:52.079 --> 00:14:55.040
um being stuck on a on a hold.

s170
00:14:55.040 --> 00:15:00.320
And when we think about crime, we often think of crime happening to somebody else.

s171
00:15:00.320 --> 00:15:06.560
You know, crime does not happen to you typically with voice agents, especially bad actors.

s172
00:15:06.560 --> 00:15:15.120
As these agents are deployed and as bad actors start to exploit a lot of the vulnerabilities, the number of incidents is about to kind of go way way up.

s173
00:15:15.120 --> 00:15:21.279
And so the reason I fear voice agents more than crime is that one of these incidents is going to impact you.

s174
00:15:21.279 --> 00:15:24.240
It already did for me.

s175
00:15:24.240 --> 00:15:24.639
Awesome.

s176
00:15:24.639 --> 00:15:25.920
So it's time for me to shill.

s177
00:15:25.920 --> 00:15:27.440
Well, we burn a lot of tokens.

s178
00:15:27.440 --> 00:15:32.959
If you are interested in working in this space, please come and talk to us.

s179
00:15:32.959 --> 00:15:41.839
And if you are deploying voice agents and want to validate whether your architectured or your eval are set up correctly, please come and talk to us.

s180
00:15:41.839 --> 00:15:42.639
We'll be outside.

s181
00:15:42.639 --> 00:15:43.760
And here's here's my number.

s182
00:15:43.760 --> 00:15:44.720
Here's my WhatsApp.

s183
00:15:44.720 --> 00:15:47.040
Thanks everyone.

s184
00:16:02.206 --> 00:16:04.206
[music]