WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=u6q-byPWUuo

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/u6q-byPWUuo.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.960 --> 00:00:14.639
Thank you so much for everyone being here.

s3
00:00:14.639 --> 00:00:16.240
We're going to get started in a second.

s4
00:00:16.240 --> 00:00:19.119
Um, but before we get started, I am curious.

s5
00:00:19.119 --> 00:00:23.199
How many of you currently work in the healthcare industry in some shape or form?

s6
00:00:23.199 --> 00:00:24.480
Oh, that's amazing to hear.

s7
00:00:24.480 --> 00:00:28.240
Uh, how many of you are clinicians by training?

s8
00:00:28.240 --> 00:00:28.880
Okay, a couple.

s9
00:00:28.880 --> 00:00:31.679
How many people in the room are engineers?

s10
00:00:31.679 --> 00:00:32.800
Okay, awesome.

s11
00:00:32.800 --> 00:00:36.960
Um, and then how many people have heard of a bridge before?

s12
00:00:36.960 --> 00:00:38.559
Okay, awesome.

s13
00:00:38.559 --> 00:00:47.120
Uh, well, I'm going to let you hear actually from our users to start off on a little about a bridge.

s14
00:00:47.815 --> 00:00:49.815
[music]

s15
00:00:53.840 --> 00:00:56.985
Full day of 22 patients out by 4:30 p.m.

s16
00:00:56.985 --> 00:00:57.120
[music]

s17
00:00:57.120 --> 00:00:58.079
Notes done.

s18
00:00:58.079 --> 00:01:00.320
That's nice.

s19
00:01:01.760 --> 00:01:08.320
When I think about a bridge, I think the thing that comes to mind is it's really a cornerstone of how I practice medicine today.

s20
00:01:08.320 --> 00:01:14.640
Um there's just no way um I would do a clinic or see a patient without using.

s21
00:01:14.640 --> 00:01:16.640
I can be present throughout my clinical encounters.

s22
00:01:16.640 --> 00:01:19.360
I don't have to think about um oh wait, did I get that?

s23
00:01:19.360 --> 00:01:20.320
Do I need to write that down?

s24
00:01:20.320 --> 00:01:25.200
Because I know a bridge has my back and has everything ready for me.

s25
00:01:25.680 --> 00:01:35.040
A bridge makes me feel free because I can really look at a patient, really listen, and not have to be thinking about what do I need to put in the computer.

s26
00:01:38.251 --> 00:01:40.251
[music]

s27
00:01:45.600 --> 00:01:46.111
Fire.

s28
00:01:46.111 --> 00:01:48.111
[music]

s29
00:02:08.640 --> 00:02:12.319
Our marketing team produces really good videos and so they always hype me up.

s30
00:02:12.319 --> 00:02:18.319
Um but the goal of this talk for me and I know that we have a lot of engineers in the room.

s31
00:02:18.319 --> 00:02:32.000
My my goal is to talk about healthcare as a domain at least I felt in the past that there was a lot of stigma around maybe the technical problems weren't as interesting in healthcare and it is true in some ways there's some parts of healthcare that might not be as tech forward a lot of things run on

s32
00:02:32.000 --> 00:02:45.120
fax machines for example um but I want to give exposure throughout this talk of two things one a bridg's journey from clinical documentation to clinical intelligence and what that looks like

s33
00:02:45.120 --> 00:02:57.280
um and then two I want to expose you to some of the technical ical problems we work on um and that have to be that are truly frontier AI product uh problems that have the highest stakes.

s34
00:02:57.599 --> 00:02:59.440
A little about me, my name is Chaitana.

s35
00:02:59.440 --> 00:03:00.720
You can call me Chai.

s36
00:03:00.720 --> 00:03:04.080
Um my career has always been in AI companies and startups.

s37
00:03:04.080 --> 00:03:11.760
I first started as in research engineering at a company called Vicarius uh which his goal was actually to develop AGI but they took very different methods.

s38
00:03:11.760 --> 00:03:16.480
They wanted methods inspired by neuroscience and probabilistic graphical models.

s39
00:03:16.480 --> 00:03:19.280
um and they concretely worked on robotics.

s40
00:03:19.280 --> 00:03:24.000
uh then I started working at this company called Glean because I faced this problem in my workplace itself.

s41
00:03:24.000 --> 00:03:40.480
How uh like information scattered all over the place context is everywhere and it's so key to decision-m and Glean was building basically the chat GPT for your workplace as about six and a half years as one of the earliest engineers as we went from 10 people to over 1100 people and work with some of the largest companies

s42
00:03:40.480 --> 00:03:41.760
all over the world.

s43
00:03:41.760 --> 00:03:43.599
Um but that journey was amazing.

s44
00:03:43.599 --> 00:03:44.400
I love that product.

s45
00:03:44.400 --> 00:03:45.360
I love that company.

s46
00:03:45.360 --> 00:03:46.799
I love the people there.

s47
00:03:46.799 --> 00:03:50.560
Um, but I've actually always really been interested in healthcare.

s48
00:03:50.560 --> 00:03:55.680
I remember a decade ago cover of Nature magazine was AI to detect skin cancer.

s49
00:03:55.680 --> 00:03:57.439
I was like, wow, is someone interested in AI?

s50
00:03:57.439 --> 00:03:58.319
That was amazing.

s51
00:03:58.319 --> 00:04:07.040
At the same time, I went to the hospital for something and I remember seeing, oh, coming back, it was a really minor thing, but I remember coming back and looking at the bill and I was like,

s52
00:04:07.040 --> 00:04:09.280
I'm not really sure what exactly I paid for.

s53
00:04:09.280 --> 00:04:16.239
Um and so there's these known problems in healthcare and I'll talk about some of them access to care and cost and then we had these amazing solutions.

s54
00:04:16.239 --> 00:04:18.720
So I was like why don't we bridge these things together.

s55
00:04:18.720 --> 00:04:20.959
Um and remember this is about a decade ago.

s56
00:04:20.959 --> 00:04:32.479
Um and so I actually started this seminar where I invited speakers who were physicians, researchers, entrepreneurs to to talk about the space and what I learned was while there was really really cool technology

s57
00:04:32.479 --> 00:04:36.479
very little of it made its way into the clinic at that time.

s58
00:04:36.479 --> 00:04:50.400
And so again my my journey went a different way but as I as I peaked my head out 10 years 10 years later then actually our technology has gotten better than ever as everyone knows and the AI wave as it's taken over the whole world has also influenced healthcare

s59
00:04:50.400 --> 00:04:58.240
and as you as you saw towards the end of that video a bridge in the matter of two to three years got its way into 300 of the largest health systems

s60
00:04:58.240 --> 00:05:10.320
uh in the United States Kaiser Mayo John Hopkins Sutter and so forth and maybe maybe you visited some of these hospital systems And once you're inside the hospital systems, you realize there's so so much more you can do.

s61
00:05:10.320 --> 00:05:13.280
And I'll talk about that journey that we've had.

s62
00:05:13.280 --> 00:05:25.440
I specifically work on um lead our engineering teams for clinical decision support and our agentic experiences that the technology is now enabled and how we can bring that to healthcare.

s63
00:05:25.919 --> 00:05:30.479
But first maybe maybe some of the problems that inspire us as a company at a bridge.

s64
00:05:30.479 --> 00:05:43.600
One, one of the things that we've noticed uh or many people econom e economists have noticed over the past few decades is actually in many other industries you actually see the cost of a good go down and that's because the productivity has increased.

s65
00:05:43.600 --> 00:05:50.000
But in healthcare we actually see administrative costs have only gone up over the past few decades.

s66
00:05:50.000 --> 00:05:53.039
Um and productivity hasn't necessarily increased.

s67
00:05:53.039 --> 00:05:59.039
Um and a lot of our problems in healthcare we solve with labor but even that we cannot actually keep up.

s68
00:05:59.039 --> 00:06:12.044
So there's this bit of this like productivity paradox you might have heard of like Balo's cost disease and it's part of partially because technology I think hasn't fully touched healthcare as much as it's touched other industries to increase that productivity

s69
00:06:12.044 --> 00:06:29.360
[clears throat] a few other problems you know hospitals are shutting down margins are actually razor thin for many health systems of course some patients have massive uh medical uh debt and then we actually and the problem that's another problem that's very near and dear to our heart is that we hear all the time that doctors are burnt

s70
00:06:29.360 --> 00:06:36.560
out and they actually often don't recommend it as a profession to uh to their children.

s71
00:06:36.639 --> 00:06:42.000
So what we started as as a company was working on clinical documentation.

s72
00:06:42.000 --> 00:06:48.800
So the idea here if you're not familiar with it is at at the end of every patient visit the the clinician must create a note.

s73
00:06:48.800 --> 00:06:57.360
Uh our typical format is the SOAP note that has a couple different formats like what's the chief complaint of the patient and a few other sections and then what's the assessment plan?

s74
00:06:57.360 --> 00:06:58.880
what do we do with this patient?

s75
00:06:58.880 --> 00:07:05.520
You have to do write this after every single visit and there's some different variations on this depending on specialty.

s76
00:07:05.520 --> 00:07:16.720
Um, typically clinicians end up often doing it takes like two hours a day to write just write these notes and you often do it what's known as pajama time after work itself.

s77
00:07:16.720 --> 00:07:22.720
Uh, and that's a common source of clinician burnout spending all this time outside of work and it's not the most fun part of the job.

s78
00:07:22.720 --> 00:07:37.599
However, these documents are actually extremely high stakes because these clinical notes are often used as a basis of billing but also uh which is of course financial things are high stakes but also have clinical impact and the reason for this is because

s79
00:07:37.599 --> 00:07:46.880
these prior not these notes are used for the next clinician or as you switch health systems they use they provide context to the clinician of the uh patient's longitudinal

s80
00:07:46.880 --> 00:07:47.680
medical record.

s81
00:07:47.680 --> 00:07:50.319
So it's actually really high stakes to get this right.

s82
00:07:50.319 --> 00:08:03.280
Um we started here because we it's a known pro a pain point that's existed for many many years but finally the technologies caught up to do really really high quality medical notes that's actually personalized to the clinician.

s83
00:08:03.280 --> 00:08:13.280
This was an amazing wedge into healthcare industry which has typically been technology reticent because it led to they actually care a lot about getting these notes high quality and right.

s84
00:08:13.280 --> 00:08:28.080
it led to higher doctor satis sat provider satisfaction and such they could actually see more patients um and it can actually help create a higher record um that helps prevent uh as it relates to billing auditing and other reasons.

s85
00:08:28.080 --> 00:08:42.640
So we started there just this product alone scaled to 300 hospital systems but I want to show you a little about where we're going next and um and the core thesis of the company is that everywhere in healthcare is everything is around the conversation

s86
00:08:42.640 --> 00:09:00.959
so we started over here with the conversation to clinical note everything else is downstream of that whether you it relates to billing whether it relates to things like clinical trial matching or whether it relates to clinical decision support It's all about the conversation, that sacred

s87
00:09:00.959 --> 00:09:06.240
doctor and patient conversation, and we've just built all this administrative machinery around that.

s88
00:09:06.240 --> 00:09:13.519
But how can we bring it back to that conversation and actually automate some of that uh administrative machinery?

s89
00:09:13.519 --> 00:09:28.240
So, to give you a tactical example of what this looks like, um, and I'll play this video of where where we're going from here.

s90
00:09:28.240 --> 00:09:35.440
We've been building a solution that allows the physician to interact with a bridge directly by using their voice.

s91
00:09:35.440 --> 00:09:36.720
Hi, ABridge.

s92
00:09:36.720 --> 00:09:40.560
Is Nathan eligible for any clinical trials?

s93
00:09:41.120 --> 00:09:43.519
He may be eligible for the abridge HF study.

s94
00:09:43.519 --> 00:09:53.519
He remains symptomatic despite maximal therapy and most screening criteria are already met, but an updated echo cardiogram is needed to confirm his ejection fraction and complete eligibility assessment.

s95
00:09:53.519 --> 00:09:56.320
All right, please order that echo for him.

s96
00:09:56.320 --> 00:09:56.800
Done.

s97
00:09:56.800 --> 00:09:58.320
Confirmary echo ordered.

s98
00:09:58.320 --> 00:10:06.160
And then when I'm done for the day, I can just ask a bridge, hey a bridge, can you prepare my charts for tomorrow?

s99
00:10:06.160 --> 00:10:09.920
And a bridge is working for me.

s100
00:10:10.160 --> 00:10:11.120
pause right there.

s101
00:10:11.120 --> 00:10:23.200
One of the things that you'll you'll notice is that we are thinking about how to uh revolutionize the entire visit uh for a clinician from previsit how um earlier in the we have suggested discussion topics.

s102
00:10:23.200 --> 00:10:28.640
Here's things that you can talk about uh with your patient whether they're clinical or more billing related.

s103
00:10:28.640 --> 00:10:32.160
Um we have after the visit we actually create everything for you.

s104
00:10:32.160 --> 00:10:38.160
The patient visit summary, the actual clinical note and we actually pend orders as you might have seen.

s105
00:10:38.160 --> 00:10:42.640
We're able to use we we're able to do this all by reading all this context.

s106
00:10:42.640 --> 00:10:45.360
We have access to all of the EHR context.

s107
00:10:45.360 --> 00:10:49.279
So we know everything about the patient, the prior labs, the prior notes.

s108
00:10:49.279 --> 00:10:52.959
We have access to the live conversation between the doctor and the patient.

s109
00:10:52.959 --> 00:10:57.760
That's where the quoteunquote debugging happens in healthcare where you learn about what the patient is facing now.

s110
00:10:57.760 --> 00:11:05.519
Um and then we have access to world's medical literature that we can ground and clinical guidelines that we can ground all of our work in.

s111
00:11:05.519 --> 00:11:18.480
So I want to I want to switch now given the context of where we're going as a as a product.

s112
00:11:18.480 --> 00:11:27.120
I want to switch into some of the key technical and engineering problems we face and inspire you on some of the what I think are very much frontier AI challenges.

s113
00:11:27.120 --> 00:11:37.760
So if you've ever worked on an agentic product before, the regardless of vertical, um the three KPIs that tend to matter are quality and then latency and cost.

s114
00:11:37.760 --> 00:11:42.880
In healthcare, I feel that we're actually playing on hard mode for all of these three KPIs.

s115
00:11:42.880 --> 00:11:46.959
This is a highstake scenario, especially when you're doing something like clinical decision support.

s116
00:11:46.959 --> 00:11:51.200
You have to be right because the downside is extremely high when you're wrong.

s117
00:11:51.200 --> 00:11:55.920
When I used to work glean, you know, while I love that product, I could be wrong and it would have been fine.

s118
00:11:55.920 --> 00:12:03.279
Maybe we answered a question incorrectly, but in healthcare, if we answer something incorrectly, there's actually consequences and we entirely lose our trust.

s119
00:12:03.279 --> 00:12:05.040
So quality needs to be absolutely high.

s120
00:12:05.040 --> 00:12:07.839
And I'll talk a little about how we keep that bar high.

s121
00:12:07.839 --> 00:12:13.200
And then latency and cost also really matter for us when you're live in the conversation.

s122
00:12:13.200 --> 00:12:20.160
You can't uh with latency, you can't act on information too late, and you have to act on the also at the right time for it to be useful.

s123
00:12:20.160 --> 00:12:24.800
And then finally, cost at the scale we're we're doing this at.

s124
00:12:24.880 --> 00:12:36.399
Um and and as an interesting aside, it actually relates to our we have a motto inside the company that our goal is to save lives, save time, save money for uh for the hospital system and for the healthcare industry as a whole.

s125
00:12:36.399 --> 00:12:44.000
And it actually I think it's funny that it really maps to the three KPIs that you care about in any agent product.

s126
00:12:44.160 --> 00:12:47.839
So talking a little about quality, how do we keep that bar high?

s127
00:12:47.839 --> 00:12:54.240
For us, we really treat eval as the operating system, the lifeblood of the of the company.

s128
00:12:54.240 --> 00:12:57.839
This starts from internal benchmarks and offline evaluation.

s129
00:12:57.839 --> 00:13:08.880
Before we develop any product, you know, whether we're talking about clinical trial matching, clinical note, clinical decision support, coding, we start with a robust set of internal benchmarks.

s130
00:13:08.880 --> 00:13:09.920
This is pre-eployment.

s131
00:13:09.920 --> 00:13:14.000
And then we test that against you know things that we've actually seen in the wild.

s132
00:13:14.000 --> 00:13:16.480
Then we have a staged roll out.

s133
00:13:16.480 --> 00:13:20.480
We know that we need to make contact with reality reality.

s134
00:13:20.480 --> 00:13:24.000
Not everything offline will perfectly represent what happens in practice.

s135
00:13:24.000 --> 00:13:25.760
And so we slowly roll it out.

s136
00:13:25.760 --> 00:13:31.040
Uh maybe it starts with the alpha group of clinicians that we trust and under the understand the stakes.

s137
00:13:31.040 --> 00:13:32.160
We rely to beta.

s138
00:13:32.160 --> 00:13:34.160
Maybe there's AB testing at scale.

s139
00:13:34.160 --> 00:13:38.959
And then even after it's fully rolled out, you always need continual monitor.

s140
00:13:38.959 --> 00:13:40.560
Again, the stakes are really high.

s141
00:13:40.560 --> 00:13:47.920
And you cannot get away with just being like a prototype that you just ship out there and be like, "Yeah, I mean, I tested on a few cases and it works."

s142
00:13:47.920 --> 00:13:52.560
How we do this is we always have expert calibrated LM judges.

s143
00:13:52.560 --> 00:13:55.519
So we have clinicians embedded throughout the entire company.

s144
00:13:55.519 --> 00:13:58.720
The clinicians are are domain experts but not all of us are clinicians.

s145
00:13:58.720 --> 00:13:59.839
I'm not a clinician.

s146
00:13:59.839 --> 00:14:08.000
So how can we the rest of the company still move fast is by encoding that clinician judgment into LM judges.

s147
00:14:08.000 --> 00:14:14.480
You know I think a really great evaluation system has a property that it reflects the behaviors that you want in your product.

s148
00:14:14.480 --> 00:14:21.839
At the end of the day, we are making a product for clinicians and sue who best other than our clinicians to actually create our judges that represent what they want.

s149
00:14:21.839 --> 00:14:31.440
And those judges once you have that create a feedback loop so that anyone whether you're a clinician or not can actually uh hill climb and learn from that.

s150
00:14:31.440 --> 00:14:40.639
We also have a lot of online signals whether how you're editing the clinical note and your typical thumbs up, thumbs down uh and star ratings and other free form text.

s151
00:14:40.639 --> 00:14:45.279
So this is a general framework we use for our for all our products.

s152
00:14:45.279 --> 00:14:50.399
I want to deep dive into the product that I work on which is clinical decision support.

s153
00:14:50.399 --> 00:14:58.560
So to give you an example of what clinical decision support looks like and specifically we're building something novel which is contextual clinical decision support.

s154
00:14:58.560 --> 00:15:04.000
Maybe a provider asks a question like hey does this patient meet the criteria for fibrill neutrifinia.

s155
00:15:04.000 --> 00:15:10.160
Um and so what we have to do here is actually a lot of context is underspecified in this question.

s156
00:15:10.160 --> 00:15:29.360
where so the first thing we're going to do is we're actually going to pull from the EHR data previous previous lab values then using that context we're going to uh use the uh uh we're also going to use the live conversation and we're going to use clinical guidelines and medical journals to com use as the reasoning sources

s157
00:15:29.360 --> 00:15:39.519
for combining all this context to actually answer the provider's question now but I want to focus on evaluation again the stakes are really high here we we really can't get this wrong.

s158
00:15:39.519 --> 00:15:42.560
So how do you tell whether or not an answer is correct?

s159
00:15:42.560 --> 00:15:48.639
And sometimes I I and this is a case where the generator and the verifier gap is really small.

s160
00:15:48.639 --> 00:15:59.920
What I mean by this is in some problems in AI such as like Sudoku, it's really really hard to generate a solution to Sudoku, but it's extremely easy to verify it uh once you do have the solution

s161
00:15:59.920 --> 00:16:04.959
and that makes uh that makes it much easier to hill climb against and build a valuation for.

s162
00:16:04.959 --> 00:16:08.639
But in a case like this, the generator and verifier gap is really small.

s163
00:16:08.639 --> 00:16:14.399
If I had a really really good generator uh verifier then that would just be my generator itself.

s164
00:16:14.399 --> 00:16:21.040
So how do I create a reference that isn't just a language model itself and ground itself so I have trust.

s165
00:16:21.040 --> 00:16:25.600
So what we do is we tackle this by having many many different signals.

s166
00:16:25.600 --> 00:16:33.040
Uh we have a clinical quality judge which I'm going to dive deep into and then we tackle from uh we have many signals from a boundary and adversarial judge.

s167
00:16:33.040 --> 00:16:41.680
We have a clinical safety judge and we also have judges that re represent product as aspects like tone and style and that matters a lot as well for AI products.

s168
00:16:41.680 --> 00:16:50.320
So all of these are different signals that try to get a piece of this like really really hard to measure problem and guarantee it in the way that we want the product to be.

s169
00:16:50.320 --> 00:16:52.079
So diving into the clinical quality judge.

s170
00:16:52.079 --> 00:16:55.040
So again I said the generator verifier gap is really small here.

s171
00:16:55.040 --> 00:16:59.920
So what we need is we actually need human references to tell are we generating the right thing?

s172
00:16:59.920 --> 00:17:06.079
But you can't just create a human golden response because there is a lot of variability in the potential responses.

s173
00:17:06.079 --> 00:17:09.120
So what we did is we took a lot of real clinical cases.

s174
00:17:09.120 --> 00:17:11.679
We had independent physicians create a rubric.

s175
00:17:11.679 --> 00:17:15.039
So this rubric said elements of what we wanted in the response.

s176
00:17:15.039 --> 00:17:22.880
So it's not here's the exact response because again there's many infinite possible responses but a good rubric elements that what a good response would look like.

s177
00:17:22.880 --> 00:17:25.839
And then we had a separate physician that actually adjudicated it.

s178
00:17:25.839 --> 00:17:32.160
brought these two independent rubrics together, created a final rubric, and we actually had a fourth clinician do QA on these rubrics.

s179
00:17:32.160 --> 00:17:35.280
Once you have these rubrics, and here here's a sample rubric looks like.

s180
00:17:35.280 --> 00:17:36.320
Here's actually a question.

s181
00:17:36.320 --> 00:17:38.640
I mean, there there's more context in the case itself.

s182
00:17:38.640 --> 00:17:41.840
You have a rubric of what are the elements that a response should look like.

s183
00:17:41.840 --> 00:17:53.840
Now, we can actually have an LLM judge that compares our agents responses to these rubric elements and does some basic semantic match to tell, hey, is our model performing well as we continue to hill climb?

s184
00:17:53.840 --> 00:17:58.880
Whether it's our agent architecture, our models or search ranking algorithms.

s185
00:18:00.640 --> 00:18:02.880
I want to talk a little now about cost and latency.

s186
00:18:02.880 --> 00:18:04.960
Two other really really hard problems for us.

s187
00:18:04.960 --> 00:18:07.120
So we do this as we said in that intro video.

s188
00:18:07.120 --> 00:18:13.440
We do this on the live in the conversation and we do on the run rate of 100 million medical conversations a year.

s189
00:18:13.440 --> 00:18:17.360
How do we do this in a way that doesn't really break the bank for us?

s190
00:18:17.360 --> 00:18:22.320
So one one place that this problem comes up or is is actually in generating the clinical note.

s191
00:18:22.320 --> 00:18:25.360
So when you're generating the clinical note, there's many different sections to it.

s192
00:18:25.360 --> 00:18:30.559
There's a history of present illness, past medical history, and there's the assessment and plan.

s193
00:18:30.559 --> 00:18:41.280
So one of the core insights for us is rather than say using a foundation model to generate all of this is we can actually break decompose this problem into simpler smaller workflows.

s194
00:18:41.280 --> 00:18:43.919
Healthcare is actually many specific workflows.

s195
00:18:43.919 --> 00:18:49.200
You don't need you know fable five to actually solve all of your clinical notes.

s196
00:18:49.200 --> 00:18:52.480
we we don't need frontier level intelligence for every problem.

s197
00:18:52.480 --> 00:19:02.559
So we actually post-train a lot of smaller models for different problems such as different actually even to the granularity of different sections in the clinical node and that lets us use much smaller

s198
00:19:02.559 --> 00:19:17.200
models because it's a more specific problem and at uh at much cheaper costs and latency and we have this data flywheel that we have this unique data set of 100 million medical conversations a year and as far as we know one else has such a

s199
00:19:17.200 --> 00:19:18.400
large data set.

s200
00:19:18.400 --> 00:19:22.320
So our key insight is having a right to tr win in training models.

s201
00:19:22.320 --> 00:19:27.840
There are problems where the quality is already maxed out and so you should train models then to reduce quality and latency.

s202
00:19:27.840 --> 00:19:32.960
But there are other problems where the quality isn't maxed out and people say oh the frontier model will just general roll you.

s203
00:19:32.960 --> 00:19:45.360
Our key insight is we can actually potentially beat the rate of change on the frontier model if we have the right to win by having the right data that they may not have and the focus on a problem that they may not be focusing on.

s204
00:19:45.360 --> 00:19:48.880
And that lets us still maximize quality.

s205
00:19:49.120 --> 00:19:52.559
Another uh problem that I'll quickly touch on is invisit orders.

s206
00:19:52.559 --> 00:20:01.200
So uh do doctors really aren't a big fans of pending orders, but often they'll mention orders during the visit itself, medication or non-medication orders.

s207
00:20:01.200 --> 00:20:11.360
So what we have is while we're listening in the visit, as the clinician says order, we actually ceue it up in the background uh and let them actually sign it off in the EHR.

s208
00:20:11.360 --> 00:20:17.360
But you can imagine if we did this in a very naive way, like every few seconds are just listening for orders, that would really break the bank.

s209
00:20:17.360 --> 00:20:28.000
Um, and so a lot of our tricks are like how do we find the right events in the conversation to actually trigger heavier models that will actually do the order matching because you need to match the order.

s210
00:20:28.000 --> 00:20:34.720
Not just is the order said, but does it match and references orders that are approved by the system and are relevant to the conversation.

s211
00:20:34.720 --> 00:20:46.320
So we have a number of different gates that are cheaper and faster that let us trigger actually larger models and hand off to them for actually doing the endto-end work.

s212
00:20:46.640 --> 00:20:56.400
Um but the last message this was of course a very quick talk but the message I want to leave you with is healthcare is a domain that needs frontier AI and actually puts it to the test at uh higher stakes.

s213
00:20:56.400 --> 00:20:59.760
In the past I as an engineer myself was weary about working in healthcare.

s214
00:20:59.760 --> 00:21:03.039
Does does like does healthcare technology actually work?

s215
00:21:03.039 --> 00:21:04.240
Well a bridge has proven this.

s216
00:21:04.240 --> 00:21:10.080
at scale for sure and as hopefully I gave you a taste of some of the frontier problems that we work on.

s217
00:21:10.080 --> 00:21:11.120
So thank you so much.

s218
00:21:11.120 --> 00:21:16.080
My name is Caitana again you and feel free to connect with me at Twitter or LinkedIn.

s219
00:21:16.080 --> 00:21:16.609
Thank you.

s220
00:21:16.609 --> 00:21:18.609
[applause]