WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=O84lhGc1OOI

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/O84lhGc1OOI.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:13.360 --> 00:00:13.759
Okay.

s3
00:00:13.759 --> 00:00:15.040
So, hi everyone.

s4
00:00:15.040 --> 00:00:17.440
I'm going to give the quickest introduction to myself.

s5
00:00:17.440 --> 00:00:18.480
I'm Ben Clavier.

s6
00:00:18.480 --> 00:00:20.880
I worked at Mix Bread where we do retrieval.

s7
00:00:20.880 --> 00:00:22.880
I'm French and I live in Tokyo.

s8
00:00:22.880 --> 00:00:28.000
And today I'm going to talk talk to you [clears throat] about the fact that agents should do knowledge work.

s9
00:00:28.000 --> 00:00:30.000
And so we should design them like knowledge workers.

s10
00:00:30.000 --> 00:00:32.960
Like we should design them like knowledge agents and not coding agents.

s11
00:00:32.960 --> 00:00:36.079
And I'm going to explain the difference and why I think that's important.

s12
00:00:36.079 --> 00:00:39.680
It's a bit of a hot tech talk but let's start now.

s13
00:00:39.680 --> 00:00:50.800
So the first thing is like first agents gave us fun trivia and I'm talking like early agents 2022 agents back when all you had was you know rag but agents couldn't even the tool call back then.

s14
00:00:50.800 --> 00:00:52.879
So all you had is like you had an if statement.

s15
00:00:52.879 --> 00:00:53.680
You did search.

s16
00:00:53.680 --> 00:00:56.160
You got like cool you could talk to your PDF.

s17
00:00:56.160 --> 00:00:58.640
That was the very first form of agentic work.

s18
00:00:58.640 --> 00:00:59.520
It was not very useful.

s19
00:00:59.520 --> 00:01:01.920
We're not going to talk about that for long.

s20
00:01:01.920 --> 00:01:05.600
What came next was programming agents and that's been all the rage.

s21
00:01:05.600 --> 00:01:16.240
Like once agents started being able to search actually properly search actually properly understand things car tasks out call tools we started designing coding agents and coding agents are a big thing.

s22
00:01:16.240 --> 00:01:19.680
I don't think there's anyone in this room that does not use coding agents.

s23
00:01:19.680 --> 00:01:24.799
would use code, codeex, etc. And that's a form of knowledge work.

s24
00:01:24.799 --> 00:01:27.759
But agents were not knowledge workers at the time.

s25
00:01:27.759 --> 00:01:29.520
Like agents were coding agents.

s26
00:01:29.520 --> 00:01:31.520
And now they're becoming knowledge worker.

s27
00:01:31.520 --> 00:01:35.600
And by knowledge worker, I mean that knowledge work is a big supererset.

s28
00:01:35.600 --> 00:01:41.439
And kind of every workflow you've thought of before is a form of knowledge agents just because of the nature of knowledge.

s29
00:01:41.439 --> 00:01:54.640
So it's like if you have an agent that's a lawyer that's looking for legal documents, if you go financial agent, if you're looking for you know medication information like you've got a lot of people on Twitter that try to do those self diagnosis and you've got

s30
00:01:54.640 --> 00:01:56.079
just a lot of medical usage.

s31
00:01:56.079 --> 00:01:58.799
All of that is coding agents for getting an agent.

s32
00:01:58.799 --> 00:02:00.880
It's trying to find say knowledge agents.

s33
00:02:00.880 --> 00:02:16.959
is trying to find knowledge, trying to make use of knowledge and coding is part of that of course and even the small rag bit that we talked about like in the first slide is part of that but that's a very very small proportion of the actual full thing like there's so much more to knowledge than any one

s34
00:02:16.959 --> 00:02:29.280
domain and what even is knowledge work because I'm saying that it's important they do knowledge work coding is knowledge work and I think there's two ways to define it in my opinion one of them is

s35
00:02:29.280 --> 00:02:32.480
knowledge work is work Your main input is information.

s36
00:02:32.480 --> 00:02:35.280
Like your main input is not an actual physical material.

s37
00:02:35.280 --> 00:02:37.519
It's not something that you can touch.

s38
00:02:37.519 --> 00:02:39.519
It's knowledge is information.

s39
00:02:39.519 --> 00:02:46.480
And the nature of knowledge work is that you process this information which is by nature very ambiguous, very diffuse.

s40
00:02:46.480 --> 00:02:50.000
And the main output you get from that is something actionable.

s41
00:02:50.000 --> 00:02:50.879
It's a judgment.

s42
00:02:50.879 --> 00:02:51.599
It's a decision.

s43
00:02:51.599 --> 00:02:57.120
It's some if it's a lawyer, you're going to get, you know, their findings on your case and they might plead for you.

s44
00:02:57.120 --> 00:03:04.159
you're going to get an actual actionable thinking item like something still not tangible but that exists as knowledge.

s45
00:03:04.159 --> 00:03:13.360
There's also a topological definition which makes sense here is that if you need search it's a knowledge problem and if it's a knowledge problem you need search.

s46
00:03:13.360 --> 00:03:26.319
So it's very easy self-defined and in the real world that's basically most of the work that we see in the service economy is a form of knowledge work like lawyers knowledge workers academics knowledge workers actuaries knowledge workers

s47
00:03:26.319 --> 00:03:39.360
software engineers researchers also knowledge workers and the fact that there's so much knowledge work in society has contributed to like a never improving structuring of knowledge work like there's actually very very well definfined workflows

s48
00:03:39.360 --> 00:03:44.720
for how we should do knowledge work for how knowledge works in itself and how we evolve that.

s49
00:03:44.720 --> 00:03:49.120
But so far, agentics kind of focused on the special case and I try to generalize from it.

s50
00:03:49.120 --> 00:03:53.040
And that special case is coding and software engineering.

s51
00:03:53.040 --> 00:03:56.959
And the thing is code is knowledge, but not all knowledge is code.

s52
00:03:56.959 --> 00:04:07.599
And code is a very very unique form of knowledge because it has very durable cues like on in a codebase there's going to be a lot of references to like an identifier or a file or a path.

s53
00:04:07.599 --> 00:04:15.920
And okay when we vibe code that can change but most of the time it's not going to change all that much like all the things are like very very durable.

s54
00:04:15.920 --> 00:04:25.840
Then you've got that surface obviously is like now and grapable like there's keywords there's method definition there's a lot of things that by definition you can grab in code

s55
00:04:25.840 --> 00:04:38.720
and the task and this one's actually very important and we don't talk about it a lot but people are like oh why is ge good enough for programming or why can an agent do programming and then you're telling me it can do like deep research for a legal question

s56
00:04:38.720 --> 00:04:50.880
and one of those is because we don't realize it but like when we interact with coding agents we are giving them extremely in our task like we're not actually expecting that much from them like everything is always kind of about a feature

s57
00:04:50.880 --> 00:05:01.360
about like a given ticket like there's a task at hand you're not going to tell the agent like discover a new programming paradigm and then implement it in this new app I don't know what's going to do good luck

s58
00:05:01.360 --> 00:05:16.080
but in know that's often the case like first of all you don't have those double cues like the meaning is always implicit and more importantly the same clue can mean a lot of different things we don't have function definitions in knowledge work like if receive 30 days.

s59
00:05:16.080 --> 00:05:19.039
If your agent's looking for 30 days, is it a deadline?

s60
00:05:19.039 --> 00:05:19.759
Is it a grace period?

s61
00:05:19.759 --> 00:05:20.880
Is it a retention rule?

s62
00:05:20.880 --> 00:05:22.240
Is it even in the same domain?

s63
00:05:22.240 --> 00:05:26.479
Like are you searching for 30 days on the contract and you're getting medication?

s64
00:05:26.479 --> 00:05:28.560
There's a lot of contextual information.

s65
00:05:28.560 --> 00:05:28.800
Yeah.

s66
00:05:28.800 --> 00:05:31.840
But more importantly, the search starts from an intent.

s67
00:05:31.840 --> 00:05:42.720
Like even if you're doing again legal example, if you're asking about like a specific rule that you want to apply to a specific domain, you're going to need to look at the international norms that apply and then do they apply in this case.

s68
00:05:42.720 --> 00:05:55.520
there's a lot of conditional information that is not predefined in the task like that's all up for the agent to find and so non-code knowledge is very contextual and meaning driven which is much harder than code

s69
00:05:55.520 --> 00:06:07.840
and that's led to the fact that none of what I'm saying is new like people have been doing knowledge work for a very very long time and that's resulted in like two endless loops you have a tool loop which is

s70
00:06:07.840 --> 00:06:13.840
at the start we were we were like talking then at some point some guy was like we write stuff down.

s71
00:06:13.840 --> 00:06:23.120
Then in Alexandria, we had the pin case which was the curator of the library of Alexandria came up with an idea that maybe we should have a way to catalog all of the books we have.

s72
00:06:23.120 --> 00:06:24.240
Then we developed writing.

s73
00:06:24.240 --> 00:06:26.000
Then we developed bibliographies.

s74
00:06:26.000 --> 00:06:33.440
Then we ended up with like the current version of the like Dewey system for libraries and nowadays we have search engines.

s75
00:06:33.440 --> 00:06:50.080
But we also had an organization loop which is joint but also disjoint from the tool one which is it used to be the one gifted expert like we've all heard of the polymath of the past the person who just knew everything about one domain or all domains and you just went to them if you had information

s76
00:06:50.080 --> 00:06:51.039
but that doesn't scale.

s77
00:06:51.039 --> 00:07:00.720
So we ended up with like monasteries which were like guardians of knowledge and then we had universities and then we ended up creating the bureaucracies and now we ended up creating the modern organization of work where

s78
00:07:00.720 --> 00:07:10.080
we have very specialized firms like at hospitals you've got the doctor you've got the senior doctor you've got the nurse practitioner the nurses the healthcare assistants and all of them kind of like

s79
00:07:10.080 --> 00:07:25.680
specialize on different levels of tasks and that's a really good form of optimization but the thing is that it's actually just the one loop like I'm showing two loops here but they're actually just the one loop which is we have new knowledge and new knowledge means that we need better tools

s80
00:07:25.680 --> 00:07:36.880
and better tools mean that we end up creating new workflows new roles like we need people that are trained to use those tools people that understand what the new tool does if you have a guy that knows how to go to the library

s81
00:07:36.880 --> 00:07:49.840
and you're like okay use Google you need the knowledge of what Google is like that person needs to be taught that's a search engine you can just type stuff in it there's no need to physically go there and that means you retrain you get new knowledge workers who are more efficient so they create more knowledge so we

s82
00:07:49.840 --> 00:07:51.919
need new tools and so on and so on.

s83
00:07:51.919 --> 00:08:01.360
So both the tool loop and the organizational loop are actually just this one selfoptimizing loop that kind of like triggers the other endlessly.

s84
00:08:01.919 --> 00:08:07.440
And the thing about like tooling and optimization is that they're not neutral add-ons.

s85
00:08:07.440 --> 00:08:12.479
Like I said that we keep optimizing tools and things come up and we create new things out of those tools.

s86
00:08:12.479 --> 00:08:14.479
But that's never actually a neutral thing.

s87
00:08:14.479 --> 00:08:17.919
Like tooling is not just oh my search is 5% better.

s88
00:08:17.919 --> 00:08:30.720
the fact that we have a tool or the fact that we don't have a tool is what decide if a task not if the task is possible because you can do things without the right tool but if the task is actually scalable and can be carried out cheaply because something being cheap means it can scale

s89
00:08:30.720 --> 00:08:43.680
and it's like yes of course if you go to the library of Alexandria before the pakis you can find your manuscript somewhere like wherever you're looking for it there it's probably going to take two three weeks so you're going to really really really need that knowledge

s90
00:08:43.680 --> 00:08:51.920
but if there's a library catalog it's going to take you 10 minutes and now like it's way easier to just oh okay I need to know something more about this so I'm going to search for it

s91
00:08:51.920 --> 00:09:06.959
likewise if you have a map directory or if you even have a map in the first place which in itself is a tool for information then exploring the world is a much better idea like you're not going to rely on randomly discovering America on your way to the indies you know like you know where you're going

s92
00:09:06.959 --> 00:09:19.519
and likewise if you have like a multimodel search platform then you can search millions of PDF in a way that we couldn't before so now there's a lot of use cases where you were like oh it's in the archives I'm not going to touch that that become actually useful

s93
00:09:20.480 --> 00:09:26.560
and in practice this kind of looks like that and I'm getting into the more technical stuff here which is on a simple deep research task.

s94
00:09:26.560 --> 00:09:34.000
So this is the browse comp plus leaderboard which is made to evaluate the quality of search tools on a very bonded deep research task.

s95
00:09:34.000 --> 00:09:37.061
You have 200,000 documents and you have like specific queries.

s96
00:09:37.061 --> 00:09:42.560
[clears throat] Bio talked about this this morning and it's a really useful benchmark to like analyze queries.

s97
00:09:42.560 --> 00:09:45.040
And what we see that like okay a bad tool.

s98
00:09:45.040 --> 00:09:47.680
So that's a thing that people often rant about.

s99
00:09:47.680 --> 00:09:49.519
You'll see that there's two BM25 here.

s100
00:09:49.519 --> 00:09:52.160
There's two like optimized and unoptimized.

s101
00:09:52.160 --> 00:09:56.240
And that's because quite often people will tell you BM25 is not great.

s102
00:09:56.240 --> 00:10:00.160
And the reason they'll tell you BM25 is not great is because there's not one BM25.

s103
00:10:00.160 --> 00:10:01.360
There's hundreds of them.

s104
00:10:01.360 --> 00:10:02.880
It's a way to do lexical search.

s105
00:10:02.880 --> 00:10:04.800
You should always optimize your baselines.

s106
00:10:04.800 --> 00:10:07.839
You should always like optimize what you're betting.

s107
00:10:07.839 --> 00:10:11.519
And so what you see here is like a badly optimized tool is useless.

s108
00:10:11.519 --> 00:10:12.880
Like 60% accuracy.

s109
00:10:12.880 --> 00:10:15.519
You're not going to trust someone that's right 60% of the time.

s110
00:10:15.519 --> 00:10:17.600
You're just going to do it yourself.

s111
00:10:17.600 --> 00:10:21.360
When you start optimizing the tools, you can see we go up to 70 80.

s112
00:10:21.360 --> 00:10:24.640
And then the best the actual best is the hybrid harness that gets to 98.

s113
00:10:24.640 --> 00:10:36.560
But that's maybe not the most interesting part because we start kind of plateauing at one point like the jump from 889.8 to 90.2 is in run variance that don't matter.

s114
00:10:36.560 --> 00:10:40.320
What matters here however is that 90.2% accuracy.

s115
00:10:40.320 --> 00:10:43.760
You reach it with 20% fewer tool calls.

s116
00:10:43.760 --> 00:10:49.279
And that's huge because in practice that's 20% fewer tokens, 20% like fewer resources that you use.

s117
00:10:49.279 --> 00:10:51.040
That's basically 20% free cash.

s118
00:10:51.040 --> 00:10:58.079
And if you compare it to the unoptimized baseline, you're spending like 5% of what you were spending in the first place.

s119
00:10:58.079 --> 00:11:00.720
So the tool is actually what makes the task worth doing.

s120
00:11:00.720 --> 00:11:02.560
Nobody would keep using that tool.

s121
00:11:02.560 --> 00:11:05.440
If it takes 25 calls, but if it takes eight calls, you're like, "Oh yeah, cool.

s122
00:11:05.440 --> 00:11:08.320
That's a workflow I can introduce."

s123
00:11:09.360 --> 00:11:16.399
And the second part which goes with tooling and I think it's just as important because Bruscom plus in the previous slide is interesting, but it's easy.

s124
00:11:16.399 --> 00:11:17.839
It's 100,000 documents.

s125
00:11:17.839 --> 00:11:19.600
It's just text.

s126
00:11:19.600 --> 00:11:20.800
It's just the one question.

s127
00:11:20.800 --> 00:11:22.079
It's not really that open-ended.

s128
00:11:22.079 --> 00:11:24.000
It's just a bit convoluted.

s129
00:11:24.000 --> 00:11:35.440
But when you're actually doing like real life knowledge work, there's that workflow that you see that I try to doodle, which is you have a client, they come to the big shot, they come to the lawyer, that's the partner of the agency,

s130
00:11:35.440 --> 00:11:37.279
and they're like, "Okay, this is my situation.

s131
00:11:37.279 --> 00:11:37.920
That's my problem."

s132
00:11:37.920 --> 00:11:44.320
And they meet together, but then what the partner does is they're not going to be the ones doing all the legal research.

s133
00:11:44.320 --> 00:11:47.680
They're not going to be the ones like doing every single step of the problem.

s134
00:11:47.680 --> 00:11:54.560
What they'll do is kind of understand that like okay this person is has this problem that's going to cause them that those are the facts.

s135
00:11:54.560 --> 00:11:59.600
I need the relevant laws with this that and so on aspects and then they've got party goals.

s136
00:11:59.600 --> 00:12:02.160
They've got assistants and the assistants are going to be doing this research.

s137
00:12:02.160 --> 00:12:15.200
They're going to be using like the sales of tools they've been trained to use and they're going to produce memo and not and then they're going to give that back to the big shots that maybe they'll [clears throat] research like one clarification point but they mostly rely on what their searcher agents if you want like the assistants

s138
00:12:15.200 --> 00:12:29.519
have found for them and that's the respond that you're going to get and that's echoing the point I made before which is in code when you're using cloud code you're kind of doing that work yourself you've already broken down the query you know like you know what you want to do you've got a linear artic you've got

s139
00:12:29.519 --> 00:12:41.839
something that you're giving the agent in the real or you've got a client that's got a very open-ended problem and you need to break it down yourself and your agent needs to break it down itself and then needs to use sub agents that do this research

s140
00:12:42.240 --> 00:12:58.079
and this is how better tools and organization work together because this one is MQA which is another form of knowledge benchmark QA [clears throat] is something that hugging face and snowflake jointly released and it's PDF based and enterprise task it's got PDFs and it's got OCR

s141
00:12:58.079 --> 00:13:09.920
OCR versions of the PDF and the current state of The board really show that like both tools and organizations are necessary and you can see that in the fact that with BM25

s142
00:13:09.920 --> 00:13:23.760
however optimized it gets the human in Gemini 3 reach the same ceiling and that doesn't mean that Gemini 3 is as good as a human that means that even the human cannot get the right information given unlimited searches with PM25.

s143
00:13:23.760 --> 00:13:26.880
So you have the tool setting and you need better tools to go forward.

s144
00:13:26.880 --> 00:13:29.920
That's the like tool optimization part of the loop.

s145
00:13:29.920 --> 00:13:31.440
Thankfully, we've got better tools.

s146
00:13:31.440 --> 00:13:34.480
You know, we've got we've got models that can handle PDS.

s147
00:13:34.480 --> 00:13:35.120
We've got vision.

s148
00:13:35.120 --> 00:13:38.079
We don't need to rely on OCR text.

s149
00:13:38.079 --> 00:13:43.920
And what we see with that is that we get another jump which is Gina and the mixbride search tool which is fully multimodel.

s150
00:13:43.920 --> 00:13:51.519
So, it can read the PDF, you get the tables, you get all that nice stuff in your search and that gets us a big jump in accuracy.

s151
00:13:51.519 --> 00:13:54.399
But the interesting part is that like that doesn't work.

s152
00:13:54.399 --> 00:14:04.240
the well that don't work that does work but that don't work as well as we would like because why is my agent getting 88.9 if the human is getting 99.4 for like that's 10% I'm leaving on the table here.

s153
00:14:04.240 --> 00:14:06.000
But that I don't understand that's an agent.

s154
00:14:06.000 --> 00:14:07.920
It gets I think 10 turns in the benchmark.

s155
00:14:07.920 --> 00:14:09.279
So it's a fully agentic system.

s156
00:14:09.279 --> 00:14:13.040
It gets to think about its results and yet it's missing performance.

s157
00:14:13.040 --> 00:14:23.760
And that's where we introduced the mixbr agent which is exactly that breaking down of work we saw earlier where we basically tell the the main agent once answering the question being like okay that's

s158
00:14:23.760 --> 00:14:26.160
that's a big topic there is like thousands of PDFs.

s159
00:14:26.160 --> 00:14:27.600
You're not going to like search yourself.

s160
00:14:27.600 --> 00:14:29.360
Just please break down the problem for me.

s161
00:14:29.360 --> 00:14:34.320
Please write queries about the aspects that you think are important to answer the actual query.

s162
00:14:34.320 --> 00:14:41.519
And we get searcher that go off on their own and they find the right results and they bring you like a little memo to your agent and then your agent actually answers that.

s163
00:14:41.519 --> 00:14:44.639
And that gets the accuracy up by 3.5 points.

s164
00:14:44.639 --> 00:14:48.480
And that don't sound like a lot, but I like to think of it as like an oracle gap.

s165
00:14:48.480 --> 00:14:54.320
And the oracle gap is the difference between perfect documents and your search system.

s166
00:14:54.320 --> 00:15:01.839
And the oracle gap here is about about 10 points before using the agents and it goes down to six points after using the agent.

s167
00:15:01.839 --> 00:15:05.519
So that means that we have about a 40% reduction in mistakes.

s168
00:15:05.519 --> 00:15:13.040
Like the gap between humans and agents goes down by 40% just by like having a better architecture to search through it.

s169
00:15:13.279 --> 00:15:17.920
And I think this is the end of my slide because I'm running out of time and that's perfect.

s170
00:15:17.920 --> 00:15:19.279
That's my takeaway slide.

s171
00:15:19.279 --> 00:15:29.040
And what I want you to like get from this talk is that we know how to design better knowledge work for humans and AI agents really benefit from this pattern like we have designed this.

s172
00:15:29.040 --> 00:15:29.920
We know how to do this.

s173
00:15:29.920 --> 00:15:31.600
Humans have worked on this for centuries.

s174
00:15:31.600 --> 00:15:32.075
Like

s175
00:15:32.075 --> 00:15:32.399
[snorts]

s176
00:15:32.399 --> 00:15:34.000
people have always needed more knowledge.

s177
00:15:34.000 --> 00:15:35.519
The empires used to have librarians.

s178
00:15:35.519 --> 00:15:36.320
We have parallel.

s179
00:15:36.320 --> 00:15:37.440
We've got legal firms.

s180
00:15:37.440 --> 00:15:42.320
We know exactly how legal the legal industry has figured out.

s181
00:15:42.320 --> 00:15:44.320
The medical industry has figured it out.

s182
00:15:44.320 --> 00:15:45.920
And none of it looks like programming.

s183
00:15:45.920 --> 00:15:57.519
Programming has a very different system because it's a very specific use case and we should really learn from the knowledge wall to know how to design agent that will do work for knowledge wall

s184
00:15:57.519 --> 00:16:04.240
and then you must not like overfeit on tools because tools don't exist as a way to do things by themselves.

s185
00:16:04.240 --> 00:16:06.240
Tools exist as a way to overcome ceilings.

s186
00:16:06.240 --> 00:16:10.800
You want a better tool when you see that you're hitting a ceiling that your performance is not where you want it to be.

s187
00:16:10.800 --> 00:16:22.000
So we design better tools to overcome that savings and more importantly the tools need to be co-designed with the agents like the agents need to know how to use tools because one thing you will often see is

s188
00:16:22.000 --> 00:16:30.000
agents will try to write grape queries because grapes everywhere in the training data BM25 is everywhere in the query in the data and that's not always what you need.

s189
00:16:30.000 --> 00:16:36.320
Sometimes you need semantic search over a PDF and you can't grab a PDF you can BM25 a PDF you need to like write a better query.

s190
00:16:36.320 --> 00:16:44.480
So it's very important that your agentic harnesses or even your agentic models know that they have got more than one tools and it's about primitives grabs a primitive.

s191
00:16:44.480 --> 00:16:53.279
PM25 is a primitive and semantic search is a primitive and all of those need to be like very well trained like the models need to know about all of them.

s192
00:16:53.279 --> 00:17:05.839
And the last one is that the right orchestration of search will get you much better results because context is a finite resource and even if we get to a model that's got like 100 million token context

s193
00:17:05.839 --> 00:17:16.880
a that's going to cost you a lot of money and b that's still nothing you're not even getting half of like one state's legal code let alone the US let alone international law let alone specialist course etc.

s194
00:17:16.880 --> 00:17:26.480
So you need to have a way to break down your task and you need to have your orchestrator, your main agents and people that can actually organize the knowledge for them.

s195
00:17:26.640 --> 00:17:32.320
And yeah, so we've got two minutes for questions.

s196
00:17:32.597 --> 00:17:34.597
[applause]

s197
00:17:34.880 --> 00:17:37.039
Thank you.