WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=cKhpeEBnT1o

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/cKhpeEBnT1o.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.719 --> 00:00:13.840
Hello everyone.

s2
00:00:13.840 --> 00:00:15.280
Can you all hear me?

s3
00:00:15.280 --> 00:00:16.240
Good.

s4
00:00:16.240 --> 00:00:17.279
Sweet.

s5
00:00:17.279 --> 00:00:18.560
My name is Jacob Hoisan.

s6
00:00:18.560 --> 00:00:20.640
I'm a Ford deployed engineer here at Exa.

s7
00:00:20.640 --> 00:00:27.519
And uh today's presentation is on how we built search for coding and review agents and how we power most of the Bay Area.

s8
00:00:27.519 --> 00:00:33.680
Um, in this regard, um, coding agents don't really need these 10 blue links that you see when you search Google.

s9
00:00:33.680 --> 00:00:44.239
Uh, what we do is we have a semantic search engine that essentially you pass in a query and we provide you context rich highlights to pass into your LLM.

s10
00:00:44.960 --> 00:00:53.840
The core problem that we have with large language models in code review and coding agent applications is that there's a knowledge cut off to all of these models.

s11
00:00:53.840 --> 00:00:59.840
As you can see from this graph, we're typically lagging around 6 months from the knowledge cutoff date to the release date.

s12
00:00:59.840 --> 00:01:10.159
In these instances, you have this gap that exists between when the model was released and when you might have an important change log or a PR push to a repository that you do have to review.

s13
00:01:10.159 --> 00:01:20.479
Um, in this instance, I do have an example where from the uh cooter vector store PR um that was made a few months back.

s14
00:01:20.479 --> 00:01:25.439
This is within our model blind spot after GPD 5.5's cutoff date.

s15
00:01:25.439 --> 00:01:37.680
So in these months leading after the cutoff date, you have all of these changes being made to real repositories that one you can't create net new repos with, but you actually can't review at all.

s16
00:01:37.680 --> 00:01:41.439
Um, so that's kind of where Exa comes into play in this instance.

s17
00:01:41.439 --> 00:01:46.960
Uh, and we'll actually just review this diff real quick so you can kind of see what I'm getting at.

s18
00:01:46.960 --> 00:01:50.479
But before we do that, like how would a human even review this?

s19
00:01:50.479 --> 00:01:56.240
Very typically a a human would probably go on to Stack Overflow and say, "Was this like random variable removed?

s20
00:01:56.240 --> 00:01:58.159
In this case, it's called inertia check."

s21
00:01:58.159 --> 00:02:00.320
Um, and like why isn't my code compiling?

s22
00:02:00.320 --> 00:02:11.840
And uh if your code's not compiling, you'll probably just paste the error code into Stack Overflow and you'd actually get the response that you would want and then you would make a PR to the repo in order to resolve it.

s23
00:02:11.920 --> 00:02:24.560
Now in this case in with using a large language model many people without web search will go see okay here's the error the diff looks consistent and it looks like a cleanup but that's really not the case.

s24
00:02:24.560 --> 00:02:32.319
The reason why we're removing this parameter uh in this instance is because we actually need to refactor to a dependency bump.

s25
00:02:32.319 --> 00:02:45.599
So using web search, you're actually able to take a look at the repository itself, any breaking changes from this repos change log and then explain the migration in depth.

s26
00:02:48.319 --> 00:02:53.120
So what's interesting here is that we don't return the entire GitHub repository page.

s27
00:02:53.120 --> 00:02:57.040
What we'll return is a very small snippet from the page explaining the change.

s28
00:02:57.040 --> 00:03:09.360
So if the query is exactly this, we have an interpretation step which will take that entire page and distill it down into exactly what the large language model needs to answer the question.

s29
00:03:09.360 --> 00:03:12.480
So you're not feeding the model 100,000 characters anymore.

s30
00:03:12.480 --> 00:03:17.120
You're feeding it quite literally only 500 characters to answer the question.

s31
00:03:20.400 --> 00:03:23.680
Now adding a search tool to your agent is not enough.

s32
00:03:23.680 --> 00:03:26.560
There's obviously kind of a two-step process here.

s33
00:03:26.560 --> 00:03:35.840
Um, if you just bolt on a web search onto your agent, many of you have seen maybe when you're using cloud code, you'll say, "Hey, I'd like to use sonnet 46 or 47 now."

s34
00:03:35.840 --> 00:03:39.200
And the model will say it doesn't exist, right?

s35
00:03:39.200 --> 00:03:43.360
So even though it has access to a web search tool, it's not actually like doing anything.

s36
00:03:43.360 --> 00:03:46.239
Um, because natively cloud code does have a web search tool.

s37
00:03:46.239 --> 00:03:52.080
Um, and in these instances, the model simply doesn't know that you're requesting a version that does exist.

s38
00:03:52.080 --> 00:03:55.440
So you actually have to provide the model with a set of rules.

s39
00:03:55.440 --> 00:04:10.799
So when you're doing a code review agent, the rule might look like when you have a diff that bumps a dependency or version, you might want to look at the upstream source to verify that this is actually true and then ground the actual review in what you find.

s40
00:04:10.799 --> 00:04:16.720
Um, so when I work with these code review and coding agent customers, there's several instances where this might happen.

s41
00:04:16.720 --> 00:04:21.199
They might actually add the tool to their agent and then adding it to the harness is simply not enough.

s42
00:04:21.199 --> 00:04:22.479
So, it's really a two-step process.

s43
00:04:22.479 --> 00:04:33.120
It's instructing your agent when to use web search uh since oftent times it's not built in and then also actually just executing the search uh in the sentence.

s44
00:04:33.360 --> 00:04:36.960
So, there's four things that AXA kind of gives us a model.

s45
00:04:36.960 --> 00:04:38.639
Uh we have transparency.

s46
00:04:38.639 --> 00:04:45.199
When you're using a native web search tool within OpenAI or Anthropic, you actually have this black box that exists.

s47
00:04:45.199 --> 00:04:47.120
They they call a web search API.

s48
00:04:47.120 --> 00:04:48.639
They synthesize all the information.

s49
00:04:48.639 --> 00:04:53.919
It might take up to 10 seconds and then you get some of the sources and then none of the content.

s50
00:04:53.919 --> 00:04:55.199
So that's really point one.

s51
00:04:55.199 --> 00:04:57.199
When you use Exo, you get the entire span.

s52
00:04:57.199 --> 00:05:03.040
You get the entire trace of what we looked for, the exact queries, the exact sources, what highlights were passed into the model.

s53
00:05:03.040 --> 00:05:10.720
Um, this is all given to you and can be passed into any of your telemetry to go investigate when something does go wrong.

s54
00:05:10.720 --> 00:05:18.639
Um, we also provide token efficient highlights, which I'll cover on the next slide, which distills an entire page down into exactly what the model needs.

s55
00:05:18.639 --> 00:05:21.039
And then it also comes down to cost.

s56
00:05:21.039 --> 00:05:27.360
Many many of the times that cost from these model providers are actually so large that you don't notice your web search bill.

s57
00:05:27.360 --> 00:05:36.400
Um but given like sufficiently large workloads were priced actually much more effectively than native web search providers and much better quality in most instances.

s58
00:05:36.400 --> 00:05:42.720
Um and then another thing that most of our customers are very happy with is that not being tied to one model provider.

s59
00:05:42.720 --> 00:05:47.680
uh when you use a thirdparty web search tool, you have this level of standardization that exists.

s60
00:05:47.680 --> 00:05:50.000
Uh you can use GLM when it comes out.

s61
00:05:50.000 --> 00:05:51.919
You can use the new anthropic models when it comes out.

s62
00:05:51.919 --> 00:05:54.160
You can use the new OpenAI models when it comes out.

s63
00:05:54.160 --> 00:06:00.560
And then you have like one fixed API that you're calling instead of everything that the other model providers are using.

s64
00:06:00.560 --> 00:06:08.160
And you have a fixed set of parameters that are extremely flexible um to what you would need across all model applications.

s65
00:06:08.319 --> 00:06:12.560
So, here's a quick gif of kind of how this works since I'm more of a visual learner myself.

s66
00:06:12.560 --> 00:06:21.120
Um, but in this instance, if you're asking about the access search API, we're quite literally just pulling certain lines at runtime, not using an LLM.

s67
00:06:21.120 --> 00:06:23.120
This is completely computational.

s68
00:06:23.120 --> 00:06:32.479
So, we actually cut zero, we add zero extra latency to our search call by providing you the contents of pretty much any page on the internet.

s69
00:06:32.479 --> 00:06:36.800
Um, and I do have an example of how this works.

s70
00:06:37.680 --> 00:06:41.039
So, let's say you're looking up.

s71
00:06:42.000 --> 00:06:44.160
Hopefully, you can see that all right on the query there.

s72
00:06:44.160 --> 00:06:54.479
If you're looking up and then a biography, you'll quite quickly get a result that I love photography and then what I do in my free time.

s73
00:06:54.479 --> 00:06:56.240
So, I like driving motorcycles.

s74
00:06:56.240 --> 00:06:57.919
As you can see, it's right here.

s75
00:06:57.919 --> 00:07:02.400
But then from that same website, I could actually ask for my phone number.

s76
00:07:02.400 --> 00:07:07.039
And then at runtime, we'll actually just get a completely different set of information.

s77
00:07:07.039 --> 00:07:10.080
So you can see this is generally applicable to pretty much any use case.

s78
00:07:10.080 --> 00:07:18.560
In a coding use case, you might have an entire GitHub repo that has like a bajillion lines of code, but you don't want the bajillion lines of code to be passed into your model.

s79
00:07:18.560 --> 00:07:21.440
And you don't want to use a model to synthesize that information either.

s80
00:07:21.440 --> 00:07:24.800
Um, so this works across any type of query.

s81
00:07:24.800 --> 00:07:29.440
If you want the latest news about Nvidia and it has to mention Jensen, then you could also get that.

s82
00:07:29.440 --> 00:07:40.639
Um but yeah, this is like one of the many applications that we have um in these instances and coding is really just the beginning.

s83
00:07:40.639 --> 00:07:44.560
Um this is one of the reasons why I joined Exa in the first place.

s84
00:07:44.560 --> 00:07:52.560
We have we had such a large swath of people that wanted to use Exa as a search API to get coding docs.

s85
00:07:52.560 --> 00:07:54.879
Um and I was super excited for the application.

s86
00:07:54.879 --> 00:08:01.599
Um but as we went forward and we took on many popular providers and they loved using us and we provided them a great experience.

s87
00:08:01.599 --> 00:08:06.879
Cursor, Cognition, Warp, Code Rabbit, all of these companies now use Exa to power their web search.

s88
00:08:06.879 --> 00:08:09.759
Um but now we have a new offering.

s89
00:08:09.759 --> 00:08:11.360
Uh it's called Exa Agent.

s90
00:08:11.360 --> 00:08:16.879
Now there's many instances where you don't want to orchestrate your own search, but you need a good search experience.

s91
00:08:16.879 --> 00:08:18.560
Uh and that's kind of where it comes in.

s92
00:08:18.560 --> 00:08:31.599
We partnered with several data providers to not just surface from the web but then also surface highlights from many different places such as similar web for web analytics, particle for podcast intelligence and crunchbas for private markets.

s93
00:08:31.599 --> 00:08:36.000
So we're powering the biggest financial firms and hedge funds uh with this information.

s94
00:08:36.000 --> 00:08:39.440
Um and if you want to learn more about this you can go to exit connect.

s95
00:08:39.440 --> 00:08:44.800
And then I do have a small little message here as well.

s96
00:08:45.040 --> 00:08:46.000
I ran a quick query.

s97
00:08:46.000 --> 00:08:56.800
If I wanted to find all of the people that are attending the AI engineer conference, you can quite literally find anyone that has ever mentioned this conference at this location.

s98
00:08:56.800 --> 00:09:00.880
And it's kind of crazy because I see people that I know and have emailed me here.

s99
00:09:00.880 --> 00:09:04.000
So, uh, these are most of the speakers that are attending the conference.

s100
00:09:04.000 --> 00:09:07.839
Um, and this applies broadly across every single use case.

s101
00:09:07.839 --> 00:09:11.920
If you wanted to find all the employees that work at Exa, you can also do that.

s102
00:09:11.920 --> 00:09:15.760
And you could also get all their emails and LinkedIns for outbound.

s103
00:09:15.760 --> 00:09:25.120
But yeah, um I think this is quite exciting and we wanted to package our search in the best way possible um because we can orchestrate our search quite well.

s104
00:09:27.360 --> 00:09:35.440
But yeah, we're you if you want to use Access today, you could simply call us via our MCP and then we're on basically every single provider that supports it.

s105
00:09:35.440 --> 00:09:39.279
Um and if you'd like any help in setting it up, that's my job.

s106
00:09:39.279 --> 00:09:41.440
So you can feel free to send me an email.

s107
00:09:41.440 --> 00:09:44.480
Um, but yeah, there's also a QR code at the top.

s108
00:09:44.480 --> 00:09:47.279
Um, if you want free credits, you could also talk to me after the talk.

s109
00:09:47.279 --> 00:09:51.680
Um, but yeah, if there's any questions, happy to take them.

s110
00:09:51.680 --> 00:09:55.680
I think I could take one question or two questions.

s111
00:09:58.480 --> 00:10:01.440
Sorry, I'll come down real quick.

s112
00:10:20.240 --> 00:10:22.720
I'll get it.

s113
00:10:23.760 --> 00:10:24.160
Hello.

s114
00:10:24.160 --> 00:10:24.640
Hello.

s115
00:10:24.640 --> 00:10:31.040
Uh he asked how we manage the reranking and organization of like billions of documents on the internet basically.

s116
00:10:31.040 --> 00:10:33.279
Um and it's a multi-stage process.

s117
00:10:33.279 --> 00:10:39.680
Um so essentially we pass in your query, we turn it into a query embedding and then once we've turned it into a query embedding.

s118
00:10:39.680 --> 00:10:43.600
There's several steps in between for keyword filtering uh alongside semantic search.

s119
00:10:43.600 --> 00:10:47.200
But um there's uh many different methods of doing this.

s120
00:10:47.200 --> 00:10:52.800
Uh but we do employ reranker ranking steps during our search uh in order to give you the best information.

s121
00:10:52.800 --> 00:10:54.800
Um we drop results that are not relevant.

s122
00:10:54.800 --> 00:10:56.399
Our index is highly curated.

s123
00:10:56.399 --> 00:11:05.360
So, we don't have as many documents as Google, but I bet you that we have very high quality documents in our like tens of billions of documents index.

s124
00:11:05.360 --> 00:11:09.279
Uh, but yeah, if you're interested about any specifics, feel free to talk to me after.

s125
00:11:09.279 --> 00:11:12.480
But yeah, cool.

s126
00:11:20.560 --> 00:11:22.560
Yeah.

s127
00:11:25.760 --> 00:11:27.279
Yeah.

s128
00:11:27.279 --> 00:11:27.519
Yeah.

s129
00:11:27.519 --> 00:11:30.000
So, we actually have this feature that's quite useful.

s130
00:11:30.000 --> 00:11:33.279
Um, so you could actually generate a schema.

s131
00:11:33.279 --> 00:11:37.040
So, you up to 10 fields on deep and then I I believe it's up to 100 on agents.

s132
00:11:37.040 --> 00:11:50.640
Uh, but you basically type whatever you want in natural language and you can generate it and then in this instance it'll generate and then you'll just click confirm and there you go.

s133
00:11:50.640 --> 00:11:56.079
So, and then we adhere to it and then if you want additional properties, you just set this to true and then you can get whatever you'd like.

s134
00:11:56.079 --> 00:11:59.040
You could also, my favorite demo is hobbies.

s135
00:11:59.040 --> 00:12:01.440
But yeah, um, great.

s136
00:12:01.440 --> 00:12:07.279
I'll take some questions off the stage, but uh, thank you all for listening today.