WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=X4w2Pkz5tDY

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/X4w2Pkz5tDY.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.719 --> 00:00:13.440
We're going to get going.

s3
00:00:13.440 --> 00:00:14.320
Nice to meet you everyone.

s4
00:00:14.320 --> 00:00:15.679
Uh my name is George.

s5
00:00:15.679 --> 00:00:24.480
Uh today uh we'll be going over a talk on search document search and uh how we get better results here at Llama Index.

s6
00:00:24.480 --> 00:00:25.680
Um my name is George.

s7
00:00:25.680 --> 00:00:28.240
I'm the head of engineering here at Llama Index.

s8
00:00:28.240 --> 00:00:30.400
We are a series A startup.

s9
00:00:30.400 --> 00:00:35.840
We focus quite a bit on document parsing, orchestration and uh knowledge management.

s10
00:00:35.840 --> 00:00:50.480
Um today the topic of the talk will center around how we've helped enterprise customers scale up search GP and generally manage context using aentic orchestration and just general infrastructure tooling.

s11
00:00:51.360 --> 00:01:10.320
Uh the plan today is going to be going through a discussion or a debate around uh how to best orchestrate and perform document search uh how to get the best results and we'll iterate through their uh technical challenges that we've seen all the way to a live demo at the end if we do have time.

s12
00:01:11.920 --> 00:01:12.400
All right.

s13
00:01:12.400 --> 00:01:24.560
So to get started um the question that I like to pose to everyone is when and why specifically do agents need vector context in some discussions or just context period.

s14
00:01:24.560 --> 00:01:32.640
Um and the major takeaway here is over the last few years models have gotten much better at orchestration and instruction following.

s15
00:01:32.640 --> 00:01:35.759
They're able to traverse local file systems.

s16
00:01:35.759 --> 00:01:41.680
They're able to work with skills MCPS in order to retrieve information in ways that are better than before.

s17
00:01:41.680 --> 00:01:45.840
But you still have your local search and your local context problems, right?

s18
00:01:45.840 --> 00:01:53.360
Um in order to manage large corpuses, in order to manage complex documents, easiest way is still to manage it locally.

s19
00:01:53.360 --> 00:02:05.680
Um and so there are kind of two evolving approaches to orchestration uh between GP or what I like to call kind of local file search and file execution or embeddings or pre-indexing

s20
00:02:05.680 --> 00:02:06.719
pre- retrieval.

s21
00:02:06.719 --> 00:02:15.520
Um we'll kind of talk through what the different approaches offer and when you might prefer one over the other or when it's appropriate to do a hybrid of both.

s22
00:02:15.920 --> 00:02:27.360
Uh to get started in understanding where this problem kind of popped up in industry u when cloud code launched uh you know there was a lot of discussion around the internal mechanics of how cloud code works.

s23
00:02:27.360 --> 00:02:37.519
It's actually a really interesting problem to think about how you're going to traverse and manage your local file system if you're going to go through coding right as a solution or a problem.

s24
00:02:37.519 --> 00:02:52.959
Um, and Anthropic's taken the approach of using your local file system prep read using glob management basically to manage your local file system and it's perfect for your local execution needs because code is easy to manage locally.

s25
00:02:52.959 --> 00:02:55.920
It's concise and it's already in text format.

s26
00:02:55.920 --> 00:03:13.519
But you'll also notice that um this tweet by Boris uh cloud code creator kind of highlights that uh cloud code experimented with a vector database didn't end up choosing it and the reasons aren't just for performance reasons but because of practical scaling and maintenance reasons.

s27
00:03:13.519 --> 00:03:20.000
It's hard to keep a pre-indexed vectorized index in sync especially as code keeps on updating.

s28
00:03:20.720 --> 00:03:30.720
And so specifically the solution that cloud code took which was no pre-indexing, no rag, no vector search, uh works really well but only because of a few characteristics of code.

s29
00:03:30.720 --> 00:03:36.560
Um they did not care about pre-indexing information because the code in your repo is usually small.

s30
00:03:36.560 --> 00:03:38.400
It's usually semistructured.

s31
00:03:38.400 --> 00:03:43.360
Usually your code leaves breadcrumbs for what else is being imported, what else is being tested.

s32
00:03:43.360 --> 00:03:48.799
Um and it leaves a natural hierarchy for how an agent might move about your file system.

s33
00:03:48.799 --> 00:03:54.400
Um and the largest part there is code is already on disk and local uh especially if your agent has context.

s34
00:03:54.400 --> 00:03:58.799
So why complicate it with a pre pre- retrieval system?

s35
00:03:59.439 --> 00:04:13.360
And the reality is when you switch over to larger company data sets uh where you have images, you have PDFs, you have powerpoints mixed in uh it's harder because there's not really a folder to grip.

s36
00:04:13.360 --> 00:04:17.759
Uh you can't really grip the folders across a company data set because it's not organized like code, right?

s37
00:04:17.759 --> 00:04:22.400
You have to think about how much effort has been put into curating the code that you write, right?

s38
00:04:22.400 --> 00:04:23.919
It must be syntactically sound.

s39
00:04:23.919 --> 00:04:25.440
It must have proper references.

s40
00:04:25.440 --> 00:04:30.400
And then you see your company database or you know whatever is in your data lake.

s41
00:04:30.400 --> 00:04:33.600
Um and you realize it's really hard to keep things in sync, right?

s42
00:04:33.600 --> 00:04:35.680
The scale is at a completely different level.

s43
00:04:35.680 --> 00:04:38.160
Uh you care quite a bit more about security.

s44
00:04:38.160 --> 00:04:40.240
Uh who can access what information.

s45
00:04:40.240 --> 00:04:41.759
Uh the format is not standardized.

s46
00:04:41.759 --> 00:04:49.759
you might have really complex PDFs, schematics, manufacturing diagrams mixed in and the scale just makes it so that you can't run this locally anymore.

s47
00:04:49.759 --> 00:04:52.000
So, how do how do you keep a system in sync?

s48
00:04:52.000 --> 00:04:52.160
Right?

s49
00:04:52.160 --> 00:04:54.800
The industry has gone through many different solutions.

s50
00:04:54.800 --> 00:05:00.560
Um, you can use MCP server, you can try to manage this using Anthropics CloudMD.

s51
00:05:00.560 --> 00:05:10.320
So, you in addition to having just this blob of a data source available somewhere, you keep track and let your agent know, hey, you can traverse the data this way, right?

s52
00:05:10.320 --> 00:05:18.639
So MCP skills, claude, claude, uh agent files, instruction files, these are all options for keeping track of some type of hierarchy.

s53
00:05:18.639 --> 00:05:29.759
But the major takeaway here is actually it really comes down to the scale of the data that you have and how much curation effort you're going to put in in order to maintain that retrieval system or that data management system.

s54
00:05:29.759 --> 00:05:31.199
That makes sense, right?

s55
00:05:31.199 --> 00:05:39.199
People are talking about context engineering, about harness, you know, making a harness, but at the end of the day, it's all about keeping the right primitives in place.

s56
00:05:39.199 --> 00:05:49.520
Um but in order to do that you have to understand what's the scale that you're dealing with right so if you have 100 to you know 100 or you know 100 to a thousand files

s57
00:05:49.520 --> 00:05:54.800
uh it may be appropriate to just keep down to local file search because your agent can manage this in context.

s58
00:05:54.800 --> 00:06:00.560
It won't burn too many tokens just iterating through files or managing the literal file names as it traverses.

s59
00:06:00.560 --> 00:06:05.600
Um but as you scale out right to thousands or millions of files that's not possible.

s60
00:06:05.600 --> 00:06:07.120
You don't want to burn those tokens.

s61
00:06:07.120 --> 00:06:08.880
uh you don't want to read the files every single time.

s62
00:06:08.880 --> 00:06:14.960
You want to have some needle in the haststack sense of roughly where you're looking for at least if it's not the needle in the haststack.

s63
00:06:14.960 --> 00:06:17.520
It's a rough compass on the direction you should be going, right?

s64
00:06:17.520 --> 00:06:29.680
Um and so we'll talk a bit about kind of what the right approach is uh between hybrid based search, semantic based search as well as uh your old school file traversal here.

s65
00:06:29.759 --> 00:06:36.080
Uh the other reason for why you might care about this is obviously as each model increases in its memory capabilities, right?

s66
00:06:36.080 --> 00:06:38.639
We got 1 million token context length now.

s67
00:06:38.639 --> 00:06:42.639
Uh before it used to be 200k, before that 100k and even smaller before that.

s68
00:06:42.639 --> 00:06:50.479
Um the important thing to think about here is actually as the model capabilities increase what you're willing to pay as you're going through this process, right?

s69
00:06:50.479 --> 00:06:53.840
Do you want to shove those tokens through the model's ingestion pipeline?

s70
00:06:53.840 --> 00:07:04.720
Do you want to uh send binary data over to your model for it to try to render in maybe its own engine and then extract data out of uh for your search?

s71
00:07:04.720 --> 00:07:07.280
um it just becomes extremely expensive, right?

s72
00:07:07.280 --> 00:07:10.319
In order to use your model context for ingestion and search.

s73
00:07:10.319 --> 00:07:22.880
Um and you know there have been solutions around how you can use sub agents right have one larger agent issue subcomands so that a smaller agent can deal with the memory context the memory bloat and just give a response back.

s74
00:07:22.880 --> 00:07:37.360
Um but the issue there is because each sub aent needs full context because sub aents uh even though they might use cheaper or simpler models uh if you've noticed in cloud code sometimes your you know main session may issue commands to weaker model

s75
00:07:37.360 --> 00:07:40.240
uh sub sessions in order to execute the jobs.

s76
00:07:40.240 --> 00:07:45.919
um the environments are actually rather tokenheavy and latencyheavy, right?

s77
00:07:45.919 --> 00:07:55.520
So in order to speed up your process and make sure that you're not burning tokens unnecessarily reprocessing documents each time, uh you may actually think about building out a pre-indexing

s78
00:07:55.520 --> 00:07:56.240
system.

s79
00:07:56.240 --> 00:07:59.759
Um and so we'll kind of talk about that here.

s80
00:07:59.759 --> 00:08:04.639
Um in kind of building out a harness and letting it agent choose what's going on, right?

s81
00:08:04.639 --> 00:08:25.440
So uh specifically we don't really want our system or any system that can scale out to be manually deciding uh should I use a hybrid rag approach or should I just individually search through files and the cleanest way that we found that in this era of engineering is

s82
00:08:25.440 --> 00:08:27.120
uh managing that with a harness.

s83
00:08:27.120 --> 00:08:40.159
So give your agent the tools needed to either search through specific files using your old school GP, your old school cat, file name filtering or metadata filtering in addition to semantic filtering.

s84
00:08:40.159 --> 00:08:43.519
Um, and that way you can get kind of the best of both worlds going through traversal.

s85
00:08:43.519 --> 00:09:00.320
uh so you can get the rough direction of which files you want to search and use semantic search to down scope your files and then using your agent's built-in file traversal capabilities to dig into the meat of the details so that you can actually ground your answers instead of hallucinating them.

s86
00:09:01.120 --> 00:09:12.720
So the five tools that we'll talk through today um in enabling kind of document search or uh complex document search uh as you might see in like insurance, finance, healthcare industries, right?

s87
00:09:12.720 --> 00:09:18.959
Where you just have a ton of unatategorized or semilablled data is what are the tools that we're going to use?

s88
00:09:18.959 --> 00:09:19.200
Right?

s89
00:09:19.200 --> 00:09:21.600
So the question first is where am I even going to look?

s90
00:09:21.600 --> 00:09:23.600
What's the compass direction we're going to go?

s91
00:09:23.600 --> 00:09:31.839
Um and then you have three tools that are going to be there and you know you have a lot more of these in your local file system to figure out what actually

s92
00:09:31.839 --> 00:09:37.600
is in this subset or this subcorpus of documents that we've identified with our compass direction.

s93
00:09:37.600 --> 00:09:37.839
Right?

s94
00:09:37.839 --> 00:09:46.880
So listing files, traversing files by metadata, uh gpping specific files to see if there are multiple keywords available that make a specific document more desirable.

s95
00:09:46.880 --> 00:09:51.040
These are all things that you could run in a pre-indexing pipeline or a live pipeline.

s96
00:09:51.040 --> 00:09:55.120
Um, and the last one is kind of, you know, what does the document actually say?

s97
00:09:55.120 --> 00:10:00.560
And specifically for multimmodal documents, you need a way to pre-render the documents.

s98
00:10:00.560 --> 00:10:13.200
And this is one kind of unique approach uh to multi-page or long schematics, which is in your pre-indexing process, one of the most important parts is to chunk or break your document down into digestible portions, right?

s99
00:10:13.200 --> 00:10:16.800
But that applies to not just the textual side, but also the rendering side.

s100
00:10:16.800 --> 00:10:25.360
uh because one of the hardest parts of free form document rendering is you actually have to rasterize or render the document into a human compatible view.

s101
00:10:25.360 --> 00:10:31.200
Uh these use cases are usually where you have an actual person that would have done this uh instead of AI, right?

s102
00:10:31.200 --> 00:10:36.399
So these schematics, these PDFs, these PowerPoints, they're not really built for pure machine consumption.

s103
00:10:36.399 --> 00:10:44.079
Um so giving the agent a pre-built screenshot index to search through is also very useful as you're performing rag.

s104
00:10:46.640 --> 00:10:47.200
All right.

s105
00:10:47.200 --> 00:10:51.519
So, we'll take a look at some of the details of the tool itself.

s106
00:10:51.519 --> 00:10:56.160
Um, so in terms of retrieval itself, right, which is the compass direction.

s107
00:10:56.160 --> 00:10:57.760
There are tons of approaches you can take here.

s108
00:10:57.760 --> 00:11:02.560
There's graph rag, there's traditional rag, there's kind of your old school keyword- based BM25 search.

s109
00:11:02.560 --> 00:11:05.920
Um, we we usually recommend a hybrid for enterprise use cases.

s110
00:11:05.920 --> 00:11:10.880
And that's just mostly because uh keyword-based search has its place for developers.

s111
00:11:10.880 --> 00:11:17.120
Keyword based search also has its place for AI agents that are looking for specific pre-indexed words in a corpus, right?

s112
00:11:17.120 --> 00:11:21.360
Um at the same time, semantic meaning and similarity is really important.

s113
00:11:21.360 --> 00:11:26.560
Uh combining the two through a hybrid based search usually gives to much gives you much better results.

s114
00:11:26.560 --> 00:11:32.240
Um you can see in various benchmarking data sets that you know just raw keyword or vector will get you to a baseline.

s115
00:11:32.240 --> 00:11:42.000
But when you put in a fusion, you know, you combine and weight uh keyword as well as vector search, you get better results uh because you aren't blinded by, you know, specifics of your search

s116
00:11:42.000 --> 00:11:43.120
uh method.

s117
00:11:43.120 --> 00:11:52.240
And re-ranking the final results, you know, you might take the top 100, re-rank them using an LLM or some other deeper process uh ultimately gets you much better retrieval results.

s118
00:11:52.240 --> 00:11:54.240
U but that's just one part of the pipeline, right?

s119
00:11:54.240 --> 00:11:55.600
That's the document search.

s120
00:11:55.600 --> 00:11:58.399
Uh we care about results.

s121
00:11:58.720 --> 00:12:11.920
Um and so tools two through four we'll talk through and kind of show later which is um once you have the compass direction of what are the what is the general group of files that might matter right what's the metadata that might associate most closely with your query

s122
00:12:11.920 --> 00:12:28.240
uh because your agent at the start might not know all the metadata or the extracted content of your system um it can then traverse through the file hierarchy uh search for similar files inside of your corpus that weren't even returned in that initial search in order to expand out and then further narrow down its results.

s123
00:12:28.240 --> 00:12:44.079
Um, so giving your agent the primitives of listing out the files GP and read that you have locally is really important in a shared and distributed file system, especially if it's one that you don't want to download onto your agents context every single time, which can get expensive.

s124
00:12:45.600 --> 00:12:49.200
And um, diving a bit deeper into the content itself, right?

s125
00:12:49.200 --> 00:13:02.880
Um this is where uh complex documents like you know stuff with charts, stuff that's scanned in, stuff that has uh you know schematics for example kind of breaks apart for your traditional document indexing and chunking workflow

s126
00:13:02.880 --> 00:13:08.480
because it is not a human read line of text right it's a separate representation.

s127
00:13:08.480 --> 00:13:17.760
Um and so thinking about how you do your pre-indexing flow and parsing flow uh so that you don't have to reread the documents every single time using your agent is also important.

s128
00:13:17.760 --> 00:13:23.839
you don't want to burn those uh you know binary encoded tokens every single time you want to read a new document.

s129
00:13:23.839 --> 00:13:29.600
So we usually recommend pre-indexing uh or pre-parssing your documents using uh structured parsing flow.

s130
00:13:29.600 --> 00:13:31.279
Um we have a product called Llama parse.

s131
00:13:31.279 --> 00:13:35.760
We've also open sourced a tool called light parse which handles structural content extraction.

s132
00:13:35.760 --> 00:13:45.839
So this is like if you have tables, charts or segmented uh information, you want to encode that as either markdown or just something that your agent really can understand the spatial reasoning of.

s133
00:13:45.839 --> 00:13:51.040
uh because when you have multimmodal data uh spatial reasoning is important.

s134
00:13:51.040 --> 00:13:57.920
Last one there is of course uh in your pre-parssing flow you usually want to generate those screenshots as you actually render the page right.

s135
00:13:57.920 --> 00:14:04.560
So whatever you're doing for pre-parssing you probably also want to store and persist screenshots as they're being generated from document rendering.

s136
00:14:04.560 --> 00:14:09.600
Uh it's not always clear if something is text or is you know got metadata associated with it.

s137
00:14:09.600 --> 00:14:12.720
You probably want to combine the two together.

s138
00:14:13.279 --> 00:14:18.160
Um and so parsing is an important part but it's not the only part of a pipeline.

s139
00:14:18.160 --> 00:14:34.800
Uh we usually you know advocate for people to think about how you can distribute parsing um and think about scaling out to improve your parsing pipeline right um there are some metrics that I would advocate for people to think about you know if they're relevant to your pipeline and uh kind of how you can optimize or

s140
00:14:34.800 --> 00:14:37.760
uh improve your pipeline for each of those dimensions.

s141
00:14:37.760 --> 00:14:49.440
So if you dump your data into a traditional OCR pipeline like Textract uh or you use a OCR engine like Tesseract um does it actually capture the tables the table representation?

s142
00:14:49.440 --> 00:14:52.000
How faithful is it to act the actual content layout?

s143
00:14:52.000 --> 00:14:52.240
Right?

s144
00:14:52.240 --> 00:14:58.240
Because if you get those wrong your LLM on its semantic reading side later down the line is going to pay for it.

s145
00:14:58.240 --> 00:15:06.880
Uh it might be fine at the start but if your LM doesn't understand the location and spatial reasoning it falls apart um at some point.

s146
00:15:08.240 --> 00:15:14.480
All right, so before we get to the demo, uh there are just a few caveats to think about with building out an actual production pipeline, right?

s147
00:15:14.480 --> 00:15:18.240
Which is when you have actual customers that care about using it and scaling it out.

s148
00:15:18.240 --> 00:15:20.959
There are a few things that will always be brought up.

s149
00:15:20.959 --> 00:15:25.279
Um there's like multi-tenency, you know, can one person see another person's data?

s150
00:15:25.279 --> 00:15:27.440
How does permissioning metadata get pulled in?

s151
00:15:27.440 --> 00:15:27.600
Right?

s152
00:15:27.600 --> 00:15:37.760
if I'm using Google Drive or if I'm using SharePoint, how do I pull in and yank that permission metadata without causing the whole system to grind to a halt or without reindexing and repparssing the pipelines each time?

s153
00:15:37.760 --> 00:15:40.320
Um, and that's related to the last one which is freshness, right?

s154
00:15:40.320 --> 00:15:50.320
This is kind of why Anthropic, a lot of other companies have kind of given up on trying to pre-index data uh because data freshness is really hard to achieve if your datas are complex,

s155
00:15:50.320 --> 00:15:55.440
your data is complex, and you also have metadata that needs to be up to date.

s156
00:15:56.000 --> 00:15:56.480
All right.

s157
00:15:56.480 --> 00:16:02.079
So, uh in order to break it down more easily, um we like to think about the synchronization process as three steps, right?

s158
00:16:02.079 --> 00:16:03.199
One is pure ingestion.

s159
00:16:03.199 --> 00:16:06.240
You know, can we get all your documents into a synchronized system?

s160
00:16:06.240 --> 00:16:12.079
Uh the second step is going to be parsing, which is how do we actually convert your document to a standardized format.

s161
00:16:12.079 --> 00:16:17.120
Uh for our use cases today, we'll talk through converting all documents into a markdown or a text format.

s162
00:16:17.120 --> 00:16:21.279
But there are also other formats you can use if you only care about multimodal representations.

s163
00:16:21.279 --> 00:16:37.839
Um and last stage is obviously uh rebungging that data and then exporting it into a pre-index state um in order to make it available for all of the tools and steps that we talked about before.

s164
00:16:39.600 --> 00:16:40.079
All right.

s165
00:16:40.079 --> 00:16:44.000
So uh I'm going to skip through this uh for the sake of time.

s166
00:16:44.000 --> 00:16:49.519
Um, one major thing that we want to highlight here is, you know, storing your data is actually quite important.

s167
00:16:49.519 --> 00:16:54.639
Um, in order to distribute your data out, you usually want some type of layered storage system.

s168
00:16:54.639 --> 00:16:59.360
Um, a lot of vector storage, uh, services store everything on memory.

s169
00:16:59.360 --> 00:17:07.280
Um, usually you're playing with some trade-off of do I want it on disk and then loaded into a faster memory store or do I want everything just in the memory store.

s170
00:17:07.280 --> 00:17:12.959
Um, for us, you know, we've dealt with this problem at the multi-tenant level to, you know, millions, tens of millions of documents.

s171
00:17:12.959 --> 00:17:21.039
Our recommendation here is to try to persist your data onto disk and then have a shielded memory layer in order to retrieve the data more efficiently.

s172
00:17:21.039 --> 00:17:24.079
Uh for our use cases today, we'll be looking at turbopuffer.

s173
00:17:24.079 --> 00:17:29.840
Um and turbopuffer as well as most vector stores will also handle uh permissioning, right?

s174
00:17:29.840 --> 00:17:31.440
So what's the metadata that you have?

s175
00:17:31.440 --> 00:17:32.720
How do you pull the data in?

s176
00:17:32.720 --> 00:17:39.200
Uh is actually quite an important part of your system success because when you have too many documents, you need to filter by metadata.

s177
00:17:39.200 --> 00:17:42.080
Um, so we push that into the vector storage layer.

s178
00:17:42.080 --> 00:17:47.679
You know, as the data is being ingested, let's just make sure we tag that into the vector stores.

s179
00:17:47.840 --> 00:17:48.320
All right.

s180
00:17:48.320 --> 00:17:51.440
So, uh, we're going to jump through a demo.

s181
00:17:51.440 --> 00:18:02.240
Um, and for the demo today, the the process to think about is, you know, how is the agent going to use the subcomponents in order to actually give you a response that's grounded

s182
00:18:02.240 --> 00:18:08.480
that actually can traverse multiple documents and actually gets you gets you a result that you're pretty happy with.

s183
00:18:08.480 --> 00:18:10.320
Um, all right.

s184
00:18:10.320 --> 00:18:12.880
So, we'll go through there here.

s185
00:18:12.880 --> 00:18:15.600
Uh, let's see.

s186
00:18:21.440 --> 00:18:22.000
All right.

s187
00:18:22.000 --> 00:18:31.919
So, in this demo, um, we will have a pre-index set of documents um that is available from a data source that we've defined ahead of time.

s188
00:18:31.919 --> 00:18:39.280
This is going to be 135 files uh that are derived from Alphabet's financial reports, right?

s189
00:18:39.280 --> 00:18:51.520
So, we have all these uh htm files, they're mixed in with raw PDF files and if you actually look at them, uh we've pulled in the metadata for where this information can be found.

s190
00:18:51.520 --> 00:19:04.400
So if you have file metadata available along with uh you know later on you can tag this with uh the year that it came from or the specific company it came from that gives your agents the capabilities to filter down uh the data.

s191
00:19:04.400 --> 00:19:15.679
Um and what we'll see here is uh in your hybrid retrieval flow right uh you'll have the option of if you want to be more keyword based or more semantic based in your retrieval flow.

s192
00:19:15.679 --> 00:19:26.640
Um let's just do a even balance for now and we're going to search for uh just any cash flow statements and we'll talk through kind of the tools that your agent would want to use along this process.

s193
00:19:26.640 --> 00:19:26.960
Right?

s194
00:19:26.960 --> 00:19:38.480
So um what you've seen here on the right is the set of chunks that are returned from our query as well as page screenshots that correspond to uh what is essentially

s195
00:19:38.480 --> 00:19:40.720
uh the page that it came from.

s196
00:19:40.720 --> 00:19:41.039
Right?

s197
00:19:41.039 --> 00:19:50.880
So these two together gives your agents the capability to uh filter down to a specific chunk and request context around that specific chunk in a much cleaner way than before.

s198
00:19:50.880 --> 00:19:57.600
Um you then might want to think about how you would GP for a specific uh piece of text in the file, right?

s199
00:19:57.600 --> 00:20:07.120
So what happens if you have a reax pattern that you're looking for uh you know now you're looking specifically for cash flow um and you can find that easily.

s200
00:20:07.120 --> 00:20:15.440
uh in order to find that in a specific file, all you need to do is issue a GP command and then read from that specific command kind of what that offset is

s201
00:20:15.440 --> 00:20:18.320
and then uh kind of what the max length is that you want.

s202
00:20:18.320 --> 00:20:19.919
So, uh this is a different file.

s203
00:20:19.919 --> 00:20:21.039
So, I'll just give an example.

s204
00:20:21.039 --> 00:20:27.120
You know, if we offset 500 characters and pull in the next thousand, we'll end up with this text being available, right?

s205
00:20:27.120 --> 00:20:33.360
So, with these two tools, this suite of tools together, your agent doesn't really need the files on a local file state anymore.

s206
00:20:33.360 --> 00:20:41.760
um you can interact with what is effectively a remote file storage without needing to spin up or download all the data that would need to be streamed to your system.

s207
00:20:41.760 --> 00:20:50.480
Um and that uh gives the agent a much more capable uh set of capabilities without actually downloading all the information and burning the tokens.

s208
00:20:50.480 --> 00:20:55.840
So we will uh go to a chat example and I'm going to pull it up here.

s209
00:20:55.840 --> 00:21:08.720
So uh give me the cash flow from as a table from 2021 to 2025 income as a table.

s210
00:21:14.960 --> 00:21:15.280
All right.

s211
00:21:15.280 --> 00:21:17.520
So give me a second here.

s212
00:21:17.520 --> 00:21:20.000
Run this again.

s213
00:21:26.960 --> 00:21:28.159
Give it a few seconds.

s214
00:21:28.159 --> 00:21:32.400
Uh, while that's running, I'm going to show you guys the end results.

s215
00:21:32.400 --> 00:21:34.480
Uh, let's go to the chat.

s216
00:21:34.480 --> 00:21:37.360
So, this is going to be in the chat flow timeline.

s217
00:21:37.360 --> 00:21:38.720
Um, let's see.

s218
00:21:38.720 --> 00:21:39.760
Still thinking there.

s219
00:21:39.760 --> 00:21:50.720
So, as the as an agent is running over the over the process of, you know, retrieving your answers from the corpus, it's going to look through uh your corpus by issuing search commands, right?

s220
00:21:50.720 --> 00:22:01.840
And as it's issuing the search commands, you want your agent to be able to review the screenshots, review the context, and then determine what is actually relevant for its search in a much

s221
00:22:01.840 --> 00:22:03.520
more holistic way, right?

s222
00:22:03.520 --> 00:22:10.400
So you get to think through the process, you get to read specific files, you get to search through files again and rescope your queries.

s223
00:22:10.400 --> 00:22:13.280
And in the end, that gives you a much cleaner result, right?

s224
00:22:13.280 --> 00:22:24.159
So uh in this agent response uh you can see the cash flow summary uh by you know and you can see that it's been pulled in from multiple files as part of its thinking process.

s225
00:22:24.159 --> 00:22:38.559
So when we do go into the traversal flow um you can usually find and ground your answers in uh in a specific file and that gives you much more confidence that your agents found the right component or the right chunk.

s226
00:22:38.559 --> 00:22:44.240
Um, you know, that's what we're looking for at the end of the day for agent-based retrieval.

s227
00:22:44.240 --> 00:22:45.919
All right, so we're going to jump back here.

s228
00:22:45.919 --> 00:22:48.960
We'll take a look at uh that session once it finishes streaming.

s229
00:22:48.960 --> 00:22:56.400
Uh, but you know, the idea here and the takeaway is that, you know, search helps you find information and the harness gives you the results that you can trust.

s230
00:22:56.400 --> 00:23:01.600
So, okay, if you're interested, um, feel free to, uh, visit the QR code here.

s231
00:23:01.600 --> 00:23:03.840
Thank you.

s232
00:23:19.751 --> 00:23:21.751
[music]