WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=xs-ob87TTzg

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/xs-ob87TTzg.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.639 --> 00:00:12.960
Hello.

s2
00:00:12.960 --> 00:00:13.519
Hello.

s3
00:00:13.519 --> 00:00:15.280
Perfect.

s4
00:00:15.280 --> 00:00:16.400
Yep.

s5
00:00:16.400 --> 00:00:18.640
So, perfect.

s6
00:00:18.640 --> 00:00:20.880
Yeah, this this volume is perfect.

s7
00:00:20.880 --> 00:00:27.439
It's just a website that I I registered the domain on Saturday just to do the registration of the workshop.

s8
00:00:27.439 --> 00:00:40.239
But for those of you who are going to follow along with me, just know that after you complete that that process in the in the workshop, you will get an invitation to the repository

s9
00:00:40.239 --> 00:00:45.360
and just view the invitation, accept it, and then you will be able to run the workshop.

s10
00:00:45.360 --> 00:00:47.680
We'll be running the workshop on GitHub Codespaces.

s11
00:00:47.680 --> 00:00:51.520
So, if you want to start that up, it takes like five minutes.

s12
00:00:51.520 --> 00:01:07.760
Um, just for reference, I'm going to use the first 30 minutes to give you an introduction to agent harness and lots of other concepts and then in the next hour and a half, we're going to go actually go through uh the workshop

s13
00:01:07.760 --> 00:01:08.960
together.

s14
00:01:08.960 --> 00:01:10.720
Sound good?

s15
00:01:10.720 --> 00:01:11.760
Okay, perfect.

s16
00:01:11.760 --> 00:01:19.680
And thank you, by the way, for being here because I know it's Monday 9:00 a.m. So, I commend you all for for being here.

s17
00:01:19.680 --> 00:01:22.080
Uh just two minutes before we begin.

s18
00:01:22.080 --> 00:01:25.119
So I will shut up.

s19
00:03:22.560 --> 00:03:23.040
All right.

s20
00:03:23.040 --> 00:03:25.920
Well, let's let's begin, guys.

s21
00:03:25.920 --> 00:03:32.080
So, thank you for being on this uh at this time here with me.

s22
00:03:32.080 --> 00:03:41.280
Um, I know it's 9:00 a.m. like I said, so hopefully I can and and we me and my team can make your time worthwhile.

s23
00:03:41.280 --> 00:03:48.239
At the end of this session, what I would like you to leave with is some knowledge on agent memory and agent harnesses.

s24
00:03:48.239 --> 00:03:55.360
How you can build your own agent harness, which is nowadays one of the hot topics on AI, I would say.

s25
00:03:55.360 --> 00:04:06.720
Um lots of people are talking about models constantly but the thing is that models like language models they are the frozen part of the reasoning right we just have to

s26
00:04:06.720 --> 00:04:17.600
accept what we are given and uh during the past few weeks you will if you're following the news you will have seen that this is this has never been more true than now.

s27
00:04:17.600 --> 00:04:21.199
So the harness is what we're going to talk about.

s28
00:04:21.199 --> 00:04:31.840
For those of you who weren't here, uh you can scan this QR code or go to that website workshopwaiting room.com and just register.

s29
00:04:31.840 --> 00:04:44.080
You will get an invitation to a GitHub repo and on this GitHub you will be able to create a GitHub code space where we will run the workshop and

s30
00:04:44.080 --> 00:04:46.160
you will have everything set up for you.

s31
00:04:46.160 --> 00:04:50.960
So just little introduction on on who I am.

s32
00:04:50.960 --> 00:04:53.919
I've been working for Oracle for seven years.

s33
00:04:53.919 --> 00:05:02.320
I've been a developer advocate for about four of them and you can find you know my talks and I'm very active on GitHub as well.

s34
00:05:02.320 --> 00:05:07.199
So if you're a GitHub user uh just check out my GitHub profile if you like.

s35
00:05:07.199 --> 00:05:10.320
Uh what is the highlight of my career so far?

s36
00:05:10.320 --> 00:05:15.360
I launched a course with Andrew Ang on agent memory.

s37
00:05:15.360 --> 00:05:24.560
So if you're interested on the memory components of what we're going to what we are going to discuss today, you could just check out that course if you'd like.

s38
00:05:24.960 --> 00:05:28.880
Um these are the things that we are going to uh talk through today.

s39
00:05:28.880 --> 00:05:29.360
Right.

s40
00:05:29.360 --> 00:05:39.120
Uh first of all, we're going to do an little introduction on what the agent stack is and then we're going to zoom into the data layer where the memory lives.

s41
00:05:39.120 --> 00:05:45.759
Then we're going to explore a little bit about the shapes, the different shapes that AI applications take nowadays.

s42
00:05:45.759 --> 00:05:52.720
Uh what an agent is uh followed by the seven different layers that make up an agent harness.

s43
00:05:52.720 --> 00:06:05.680
So we if you just follow these seven different structures, you will be able to create an agent harness, a minimal agent harness that you can connect any model to.

s44
00:06:05.759 --> 00:06:20.000
Then we're going to finally talk a little bit about continual learning and how as as Oracle we are uniquely positioned to help you achieve and and develop agent harnesses and and AI applications.

s45
00:06:20.240 --> 00:06:24.560
So the agent stack and by [clears throat] the way if you have any questions just raise your hand.

s46
00:06:24.560 --> 00:06:27.600
I'm very happy to to take questions as well.

s47
00:06:27.600 --> 00:06:36.720
So the agent stack, the agent stack like every agent, every AI agent sits on on these five layers.

s48
00:06:36.720 --> 00:06:41.039
You either you have an application which is the product surface, right?

s49
00:06:41.039 --> 00:06:47.039
What we interact with as users, you have data and that has lots of components.

s50
00:06:47.039 --> 00:06:57.680
You have memory, you have knowledge, you have retrieval, encoding, search, you have the model itself, the reasoning large language model that is behind everything.

s51
00:06:57.680 --> 00:07:04.880
Uh infrastructure which is the orchestration, what model do we serve depending on the reasoning effort that we need and things like that.

s52
00:07:04.880 --> 00:07:19.360
And then we have compute which is the cloud or GPUs and also the the database engine and the application the model the infrastructure and the compute these four layers except for the data

s53
00:07:19.360 --> 00:07:22.960
they are uh increasing increasingly commoditized.

s54
00:07:22.960 --> 00:07:24.639
What do I mean by that?

s55
00:07:24.639 --> 00:07:32.639
They are trying to take away the complexity from these layers out of out of our domain.

s56
00:07:32.639 --> 00:07:33.120
Right?

s57
00:07:33.120 --> 00:07:48.240
So the thing that we have the most control over when working with AI applications is actually the data and that's the part that we're going to focus in because the agent harness excels and you know lives very closely with the with the data layer.

s58
00:07:48.880 --> 00:07:53.280
So let's focus on the on the data layer.

s59
00:07:53.280 --> 00:08:00.720
The data layer is where an agent harness appears and it has you know many components.

s60
00:08:00.720 --> 00:08:11.919
The first component is the gateway and MCP is interesting because that's the one that connects an agent harness to data and tools.

s61
00:08:11.919 --> 00:08:23.520
So kind of think of of a large language model like a isolated thing that wouldn't be able to work at all if it didn't have access to things like

s62
00:08:23.520 --> 00:08:24.000
data.

s63
00:08:24.000 --> 00:08:24.800
Right?

s64
00:08:24.800 --> 00:08:38.080
So you you you then have like the memory layer, the semantic layer, retrieval layer, context layer, and tools and skills that are built on top of the gateway and MCP

s65
00:08:38.080 --> 00:08:38.320
layer.

s66
00:08:38.320 --> 00:08:45.519
Let's say for instance that you have a I don't know an application on your computer and your large language model doesn't have access to it.

s67
00:08:45.519 --> 00:08:48.000
For instance, Outlook, right?

s68
00:08:48.000 --> 00:09:01.120
So if you might want to have your large language model connect to that, you can create an MCP, you specify some functions and then the LLM all of a sudden is able to communicate with this program.

s69
00:09:01.120 --> 00:09:12.720
So gateway and MCP is like the layer that connects a large language model to the outside world or the factor to our computer or wherever you're working.

s70
00:09:12.720 --> 00:09:26.080
And AI applications today they take up four different um four different shapes right you have LLM chat bots which are very passive uh they just respond when you ask a question

s71
00:09:26.080 --> 00:09:42.240
you also have rag applications which are semi-passive because they have to do some kind of processing in the background but then you also have a passive nature of it and then you have the more active components of AI applications which are what we kind of use every day

s72
00:09:42.240 --> 00:09:59.839
like cloud code etc etc which are a combination of LLM driven workflows that provide automation and AI agents that provide autonomy and we'll go we'll we'll explain later what what I mean by that but first I want you to have

s73
00:09:59.839 --> 00:10:14.480
a very clear definition of what an AI agent is so to me an AI agent is essentially a model large language model plus a hardness The model itself will be the reasoning

s74
00:10:14.480 --> 00:10:17.760
and everything else will be the harness.

s75
00:10:17.760 --> 00:10:22.320
Um so this definition is something like this.

s76
00:10:22.320 --> 00:10:28.880
An autonomous entity whose cognitive functions are powered by a large language model for reasoning.

s77
00:10:28.880 --> 00:10:34.240
They are augmented by a database or files for memory.

s78
00:10:34.240 --> 00:10:42.160
They are extended through tools for actions and grounded in inputs that let it p perceive uh its environment.

s79
00:10:42.160 --> 00:10:47.760
So an agent is a a model plus the harness.

s80
00:10:47.760 --> 00:10:58.640
So if the agent is the model plus the harness in this diagram, right, we have reasoning and the reasoning is the part that we don't control.

s81
00:10:58.640 --> 00:11:02.160
It's the part that we are that's that's heavily subsidized.

s82
00:11:02.160 --> 00:11:04.079
the part that we rent.

s83
00:11:04.079 --> 00:11:13.360
If uh if you're like me, you're subscribed to every imaginable subscription on earth and that's the thing that we do not control, right?

s84
00:11:13.360 --> 00:11:25.040
We have no control over what we are offered and then we have memory tools and perception that actually we get some customiz customizability that we can do.

s85
00:11:25.040 --> 00:11:35.040
Um so the goal of harness engineering is to create reliable and predictable outputs over and over.

s86
00:11:35.040 --> 00:11:39.200
Whereas a reasoning model is very non-deterministic.

s87
00:11:39.200 --> 00:11:44.720
You might give it the same out the same input and it might produce different outputs every time.

s88
00:11:44.720 --> 00:11:46.720
Right?

s89
00:11:46.959 --> 00:11:59.120
So I said that you know in a AI applications the most typical ones nowadays cloud codeex uh and any other type that you can think of is a combination

s90
00:11:59.120 --> 00:12:02.160
of automation plus autonomy.

s91
00:12:02.160 --> 00:12:02.800
Why?

s92
00:12:02.800 --> 00:12:09.279
Well because automation provides reliability to a system and autonomy gives flexibility.

s93
00:12:09.279 --> 00:12:15.920
And this is a combination that is very convenient when we are developing ourselves.

s94
00:12:15.920 --> 00:12:17.200
By the way, raise of hands.

s95
00:12:17.200 --> 00:12:22.639
Who is working as an AI engineer or as an AI developer?

s96
00:12:22.639 --> 00:12:23.279
Oh my god.

s97
00:12:23.279 --> 00:12:24.959
Okay, good.

s98
00:12:24.959 --> 00:12:29.279
So, you must all have used one of these systems, right?

s99
00:12:29.279 --> 00:12:36.000
So, all of them they have this commonality which is they have autonomy and they have flexibility.

s100
00:12:36.000 --> 00:12:38.720
Um, yeah.

s101
00:12:38.720 --> 00:12:54.480
So the idea is that these systems right they are built on top of an a proprietary agent harness and an agent harness is nothing more than everything that we spoken about an AI agent

s102
00:12:54.480 --> 00:13:09.120
all the things that you need to do around that to enable it to produce reliable and repeatable outcomes right so the model itself nondeterministic same input different outputs every

s103
00:13:09.120 --> 00:13:21.040
But the harness, what we want to do with the harness is to turn this nondeterministic nature of a large language model and be able to produce reliable and repeatable results.

s104
00:13:21.760 --> 00:13:32.160
So the reasoning which is the part that we do not control, we're not going to focus actually the harnesses are built on top of models that are kind of swappable.

s105
00:13:32.160 --> 00:13:38.320
you just need like a common interface like an open AI protocol or the anthropic API specification, right?

s106
00:13:38.320 --> 00:13:47.279
All these things make it so that the model part is swappable and the harness is what we're going to focus on today.

s107
00:13:48.480 --> 00:13:52.800
So, seven things that make up an agent harness.

s108
00:13:52.800 --> 00:13:58.399
And as I said, we're not going to touch on the model layer because we have no control over it.

s109
00:13:58.399 --> 00:14:02.720
But let's go a little bit more uh in detail into each of these.

s110
00:14:02.720 --> 00:14:03.519
Right?

s111
00:14:03.519 --> 00:14:15.519
You need a storage layer on your agent harness that essentially determines where the data is going to live, where the memory physically lives.

s112
00:14:15.519 --> 00:14:29.440
uh and we'll see about this dilemma that has been going on about the last six months about files versus databases and why I think a hybrid uh combination of both is actually the best part.

s113
00:14:29.440 --> 00:14:38.720
Then you also have memory engineering components which are all the encoding, the search and the retrieval components of it.

s114
00:14:38.880 --> 00:14:53.360
And also the semantic layer which is kind of the the hidden things that happen or the hidden vocabulary that we assume that a large language model knows that is kind of proprietary to our companies or our knowledge.

s115
00:14:53.360 --> 00:14:57.440
What we don't say to the LLM kind of that's the semantic layer.

s116
00:14:57.440 --> 00:15:10.320
We'll lightly touch on agent loops and what an agent loop is and how to implement a very very minimalistic agent loop and finally go about some context engineering techniques

s117
00:15:10.320 --> 00:15:16.079
that you know keep the window as salient as possible the context window as salient as possible.

s118
00:15:16.079 --> 00:15:24.720
You want to minimize the context window as much as possible so that that the task that you're solving stays relevant.

s119
00:15:24.720 --> 00:15:38.160
So until here we have done an introduction to what an AI agent is right its use cases and now we're going to dive deeper into uh an agent harness and each one of the individual

s120
00:15:38.160 --> 00:15:39.120
components.

s121
00:15:39.120 --> 00:15:52.399
So the model layer right the frozen reasoning core I say it's frozen because typically the weights of a model don't change and I say typically because uh we are actually

s122
00:15:52.399 --> 00:15:55.279
I'm actually in the process with Casio sitting right there.

s123
00:15:55.279 --> 00:16:01.199
We're going to record a new course with Andrew on continue learning for a agents.

s124
00:16:01.199 --> 00:16:04.800
So if you're interested just check that out in a couple couple weeks.

s125
00:16:04.800 --> 00:16:18.800
But the thing is that 99.9% of the time you will have a model and the weights of the model will never change uh unless you have millions of dollars or you know a lot of time or GPUs

s126
00:16:18.800 --> 00:16:21.120
it's very hard to change the weights of a model.

s127
00:16:21.120 --> 00:16:27.440
So there are other ways in which you can affect the reasoning without actually changing the weights of the model.

s128
00:16:27.440 --> 00:16:33.920
But this is motivation for for the continual learning part that we will see in the workshop.

s129
00:16:34.000 --> 00:16:47.680
where does the memory live right the files versus databases dilemma that we've been having uh since January kind of some people are very maximalists of files and some of us are

s130
00:16:47.680 --> 00:17:00.800
well oh I will not include myself but some of some people are also maximalists of the database um and you know both things are right files have convenient things convenient characteristics

s131
00:17:00.800 --> 00:17:19.520
and also the databases So the files right they are very easy to like they are very they match the model's instincts they are very easy to create they are very easy to insert and append data into files right it's very it has a very unstructured

s132
00:17:19.520 --> 00:17:31.039
nature to it and databases on the other hand or the conception that people have is that they have a very structured way right but that's when you're thinking about SQL like SQL

s133
00:17:31.039 --> 00:17:38.160
And the thing is that you don't actually need to like choose one or the other.

s134
00:17:38.160 --> 00:17:40.080
You can actually use both of them.

s135
00:17:40.080 --> 00:17:48.320
Like files are attractive because the models picks them and they kind of work very very easily with operating systems.

s136
00:17:48.320 --> 00:17:50.080
They follow posic semantics.

s137
00:17:50.080 --> 00:17:57.440
So they are compatible on Debian, Ubuntu, any other operating system that you might want.

s138
00:17:57.440 --> 00:18:00.320
And they also have some disadvantages.

s139
00:18:00.320 --> 00:18:04.080
for instance they don't have transactional consistency.

s140
00:18:04.080 --> 00:18:08.400
So this is uh one of the problems that I wanted to talk about.

s141
00:18:08.400 --> 00:18:18.160
If you are a degenerate like me and you are working I don't know with 8 16 32 agents at a time.

s142
00:18:18.160 --> 00:18:25.120
Uh the problem with this is that files cannot be modified and inserted and modified at the same time.

s143
00:18:25.120 --> 00:18:32.320
So what is the solution nowadays to not having transactional consistency and working with files?

s144
00:18:32.320 --> 00:18:35.120
Any suggestions?

s145
00:18:35.600 --> 00:18:36.240
Work trees.

s146
00:18:36.240 --> 00:18:36.640
Exa.

s147
00:18:36.640 --> 00:18:37.600
Exactly.

s148
00:18:37.600 --> 00:18:46.960
So agents what they when they want to modify a file but another agent is working on this thing, they just create a different work tree, right?

s149
00:18:46.960 --> 00:18:55.520
They will do all the progress in the work tree and then after the implementation is done they will merge to master or the merge to main sorry.

s150
00:18:55.520 --> 00:19:14.640
So this is the way that is a workaround against not having transactional consistency right and you also have other characteristics like for instance hybrid search um this is not available on files but is very very easily achieved on databases.

s151
00:19:14.640 --> 00:19:17.280
Um you don't have backups either.

s152
00:19:17.280 --> 00:19:23.200
So if your operating system gets corrupted or something, you will just lose everything.

s153
00:19:23.200 --> 00:19:30.160
And these are things that the database fixed 354 years ago and people have kind of forgotten about that.

s154
00:19:30.160 --> 00:19:38.640
So what I want to do is to give you kind of the best of each of the of the implementations, right?

s155
00:19:38.640 --> 00:19:43.200
You can get the benefits of files and the benefits of databases in the same place.

s156
00:19:43.200 --> 00:19:44.320
And we'll see why.

s157
00:19:44.320 --> 00:19:46.080
But these are some of the advantages, right?

s158
00:19:46.080 --> 00:19:49.200
You have acid consistency.

s159
00:19:49.200 --> 00:19:55.760
So atomic operations, consistent operations, isolated and durable.

s160
00:19:55.760 --> 00:19:57.039
You have high availability.

s161
00:19:57.039 --> 00:20:04.400
You might replicate the database and let it be, you know, in three different places in the world with a replication factor.

s162
00:20:04.400 --> 00:20:07.440
You also have vector search, which is very easy.

s163
00:20:07.440 --> 00:20:14.640
In files, you just have to do like regular expression matching or a derivative of that.

s164
00:20:14.640 --> 00:20:17.440
and you know lots of other lots of other things.

s165
00:20:17.440 --> 00:20:35.840
So what we will do on the on the actual workshop is we're going to run an actual example of trying to modify a file with three different agents that will be working on the same file and trying to update a counter on this file and let's see who's faster.

s166
00:20:35.840 --> 00:20:40.400
I know the answer of course but you will you'll get to know it later.

s167
00:20:40.400 --> 00:20:54.000
But what I want to introduce to you is that we have a thing called the Oracle DBFS or the database file system where you can store files inside the database on a file system and that

s168
00:20:54.000 --> 00:20:57.679
gives you you know lots of the advantages that we said.

s169
00:20:57.679 --> 00:21:01.679
You will get files with acid trans uh transactional consistency.

s170
00:21:01.679 --> 00:21:08.880
You will get vector search relations security high availability etc. Right?

s171
00:21:08.880 --> 00:21:24.880
And my suggestion is that since a hybrid system works best um we can have things like for instance uh short-term memory right that lives in files and when something needs to be promoted into a long-term memory for instance user preferences

s172
00:21:24.880 --> 00:21:37.520
things like this they can go into a more structured space like a database and this is what we'll do in the workshop for the encoding the search and the retrieval which was another

s173
00:21:37.520 --> 00:21:41.760
of the components in the agent harness.

s174
00:21:41.760 --> 00:21:55.840
We also have a lang chain integration that I want to mention called langchain Oracle DB that makes it very easy to insert into vector stores, search in the vector stores and retrieve from the vector stores.

s175
00:21:55.840 --> 00:22:00.159
So also we have another thing called in database embeddings.

s176
00:22:00.159 --> 00:22:02.799
Have you ever heard about in database embeddings?

s177
00:22:02.799 --> 00:22:03.760
Yes.

s178
00:22:03.760 --> 00:22:04.400
Okay.

s179
00:22:04.400 --> 00:22:18.320
So in database embeddings is very convenient especially for enterprise customers because you will get the embedding model inside the database so that when you're doing embeddings you don't have to call a third party service

s180
00:22:18.320 --> 00:22:24.400
and that's very convenient for isolate like data retention and data security purposes.

s181
00:22:25.600 --> 00:22:41.200
So this is what a an an embedding searching and reranking model would look like from a chatbot interface for instance right you have a lot of documents then you kind of use a benccoder so a benccoder is essentially an embedding model

s182
00:22:41.200 --> 00:22:53.600
you will create eno embeddings out of these document you will split them and create vectors put it into a vector store right and then you will get a user prompt a user query a question

s183
00:22:53.600 --> 00:23:00.320
you can also create an embedding out of that and then compare it to what you had previously on your vector store.

s184
00:23:00.320 --> 00:23:06.960
And this is how you get the most relevant vectors in in an answer.

s185
00:23:06.960 --> 00:23:19.600
Then you will run an a cross encoder which is a reranker and take a look at the question plus the result and that's how questions are answered kind of in in in rag applications.

s186
00:23:19.600 --> 00:23:20.400
Right?

s187
00:23:20.400 --> 00:23:32.720
So these things we are also going to to touch briefly on the workshop and the things that you know imagine this rag application from the beginning from the documents

s188
00:23:32.720 --> 00:23:45.520
you create you do lots of things right you do tokenization then you create the embeddings um you have to do things like the duplicating the data normalization uh personal

s189
00:23:45.520 --> 00:23:58.799
like redacting personally identifiable information lots of things right and then on your on your store you have textual data from the documents you have metadata which is typically stored in JSON

s190
00:23:58.799 --> 00:24:15.840
you have vectors which are represented as as dense embeddings of 32 uh bits like you have so many types of data that you need to work on that typically what people have is for instance I don't know I will not name names so I don't get in trouble but

s191
00:24:15.840 --> 00:24:20.799
you know you might need like different databases for each one of these, right?

s192
00:24:20.799 --> 00:24:22.159
You you get what I mean?

s193
00:24:22.159 --> 00:24:32.080
So there is a very high nowadays very high data synchronization logic overhead for AI engineers or even for agents, right?

s194
00:24:32.080 --> 00:24:37.679
So lots of maintenance required and and and lots of engineering effort on it.

s195
00:24:37.679 --> 00:24:40.960
And this is something that we want to avoid.

s196
00:24:40.960 --> 00:24:46.880
So what I want you to do today is just to try us out as Oracle.

s197
00:24:46.880 --> 00:24:48.799
Uh try our database.

s198
00:24:48.799 --> 00:24:54.080
We have support for every type of data imaginable that you can think of.

s199
00:24:54.080 --> 00:24:56.480
Um we are called the converg database.

s200
00:24:56.480 --> 00:25:08.720
We are the only converg database in the market that we support JSON, we support relational, we support uh spatial, graph, JSON, anything that you can think of you'll be able to to create with us

s201
00:25:08.720 --> 00:25:17.200
and you'll get like one database engine uh one query interface and uh one single development stack.

s202
00:25:17.200 --> 00:25:18.960
everything will be in the same database.

s203
00:25:18.960 --> 00:25:28.640
So also for data security purposes, you just have to you know secure and save all your data in one place.

s204
00:25:28.640 --> 00:25:32.559
So a single attack vector is what you need to worry about.

s205
00:25:32.559 --> 00:25:35.360
You don't need to worry about updating five different database.

s206
00:25:35.360 --> 00:25:40.559
You can just worry on securing one database.

s207
00:25:40.559 --> 00:25:50.799
So we can be the engine of your AI applications, not just a single step, which is what people think of when they're working with databases, right?

s208
00:25:50.799 --> 00:25:54.080
Um, yeah.

s209
00:25:54.080 --> 00:26:11.039
So agent memory, agent memory is essentially a description of all the mechanisms and the systems that allow an agent to retain, reuse, refine and recall information.

s210
00:26:11.039 --> 00:26:14.640
We want to reuse the data and refine it in the process.

s211
00:26:14.640 --> 00:26:15.919
But we want to reuse the data.

s212
00:26:15.919 --> 00:26:26.799
So that the next time that an agent or us as engineers we are working on a problem that took us three hours, the next time that we observe this problem,

s213
00:26:26.799 --> 00:26:31.760
the problem becomes easier either for AI agents or for us.

s214
00:26:31.760 --> 00:26:34.960
And agent memory has lots of lots of components, right?

s215
00:26:34.960 --> 00:26:41.840
We have short-term memory, we have long-term memory, and then we have shared memory, which is something that's relatively new.

s216
00:26:41.840 --> 00:26:54.799
Um and this shared memory is kind of what happens when a sub agent is communicate with it is communicating with it with its parent for instance or two agents are trying to collaborate on solving one specific problem uh together

s217
00:26:54.799 --> 00:27:09.520
and then what I want you to to see is that depending on the type of memory or or or the type of thing that we want to store it will be in one place or the other right so short-term memory is kind of ephemeral

s218
00:27:09.520 --> 00:27:15.200
is very shortlived And it's very useful for things that are happening right now.

s219
00:27:15.200 --> 00:27:19.039
For instance, the to-do list on a coding agent, right?

s220
00:27:19.039 --> 00:27:25.679
It's happening right now, but you actually don't want to save that u you know in long term.

s221
00:27:25.679 --> 00:27:38.640
But then there are things like for instance episodic memory things that previous conversations that you've had that's very useful to have for instance um I don't know if you if you use clot some people are using cloud

s222
00:27:38.640 --> 00:27:49.440
um here uh but in in cloud you might take your previous conversations and try to refine all your workflows and your skills based on the things that you've done in the past.

s223
00:27:49.440 --> 00:27:54.559
So this is something that makes sense to to save in the long run, right?

s224
00:27:54.559 --> 00:28:00.320
You also have things like procedural memory, previous workflows that have worked very well for your system.

s225
00:28:00.320 --> 00:28:08.080
For instance, you worked on this front end and then you created a very beautiful design that you like.

s226
00:28:08.080 --> 00:28:19.520
You might take the whole conversation and turn that into a workflow that is repeatable and re reusable so that the next time you're working on a front end the results will be similar to the previous one.

s227
00:28:19.520 --> 00:28:20.480
Right?

s228
00:28:20.480 --> 00:28:30.159
So these are the things that we we will see on the workshop and some people say okay why do I even need all of these like people that are very

s229
00:28:30.159 --> 00:28:47.520
that have animosity towards agent memory people say okay let's just put like 15 million uh context window even though it's not not possible yet but some people really believe that this is the the thing right but the context window is a type of short-term memory

s230
00:28:47.520 --> 00:28:51.520
so it's useful for some things but not for all of them.

s231
00:28:51.520 --> 00:29:02.320
And one of the problems that that happen with with working with a with a context is this thing called context rot or context degradation over time.

s232
00:29:02.320 --> 00:29:12.799
And what happens is that the more things that you put into the context window, the less attention there will be for each one of the things that are in the context.

s233
00:29:12.799 --> 00:29:22.240
So at the beginning of a conversation and this is a a famous problem that that the context window has is at the beginning of the conversation it will stay on track

s234
00:29:22.240 --> 00:29:26.960
a lot because you are just you just started the conversation.

s235
00:29:26.960 --> 00:29:29.760
So let's say that for instance like in school, right?

s236
00:29:29.760 --> 00:29:38.640
Or if I'm having a conversation with you, um I might have the a chat with you for 30 minutes and your attention to me is very very high because

s237
00:29:38.640 --> 00:29:40.720
I just started speaking.

s238
00:29:40.720 --> 00:29:45.440
But if the conversation goes on for eight hours, then you want to punch me, right?

s239
00:29:45.440 --> 00:29:50.000
Because I haven't shut up and you haven't learned almost anything at the end.

s240
00:29:50.000 --> 00:29:54.880
And the the problem is that attention like us humans is very limited.

s241
00:29:54.880 --> 00:30:07.120
So the more things that you put in the context window, the attention matrix of the neural network will all also degrade and it will like scale quadratically because the attention matrix

s242
00:30:07.120 --> 00:30:09.120
you know is one token.

s243
00:30:09.120 --> 00:30:14.320
It's essentially a reference of one token for every other token in the context window.

s244
00:30:14.320 --> 00:30:22.720
So the bigger the context window is, the matrix scales on the number of rows and on the number of columns as well, which is a problem.

s245
00:30:22.720 --> 00:30:33.279
So you want to keep the context window as small as possible to avoid context rot and memory engineering the components of memory engineering.

s246
00:30:33.279 --> 00:30:50.960
Um so it's like designing building and doing everything around building agent memory for AI agents and we want to retain recall reuse and refine this data in some type in some way.

s247
00:30:50.960 --> 00:31:00.640
So it is a discipline right and here we have Valentine for instance and we have people from Oracle uh my colleagues all over the the room.

s248
00:31:00.640 --> 00:31:04.000
So if you see them you can say hi to them.

s249
00:31:04.000 --> 00:31:11.200
Valentine here he's working on the development or he worked on the development of this agent memory package.

s250
00:31:11.200 --> 00:31:14.000
So if you have any questions about this you can ask him.

s251
00:31:14.000 --> 00:31:26.720
Um, OAMP or Oracle agent memory package is the managed answer that we have in Oracle to lots of the problems that you will find when working with this type of data.

s252
00:31:26.720 --> 00:31:34.480
For instance, as an engineer, if you do not have a managed solution, you need to make a lot of decisions.

s253
00:31:34.480 --> 00:31:45.200
You need to see when do I do context compaction, when do I summarize, how do I write it, the summarizer, what do I keep, what do I not keep,

s254
00:31:45.200 --> 00:31:56.240
um what do I extract from my previous conversations and when, how many tokens do I use for this problem and lots of these things, right?

s255
00:31:56.240 --> 00:32:04.159
With OAMP, an Oracle agent memory package, you can actually just do all of this in one single line of code.

s256
00:32:04.159 --> 00:32:13.360
And we want to make it easier so that we reduce the cognitive load of AI engineers and AI agents as well.

s257
00:32:13.360 --> 00:32:19.919
And with this context card uh thing, you will get something like what you see on the left.

s258
00:32:19.919 --> 00:32:26.000
And this is kind of an explanation of what each part does on it.

s259
00:32:26.000 --> 00:32:35.120
But essentially the the topics uh that you see for instance from any conversation you can create a context uh card from the thread or from the conversation.

s260
00:32:35.120 --> 00:32:37.600
The topics will orient the model.

s261
00:32:37.600 --> 00:32:50.000
The summary will compact the thread and it will state the current intent of an AI agent and the relevant information has three different parts which is the the facts,

s262
00:32:50.000 --> 00:32:55.200
the preferences and the memories that are associated to this conversation.

s263
00:32:55.200 --> 00:33:08.399
uh then you have the episodic memories that explicit explicitly track the unanswered question that is going on right now and the recent messages give like local context to the model.

s264
00:33:08.399 --> 00:33:18.240
So whenever you're feeling like unsure what do I need to do right now with the data that I have or this conversation you might use the Oracle agent memory package

s265
00:33:18.240 --> 00:33:23.279
on on Python and it is all assembled by one single call.

s266
00:33:23.279 --> 00:33:24.799
Yes.

s267
00:33:24.799 --> 00:33:29.120
So you're not suggesting that this goes directly into the model.

s268
00:33:29.120 --> 00:33:32.399
This is actually something that is used by the harness.

s269
00:33:32.399 --> 00:33:33.519
Exactly.

s270
00:33:33.519 --> 00:33:35.039
What's going into the model?

s271
00:33:35.039 --> 00:33:35.360
Yes.

s272
00:33:35.360 --> 00:33:35.840
Yes.

s273
00:33:35.840 --> 00:33:41.120
The component that you're talking about that uses it knows about the structure.

s274
00:33:41.919 --> 00:33:42.320
Exactly.

s275
00:33:42.320 --> 00:33:42.720
Exactly.

s276
00:33:42.720 --> 00:33:44.640
So this structure

s277
00:33:44.640 --> 00:33:50.880
this is an abstraction that we build on top of the model because the model we have no control over in most cases, right?

s278
00:33:50.880 --> 00:34:01.600
Um the thing is that we can build on top of that whatever abstractions we want to make the model perform as reliable as reliably as possible.

s279
00:34:01.600 --> 00:34:03.600
Yep.

s280
00:34:04.240 --> 00:34:06.960
So the memory and the semantic layer they kind of work together.

s281
00:34:06.960 --> 00:34:10.639
We I know we talked only about memory components.

s282
00:34:10.639 --> 00:34:19.359
Uh I'm going to walk you quickly because I'm I don't have a lot of time um to the semantic layer to the semantic components of it.

s283
00:34:19.359 --> 00:34:19.599
Right?

s284
00:34:19.599 --> 00:34:27.919
And the semantic layer is the the meaning of what's going on behind it that you kind of assume that that happens.

s285
00:34:27.919 --> 00:34:28.639
Right?

s286
00:34:28.639 --> 00:34:32.079
So anyone from Germany?

s287
00:34:32.079 --> 00:34:32.320
Okay.

s288
00:34:32.320 --> 00:34:36.000
So I apologize for my pronunciation but I'm going to try.

s289
00:34:36.000 --> 00:34:45.280
So the is like the ambient of a model and this was coined by Jacob von Wexul.

s290
00:34:45.280 --> 00:34:57.440
Um and this guy said that essentially every organism in the world uh that is living perceives its reality through a lens and the lens is what it has access to.

s291
00:34:57.440 --> 00:35:01.839
For us humans for instance, we have our eyes, our senses, right?

s292
00:35:01.839 --> 00:35:09.680
So everything that we perceive and everything that we leave, all our experiences are seen through this lens, right?

s293
00:35:09.680 --> 00:35:22.960
And an agent doesn't have human like senses, but it has also a kind of semantic layer or a semantic lens that everything that you ask it is filtered through.

s294
00:35:22.960 --> 00:35:26.079
And this lens is essentially what you train it with.

s295
00:35:26.079 --> 00:35:29.520
And then what you also give context to.

s296
00:35:29.520 --> 00:35:43.680
So everything that you talk to the agent right when you talk to the agent this agent will look it through the belvelt and the semantic layer is essentially the agent's velvet.

s297
00:35:43.680 --> 00:35:58.640
So the onset is what what what I want to focus on like for instance organizational knowledge or enterprise knowledge things that when you're working with a colleague you don't mention this because this is already

s298
00:35:58.640 --> 00:36:04.640
you know known between you and your colleague you need to specify everything that you work on every day.

s299
00:36:04.640 --> 00:36:16.000
If there was someone else like a a child that wanted to that [clears throat] wanted to start uh working with you, you would have to specify everything very very in detail.

s300
00:36:16.000 --> 00:36:20.000
These are all the answers and this is what the semantic layer captures.

s301
00:36:20.000 --> 00:36:34.000
So the answer is the actual tribal knowledge that that an enterprise has right institutional knowledge as well like how the data is modeled, how the queries are executed, what is the metadata,

s302
00:36:34.000 --> 00:36:38.960
all these things are in the semantic layer.

s303
00:36:38.960 --> 00:36:45.440
Um and quickly just for you to know that we're also going to implement a very minimalistic agent loop.

s304
00:36:45.440 --> 00:36:48.560
And an agent loop is like the driver of the model, right?

s305
00:36:48.560 --> 00:37:02.320
It lets a model be kind of independent and autonomous and it is what makes a model it it turns a model into an agent, right?

s306
00:37:02.320 --> 00:37:14.240
And this is like the simplest agent look that you can find is kind of this observing and reasoning and then acting part that happens all of the time.

s307
00:37:14.240 --> 00:37:15.599
All of the time.

s308
00:37:15.599 --> 00:37:23.119
And you know it's it has to be failure uh failure resistant so that we never exit the loop.

s309
00:37:23.119 --> 00:37:27.040
This is the idea that the agent you give autonomy to the agent.

s310
00:37:27.040 --> 00:37:31.680
Well you give autonomy to the model so that it becomes the agent.

s311
00:37:31.680 --> 00:37:38.160
And then on the context engineering part which is the last part of the seven layers of an agent harness.

s312
00:37:38.160 --> 00:37:41.040
You also have lots of things that you can do.

s313
00:37:41.040 --> 00:37:47.280
For instance, the toolbox pattern and the skillbox pattern which we are this is on the workshop as well.

s314
00:37:47.280 --> 00:37:56.960
And these are ways in which you can store the available tools and the available skills of a model so that they are retrieved optimally.

s315
00:37:56.960 --> 00:38:07.119
And what you want to do is only retrieve the tools and the the skills when they are actually needed and put it on the context window only when needed.

s316
00:38:07.119 --> 00:38:23.760
every iteration of an agent loop you will see if this is actually the right place and then if it's not you can just take them out temporarily so uh oh sorry I thought I saw a question

s317
00:38:23.760 --> 00:38:31.760
so we will see all of this in the workshop I don't want to take up too much time but the idea is that we will assemble the context at every iteration

s318
00:38:31.760 --> 00:38:45.520
on the agent loop and then continue learning is the part that that we talked before about the ability to get better over time with the things that we've done with a model, right?

s319
00:38:45.520 --> 00:38:50.400
So a frozen model as we saw it doesn't get better, right?

s320
00:38:50.400 --> 00:39:01.920
But there are ways in which we can make an agent improve in the weights which is the parts that we are going to work on uh in the representation

s321
00:39:01.920 --> 00:39:02.480
part.

s322
00:39:02.480 --> 00:39:11.839
So on the embedding and the reranking part and also in the context window and these are the three types of continual learning techniques.

s323
00:39:11.839 --> 00:39:22.160
We are going to focus on the workshop on the context and the token space because it's one of the easiest ones and one of the least expensive ones as well.

s324
00:39:22.160 --> 00:39:26.800
I assume no one is a millionaire or not many of us are millionaires.

s325
00:39:26.800 --> 00:39:35.680
So this is also the most achievable and the most realistic way to change the model behavior over time.

s326
00:39:36.160 --> 00:39:47.040
So without further ado, I just want to introduce you to this part which is skill promotion and workflow workflow promotion.

s327
00:39:47.040 --> 00:39:53.359
So those skills or those workflows that you've done for three four hours, right?

s328
00:39:53.359 --> 00:40:00.160
you've been working for the whole day on a workflow, you were able to successfully do your job.

s329
00:40:00.160 --> 00:40:08.079
Um, and then these things can actually be retrieved, they can be stored into the memory components that we'll see.

s330
00:40:08.079 --> 00:40:10.720
And then we will see about skill promotion.

s331
00:40:10.720 --> 00:40:20.079
So if we promote a skill for instance, we can promote it through a distillation process and create a better skill.md than the original.

s332
00:40:20.079 --> 00:40:25.280
So we will retire the old version and update from the new version.

s333
00:40:25.280 --> 00:40:38.160
And this allows us to do some kind of continual learning on our own skills that turn them into more customized skills for ourselves, for our tone, our way to work,

s334
00:40:38.160 --> 00:40:40.320
our preferred like our preferences.

s335
00:40:40.320 --> 00:40:45.760
Like for instance, let's use this specific library because I really like the look and feel of it.

s336
00:40:45.760 --> 00:40:53.440
let's use this uh specific database engine because it has less bugs or I found it easier to work with.

s337
00:40:53.440 --> 00:41:00.800
All these things can be promoted into reusable and improvable skills over time.

s338
00:41:00.800 --> 00:41:17.839
So this is what the whole harness would look like at the end of the of the workflow uh sorry at the end of the workshop and hopefully what you leave this uh you'll leave today with a better understanding of all the specific components

s339
00:41:17.839 --> 00:41:20.960
that make up an agent harness.

s340
00:41:20.960 --> 00:41:27.680
So let me go to here before I forget.

s341
00:41:27.680 --> 00:41:39.920
If you are interested in the Oracle agent memory package or are working on the agent memory package, we have a discord server in which you can just chat with us as stuff.

s342
00:41:39.920 --> 00:41:44.560
If you're a Discord user, just feel free to to join this discord server.

s343
00:41:44.560 --> 00:41:46.480
I'll put the link later as well.

s344
00:41:46.480 --> 00:41:54.960
But without further ado, let's begin with the actual workshop and I will tell you how.

s345
00:41:54.960 --> 00:42:00.800
So let me show you first what we are going to build right this is an app book.

s346
00:42:00.800 --> 00:42:05.280
Oh sorry uh you don't see this.

s347
00:42:06.079 --> 00:42:08.079
Yeah.

s348
00:42:15.760 --> 00:42:18.960
First, yes, the workshop instructions, right?

s349
00:42:18.960 --> 00:42:30.480
So, for those of you who weren't here, you can just go into this website, register with your GitHub user, and then you will get an invitation like this to

s350
00:42:30.480 --> 00:42:36.000
a GitHub repo and from here we will create a GitHub code space.

s351
00:42:36.000 --> 00:42:46.800
Uh, so please make sure to do that right now and all my colleagues are are around the room to answer any questions that you might have during the creation of the code space, etc.

s352
00:42:49.920 --> 00:42:50.560
Yes.

s353
00:42:50.560 --> 00:42:52.560
Yeah.

s354
00:42:55.520 --> 00:42:56.240
The internet.

s355
00:42:56.240 --> 00:42:58.560
Are you having issues with the with the internet?

s356
00:42:58.560 --> 00:42:58.800
Okay.

s357
00:42:58.800 --> 00:43:00.400
Let's see.

s358
00:43:00.400 --> 00:43:02.880
So, do you guys have my the same Wi-Fi?

s359
00:43:02.880 --> 00:43:07.119
Um, AI.gineer Wi-Fi.

s360
00:43:07.119 --> 00:43:07.680
Okay.

s361
00:43:07.680 --> 00:43:12.880
So, can someone assist people with the Wi-Fi if possible?

s362
00:43:13.680 --> 00:43:17.280
Whether you can help each and every one of us, you need to fix the system.

s363
00:43:17.280 --> 00:43:19.040
Yeah, please fix the system.

s364
00:43:19.040 --> 00:43:20.880
Whoever

s365
00:43:20.880 --> 00:43:22.640
do it.

s366
00:43:22.640 --> 00:43:25.359
I can't I can't.

s367
00:43:28.640 --> 00:43:29.119
Yeah.

s368
00:43:29.119 --> 00:43:29.680
Yeah.

s369
00:43:29.680 --> 00:43:35.520
Uh my colleagues will we'll take a look at that if

s370
00:43:37.040 --> 00:43:39.040
Right.

s371
00:43:43.280 --> 00:43:43.839
Yeah.

s372
00:43:43.839 --> 00:43:44.319
Yeah.

s373
00:43:44.319 --> 00:43:46.560
It always happens, you know.

s374
00:43:46.560 --> 00:43:48.560
Yep.

s375
00:44:00.480 --> 00:44:02.640
All right.

s376
00:44:04.400 --> 00:44:05.280
Say again, sir.

s377
00:44:05.280 --> 00:44:07.760
slides that you were showing or they

s378
00:44:07.760 --> 00:44:12.640
Yes, they will go into the AI I believe they they will go into the AI engineer.

s379
00:44:12.640 --> 00:44:18.560
So, if you go into the session, uh I'll make sure to to to go there.

s380
00:44:18.560 --> 00:44:25.359
If not, if you either join the discord or any other, you know, you can just message me as well on LinkedIn.

s381
00:44:25.359 --> 00:44:29.599
I'll gladly give you the the slides if you want.

s382
00:44:29.599 --> 00:44:30.800
All right.

s383
00:44:30.800 --> 00:44:37.680
Um, video team, can you turn me on?

s384
00:44:37.680 --> 00:44:42.079
Uh, sorry, switch me on, please.

s385
00:44:43.200 --> 00:44:44.800
Uh, oh, perfect.

s386
00:44:44.800 --> 00:44:45.920
Thank you.

s387
00:44:45.920 --> 00:44:55.119
So, what I'd like you to to show you is that we have built also apart from the notebook that we're going to go through, we also built this app book.

s388
00:44:55.119 --> 00:45:03.839
And with the app book, you can actually test every of the individual components of an agent harness uh individually, right?

s389
00:45:03.839 --> 00:45:10.560
So just for me to show you that this is possible and you will get this automatically deployed in GitHub code spaces as well.

s390
00:45:10.560 --> 00:45:18.319
So you will have um this uh already deployed and you will say well how am I actually making requests?

s391
00:45:18.319 --> 00:45:23.200
We are going to be making requests to uh Oracle.

s392
00:45:23.200 --> 00:45:25.920
Uh oh.

s393
00:45:26.079 --> 00:45:26.480
Okay.

s394
00:45:26.480 --> 00:45:28.160
Not file.

s395
00:45:28.160 --> 00:45:29.280
Demo time.

s396
00:45:29.280 --> 00:45:33.599
Um the idea is that Yeah.

s397
00:45:33.599 --> 00:45:34.000
Okay.

s398
00:45:34.000 --> 00:45:35.440
I I know what's happening.

s399
00:45:35.440 --> 00:45:38.720
So I lost connection to my to my code space because of inactivity.

s400
00:45:38.720 --> 00:45:40.960
Let me restart.

s401
00:45:40.960 --> 00:45:55.520
Uh this appbook is going to allow you to create and chat and interact with the whole agent harness and the models that we're going to use are actually deployed on a managed service that we have on Oracle called

s402
00:45:55.520 --> 00:45:57.680
OCI the generative AI service.

s403
00:45:57.680 --> 00:46:06.240
We have partnerships with Google, with Meta and with OpenAI and with XAI for the time being.

s404
00:46:06.240 --> 00:46:11.440
And we can actually provide inference to their models through our managed uh server.

s405
00:46:11.440 --> 00:46:17.839
So think of us as the enterprise open router if you'd like.

s406
00:46:17.839 --> 00:46:20.560
So let me just go so you get started.

s407
00:46:20.560 --> 00:46:21.680
Uh you can get started.

s408
00:46:21.680 --> 00:46:23.200
This is the repo, right?

s409
00:46:23.200 --> 00:46:26.160
So the agent harness workshop.

s410
00:46:26.160 --> 00:46:30.079
If you're here and thank you for starting that by the way.

s411
00:46:30.079 --> 00:46:42.319
Um if you're here you just have to click on open in GitHub code spaces and it will take you here and you can select as many course as you like.

s412
00:46:42.319 --> 00:46:51.520
uh if you so if you want to create this with eight or 16 please don't because I I don't I'm paying for this myself but uh you might also

s413
00:46:51.520 --> 00:47:00.720
create this with more resources but just create the code space and I'm going to pay for it as I said so don't worry about that and this will create

s414
00:47:00.720 --> 00:47:14.400
a new a new code space instance and once it finishes which it hasn't yet I will show you what we can do with the appbook and the notebook.

s415
00:47:14.400 --> 00:47:31.040
But the idea is to use the remainder of the time that we have 1 hour and 15 minutes to go through the notebook and you will actually have to let me show you on on GitHub actually.

s416
00:47:31.040 --> 00:47:47.280
Um you can go here and inside the notebook after you deploy the code space you will get uh a student notebook here and this is one part of the of the workshop right and here we're going to implement

s417
00:47:47.280 --> 00:47:51.119
the whole agent harness substrate from scratch.

s418
00:47:51.119 --> 00:47:59.680
So, we're going to start with only the model and then we're going to keep adding layers to the agent harness as we saw the seven layers, right?

s419
00:47:59.680 --> 00:48:02.880
And we're going to be here to to assist you.

s420
00:48:02.880 --> 00:48:04.720
You will have to do some to-dos.

s421
00:48:04.720 --> 00:48:06.240
So, let me show you.

s422
00:48:06.240 --> 00:48:10.319
There are a couple of things to do for you.

s423
00:48:10.800 --> 00:48:17.520
So, for instance, the first thing that you need to do, you need to create a question, right?

s424
00:48:17.520 --> 00:48:24.400
The simplest thing of everything, you just have to communicate with a model with no agent harness implemented, right?

s425
00:48:24.400 --> 00:48:29.040
So the first thing you'll need is to ask any question that you like.

s426
00:48:29.040 --> 00:48:34.400
This will go through the OpenAI completions API and it will return you a response.

s427
00:48:34.400 --> 00:48:35.839
This is the simplest of all.

s428
00:48:35.839 --> 00:48:44.160
And then we will start adding search, retrieval, encoding and all the other components that we that we have seen.

s429
00:48:44.160 --> 00:48:47.839
There are a total of 19 things that you need to do.

s430
00:48:47.839 --> 00:48:57.440
If you finish first, raise your hand and I will give you a hug because I don't have anything else.

s431
00:48:57.599 --> 00:49:03.440
And yeah, so anyone already deployed the code space?

s432
00:49:04.000 --> 00:49:04.960
Okay, one person.

s433
00:49:04.960 --> 00:49:06.160
Okay, good job.

s434
00:49:06.160 --> 00:49:12.240
So any yeah, if you have any questions or any problems, uh, let me know.

s435
00:49:12.240 --> 00:49:14.640
But this is what it looks like when you have it deployed.

s436
00:49:14.640 --> 00:49:15.200
Okay.

s437
00:49:15.200 --> 00:49:19.119
So, let me go through this quickly.

s438
00:49:19.680 --> 00:49:24.800
So, you will get an app, right?

s439
00:49:24.800 --> 00:49:29.280
Um, and the app will already have everything that you need.

s440
00:49:29.280 --> 00:49:39.760
If you want to deploy this app yourself, you might change this total recall port here.

s441
00:49:39.760 --> 00:49:41.440
Let me show you how I did it again.

s442
00:49:41.440 --> 00:49:43.440
I go into ports.

s443
00:49:43.440 --> 00:49:50.240
I clicked on the visibility of the port and I changed this to public.

s444
00:49:50.240 --> 00:49:54.400
And then this is now using a public gateway.

s445
00:49:54.400 --> 00:50:01.680
So that if I open the browser, I can actually get access to my individual u total recall instance.

s446
00:50:01.680 --> 00:50:10.800
So for instance, if I ask a question like show the total revenue by product category and of course this is mission control.

s447
00:50:10.800 --> 00:50:17.040
So this is this has all of the components that we've uh spoken about implemented already.

s448
00:50:17.040 --> 00:50:22.319
You will get also a context window visualization of the things that are going on on the background.

s449
00:50:22.319 --> 00:50:30.319
For instance, these are the tools that were selected by the agent harness to be loaded into the context to answer this question.

s450
00:50:30.319 --> 00:50:32.559
This is the schema that's happening.

s451
00:50:32.559 --> 00:50:37.200
And then we can also take a look at the individual agent traces that are going on.

s452
00:50:37.200 --> 00:50:51.119
For instance, which skills, which skills are being loaded, what sources of data are we taking and what are the tool calls being used like for instance running SQL commands etc to answer your question.

s453
00:50:51.119 --> 00:50:53.599
So the question is still being built.

s454
00:50:53.599 --> 00:51:09.920
It's taking 16 steps and you know it's gonna for instance here it detected an error right but because our agent harness is um fault tolerant it will keep trying because it's part it has an agent loop implemented

s455
00:51:09.920 --> 00:51:23.040
etc right so all these things will actually yield you this result uh from from the data in the database right and you can actually go into the context window see how many tokens we're using

s456
00:51:23.040 --> 00:51:35.040
And if you're particularly interested in some of these parts for instance the Oracle Asian memory package for instance you can interact also with only the uh the context card how to

s457
00:51:35.040 --> 00:51:43.680
how the context card is being created etc etc. So you will all get this deployed in your codebase.

s458
00:51:44.079 --> 00:51:46.079
Yeah.

s459
00:51:58.800 --> 00:52:13.440
So his question for those of you who didn't uh listen where what happens if you have thousands of tools in an organization right well we introduced this concept called the toolbox pattern

s460
00:52:13.440 --> 00:52:23.359
in this course with Andrew Yang and the thing is that you can optimize so that the retrieval of these tools is negligible.

s461
00:52:23.359 --> 00:52:36.720
So you will use uh hierarchical navigable small world indexes that use a graph u a graph structure and then each node in the graph is a vector index or a vector store

s462
00:52:36.720 --> 00:52:46.720
and then you can actually like hnssw indexes they can be create created for these types of problems only in the database not not in files.

s463
00:52:46.720 --> 00:52:49.280
Um, so great question.

s464
00:52:49.280 --> 00:52:58.480
It doesn't have to worry you until you reach millions and millions of users and tool calls like different specific tool calls.

s465
00:52:58.480 --> 00:53:01.040
You might not get 5 million.

s466
00:53:01.040 --> 00:53:14.160
It's more like reading a file, writing a file, grapping, all these kinds of tool calls that that that we do every day, they typically don't ex like exceed 100 or a thousand.

s467
00:53:14.160 --> 00:53:21.920
Um but by being on a vector store you abstract the complexity uh the complexity and the amount of it.

s468
00:53:21.920 --> 00:53:34.559
You can just make a a query 2,000 just as simply as you would 10,000 because of the storage component that we we choose it which is an HNSW index.

s469
00:53:45.200 --> 00:53:48.880
Yeah, some of them they have access to confidential data for instance.

s470
00:53:48.880 --> 00:53:51.599
Some of them don't.

s471
00:54:01.839 --> 00:54:03.839
Yes.

s472
00:54:08.160 --> 00:54:08.880
Great question.

s473
00:54:08.880 --> 00:54:15.520
So his question was what happens if the tool descriptions that two different companies have are very similar, right?

s474
00:54:15.520 --> 00:54:27.040
And one of the things we we can do on the toolbox pattern is actually generate with LLM enhanced toolbox descriptions for these for specific tools to increase the separability

s475
00:54:27.040 --> 00:54:28.480
of the of the tools.

s476
00:54:28.480 --> 00:54:39.200
So if you think that the current descriptions of a tool or of a skill as well are not enough, you can actually enhance them with LLM retrieval like you would

s477
00:54:39.200 --> 00:54:45.760
uh instead of running for instance named entity recognition which is very uh caveman style.

s478
00:54:45.760 --> 00:54:59.599
You can also do something more sophisticated which is enhancing the kind of like dock string uh enhanced representations of a tool so that you increase the separability when you're doing vector search.

s479
00:54:59.599 --> 00:55:02.640
Does that answer the question?

s480
00:55:08.640 --> 00:55:10.640
the agent loop is that?

s481
00:55:10.640 --> 00:55:12.640
Yeah.

s482
00:55:15.680 --> 00:55:16.400
Oh yes.

s483
00:55:16.400 --> 00:55:18.400
Yes.

s484
00:55:20.240 --> 00:55:23.200
We have seen some.

s485
00:55:45.760 --> 00:55:46.160
Yes.

s486
00:55:46.160 --> 00:55:51.839
So there there is a there is a limit of course because we don't have infinite money.

s487
00:55:51.839 --> 00:55:57.680
So we can't just keep trying and trying over and over if the generations are just hallucinations right.

s488
00:55:57.680 --> 00:56:10.799
Uh there is a cutff point that I set depending on the frontier LLM that I'm using for instance for Grog 4.1 fast reasoning which is the one that we're using here.

s489
00:56:10.799 --> 00:56:20.559
Um I found that a value of 8 to 12 like maximum number of tool calls before giving up is correct.

s490
00:56:20.559 --> 00:56:33.200
Uh depends also on the on the accuracy and the and the correctness of the model like for instance in this case it was just able to to to to show it in two before it was able to find it in 16.

s491
00:56:33.200 --> 00:56:39.839
So t sometimes it will have a faster retrieval, sometimes you will need to be a little bit more patient.

s492
00:56:39.839 --> 00:56:49.680
But what I like to define is a variable like a hysterosis variable that holds the amount of patience that the harness will get with the model.

s493
00:56:49.680 --> 00:56:54.720
Then you can do some other things like for instance if the model is garbage.

s494
00:56:54.720 --> 00:57:13.280
You can just use or use like a router for more different like for difficult types of problems you will route this problem to a frontier LLM and then for the easier types of problems you can just attach an open weights SLM for instance which will be more more interesting.

s495
00:57:13.280 --> 00:57:17.760
Do you recommend models for

s496
00:57:17.760 --> 00:57:18.160
Yes.

s497
00:57:18.160 --> 00:57:18.559
Yes.

s498
00:57:18.559 --> 00:57:28.960
I think that's like my personal opinion is that the future is a mixture of small experts for for each type of problem.

s499
00:57:28.960 --> 00:57:43.119
some companies that they have developed like a 100 million parameter models that work exceptionally well for one type of problem and if you just have an aggregator and an orchestrator that routes the correct

s500
00:57:43.119 --> 00:57:52.400
model to that like the correct query to that model then you will have a very token efficient type of uh agent harness.

s501
00:57:52.400 --> 00:57:56.160
So you you can actually do model routing inside the agent harness.

s502
00:57:56.160 --> 00:58:08.160
Some companies are actually essentially only doing that and they will charge you like let's do I'm going to charge you 10% of the tokens that I'm going to save you from the original

s503
00:58:08.160 --> 00:58:20.480
uh amount of money that you were going to spend right uh so let's uh let me show you the the student notebook right so once you are inside the student notebook

s504
00:58:20.480 --> 00:58:31.760
for those of you who are not familiar with Visual Studio Code you might need to select a kernel here so that you run the the notebook.

s505
00:58:31.760 --> 00:58:37.920
So you might select Python 3.12 here and then you can just start reading.

s506
00:58:37.920 --> 00:58:49.680
If you stumble into a to-do that you need to do, you have a docs folder with all the explanations, the individual explanations that you need to solve this specific problem.

s507
00:58:49.680 --> 00:58:56.640
For instance, the first to-do which is just talking to the reasoning core to the model layer without doing anything else.

s508
00:58:56.640 --> 00:59:06.880
It will just explain what you need to implement on that sale so that it works and you can proceed to the next one and also have the solution.

s509
00:59:06.880 --> 00:59:17.680
But if you if you're not lazy, you will try and hope that you that you try and we will be here answering questions around the room.

s510
00:59:17.680 --> 00:59:27.359
I'm gonna turn off my microphone, just come down with my colleagues and then let's chat about it uh for the remainder of the of the session.

s511
00:59:27.359 --> 00:59:36.160
And if you have any questions or you like to talk um more to us, please come by and and swing by the booth, the Oracle booth.

s512
00:59:36.160 --> 00:59:39.760
We'll be there every day uh all the time.

s513
00:59:39.760 --> 00:59:45.119
And you know, it makes it makes us feel good like we are wanted and we have friends.

s514
00:59:45.119 --> 00:59:49.680
So if you want to come up to us, just chat with us a little bit.

s515
00:59:49.680 --> 00:59:52.559
It it will be nice.

s516
00:59:55.359 --> 00:59:55.839
Internet.

s517
00:59:55.839 --> 00:59:58.319
How is the internet?

s518
01:00:10.960 --> 01:00:13.839
I feel like Cesar.

s519
01:00:13.839 --> 01:00:15.839
Right.

s520
01:00:21.040 --> 01:00:23.119
So, I'm gonna leave this here.

s521
01:00:23.119 --> 01:00:25.760
I'm going to keep this here.

s522
01:00:25.760 --> 01:00:28.720
And I'm going to come down.

s523
01:00:45.117 --> 01:00:47.117
[music]