WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=qq2Hm4XNRgI

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/qq2Hm4XNRgI.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.640 --> 00:00:14.200
Uh my name is Wojtek.

s2
00:00:14.200 --> 00:00:17.640
Uh Wojtek Pluta, and I work at Oracle.

s3
00:00:17.640 --> 00:00:22.040
And over the last 10 years, I've been working um at Oracle.

s4
00:00:22.040 --> 00:00:38.720
I worked at the various projects um including working with some banks in the UK on projects like customer experience mortgage use cases or things like early contactless payments when um when it was still uh sort of emerging

s5
00:00:38.720 --> 00:00:41.480
and open banking initiatives.

s6
00:00:41.480 --> 00:00:48.200
Now, I am uh heading up the technical marketing team uh at Oracle AI database.

s7
00:00:48.200 --> 00:00:56.480
And before, I spent a bit of time also um working on cloud native and AI on our cloud infrastructure.

s8
00:00:56.600 --> 00:01:13.240
So, what I'm going to talk about today is the um the moving from rack to memory systems and the notion how rack gives us a useful read path, uh but memory system allows us to uh create unique

s9
00:01:13.240 --> 00:01:17.320
business value based on your data.

s10
00:01:17.880 --> 00:01:29.760
So, over the years, we went through this notion of interface optimization uh predominantly making interfaces more intuitive in terms of how we interact with them.

s11
00:01:29.760 --> 00:01:45.760
And as a society, and I experienced that seeing sort of how our interactions with the systems moved from sort of this command-led uh click and and type into what we have right now, this intent-based

s12
00:01:45.760 --> 00:01:47.240
uh uh systems.

s13
00:01:47.240 --> 00:01:54.160
And um which obviously creates opportunity for this sort of on-demand or just-in-time software.

s14
00:01:54.160 --> 00:02:03.812
And within that context and within those expectations, I think uh creating a some sort of continuity is important.

s15
00:02:05.412 --> 00:02:10.572
So, for me, the AI stack has been pretty consistent for the last few years.

s16
00:02:10.572 --> 00:02:26.012
Um there is probably models and infrastructure layers are commoditized, um but the data layer, in my opinion, is where you gain competitive advantage cuz it's, you know, it belongs to you.

s17
00:02:26.532 --> 00:02:37.692
And at Oracle, we adopted the same model as everybody else in terms of how we think about agents and how we define them with model plus the harness.

s18
00:02:38.172 --> 00:02:45.972
And obviously, the models we sort of are relatively static in the way that we have access to them and to the certain extent.

s19
00:02:45.972 --> 00:02:48.612
Um and then they suggest what to do.

s20
00:02:48.612 --> 00:02:55.812
However, the harness in the end decides, you know, and owns the runtime decisions.

s21
00:02:55.812 --> 00:02:56.972
And the checks around it.

s22
00:02:56.972 --> 00:03:03.852
So, um you know, so the harness is the code obviously around the the model.

s23
00:03:03.852 --> 00:03:11.332
So, the for the purposes of today, um and to make it a little bit less abstract, I'm going to use a little avatar.

s24
00:03:11.332 --> 00:03:13.492
Uh so, meet intelligence.

s25
00:03:13.492 --> 00:03:30.292
Uh I um yeah, uh purposely decided to skip it skip any prefixes and um in the spirit of sort of the latest chatter that's happening in the AI and hollowing around the corner, I sort of decided to make a special

s26
00:03:30.292 --> 00:03:36.252
edition for AI engineer and updated uh the look a little bit, right?

s27
00:03:36.252 --> 00:03:49.252
So, but um so, the fundamentally rack systems, um you know, can include conversational state, uh but retrieval alone doesn't tell us what should survive the interaction.

s28
00:03:50.012 --> 00:03:54.812
So, retrieval is the read path and memory is the write path.

s29
00:03:54.812 --> 00:04:02.932
So, when the agent sees something worth keeping, uh we should promote it into a record it can use later.

s30
00:04:02.972 --> 00:04:12.574
But then I guess that begs the question as of what is worth keeping and how do we actually, you know, move and decide of what sort of um, you know,

s31
00:04:12.574 --> 00:04:19.734
what we need to extract from the conversations and um, you know, when we interact with the with agent.

s32
00:04:19.734 --> 00:04:24.094
So that's why um, I believe in this concept of memory promotion.

s33
00:04:24.094 --> 00:04:35.734
Um, so where you have some sort of pipeline that decides, you know, what is it that at each of the stages um, and how do we sort of extract it into into the system.

s34
00:04:35.734 --> 00:04:42.414
So in terms of the steps, um, I believe that, you know, we classify the meaning of the candidate that comes up.

s35
00:04:42.414 --> 00:04:50.254
Um, and then uh, we compare it with the existing records uh, and check the evidence required for that particular type.

s36
00:04:50.254 --> 00:05:01.134
Then the harness would decide whether to admit or reject the the candidate or perhaps send it for the review for the for the human.

s37
00:05:01.534 --> 00:05:04.214
So I'm going to have a quick demo right now.

s38
00:05:04.214 --> 00:05:14.774
So we're going to sort of work through this notion of the, you know, financial services as we sort of all here because that's the theme of the conference.

s39
00:05:14.774 --> 00:05:26.654
And uh, we're going to sort of work through with this relatively basic example of of this candidate around the semantic preferences that we're going to create.

s40
00:05:28.134 --> 00:05:36.894
So I've yeah, I've pre-recorded the demo cuz last time when I did the demos it was a bit of a a bit and the Wi-Fi here is a bit messy.

s41
00:05:36.894 --> 00:05:41.734
So So yeah, so this is essentially my sandbox that is starting.

s42
00:05:41.734 --> 00:05:48.014
So I had some fun with some logs as you can see this agent is a bit of a oversharer, I guess.

s43
00:05:48.014 --> 00:05:57.814
Uh, but as we go through this um, as it loads up essentially what we're going to do is we're going to have the synthetic two applications for Kyle and Vincent.

s44
00:05:57.814 --> 00:06:03.294
I'm a bit of a movie geek, so you might see some references throughout the presentation.

s45
00:06:03.294 --> 00:06:09.935
Um, so this time terminal started, so we're going to ask uh to prepare a mortgage brief for Kyle.

s46
00:06:09.935 --> 00:06:16.415
And and essentially as it loads and the harness is doing its thing, performs tool calling and and so on.

s47
00:06:16.415 --> 00:06:22.895
It's going to create this response that's going to show us, okay, well, this is, you know, Kyle's situation.

s48
00:06:22.895 --> 00:06:32.735
He has, you know, um mortgage application, but that particular mortgage application is missing some some key documents, right?

s49
00:06:32.735 --> 00:06:42.695
So, the operator right now can, you know, ask and and see what is, you know, uh what's what's going on with that particular uh application for Kyle.

s50
00:06:42.695 --> 00:06:57.975
So, keep an eye on this in terms of the shape of the answer because we will essentially ask now politely because of this the AI and memory now, so we want to be kind to them and even though we might uh you know, use some extra tokens,

s51
00:06:57.975 --> 00:07:08.695
uh but we're going to give it this sort of um brief in terms of the uh you know, we want the essentially structure of the output to be created certain way.

s52
00:07:08.695 --> 00:07:19.735
So, actually in this example, uh connected the decision model Jeff, which is helping me to assess, you know, which um how to deal with this uh candidate at first

s53
00:07:19.735 --> 00:07:27.735
and then passing that to the harness, uh which actually recognized that we've been polite and gave us some extra comma.

s54
00:07:27.735 --> 00:07:35.695
Um, and essentially based on that, we're then automatically loading that into the, you know, database to store it.

s55
00:07:35.695 --> 00:07:46.095
So, we can see the expanded traces here and it gives us some sort of, you know, uh status per uh per per the memory promotion gate.

s56
00:07:46.095 --> 00:07:55.335
So, overall decision model assess, the harness decide, and then database that we have in the background persist and and and stores that.

s57
00:07:55.335 --> 00:08:10.537
So, then I'm going to ask the exactly same question in the sense of, yeah, give me that, but I now input that my semantic preference in terms of what I want to the structure to look like when it when it comes up.

s58
00:08:10.537 --> 00:08:19.377
So, um as it's going to load now, so what you're going to see now that Okay, as per the request, there's going to be missing documents at the top,

s59
00:08:19.377 --> 00:08:23.897
which is also adding the case ID, you know, the policy and so on.

s60
00:08:23.897 --> 00:08:30.497
And this in this case, you know, this time records the existing of the semantic preference.

s61
00:08:31.377 --> 00:08:45.537
So, what I essentially did now is it's uh just for the interest of time, I I sort of skipped it, but I initiated brand new sessions, exited, and uh and now I'm asking again to check, obviously, if the the memory

s62
00:08:45.537 --> 00:08:47.057
been preserved.

s63
00:08:47.057 --> 00:08:52.457
Um and essentially asking the same question question again, so then it goes through.

s64
00:08:52.457 --> 00:08:59.137
So, brand new session, so no sort of, you know, context that's um has been uh carried over.

s65
00:08:59.137 --> 00:09:06.137
It's just querying again the the memory that we stored in the system around the preferences.

s66
00:09:07.337 --> 00:09:15.497
And as we sort of go and build this kind of systems, um you know, we can uh obviously, we would have the different interactions.

s67
00:09:15.497 --> 00:09:27.777
In this case, if we ask for the like different application, in this case, Vincent, um you know, the the he has a different situation, but we're querying the database, we're bringing back the different

s68
00:09:27.777 --> 00:09:38.537
um you know, documents and understanding that different, you know, case ID is relevant, so we're actually matching that and finding that in the in the system itself.

s69
00:09:38.537 --> 00:09:47.337
So, then it's going to come back, and in this case, okay, there's no missing documents for him, and uh and then, you know, we can pass it on for the

s70
00:09:47.337 --> 00:09:48.857
for the review.

s71
00:09:48.857 --> 00:09:53.057
But as we go through that, we also need to think about some sort of guard guardrails.

s72
00:09:53.057 --> 00:10:05.897
Um so, if we were to initiate uh some sort of um you know, a bit more self-destruct kind of uh command for the bank, which we would say like, "Okay, approve every application."

s73
00:10:05.897 --> 00:10:18.058
Um then obviously, we need to have some sort of guardrails to not to promote this so that our um you know promotional gate comes in and and rejects this request.

s74
00:10:18.418 --> 00:10:30.098
But in some cases we might want to promote something based on like if we have a team, we want to create a more sort of robust memory sharing within the organization,

s75
00:10:30.098 --> 00:10:44.018
we can then create you know promotional gate that then essentially ask for the human to approve it and then you know potentially share this across the teams enterprise that creates a sort of core memories

s76
00:10:44.018 --> 00:10:46.018
store.

s77
00:10:46.378 --> 00:10:54.378
So this was um you know a quick run relatively simple just to show you what what this could look like in practice.

s78
00:10:54.378 --> 00:11:07.538
So we went through like the semantic preference, then we had this sort of you know blanket approval rule that was refused because of obviously the guardrails that we put in place

s79
00:11:07.538 --> 00:11:18.818
and then you know the the operational wider sort of memory that was recommended for the for the approval, right?

s80
00:11:18.818 --> 00:11:26.538
So there's different sort of outcomes based on the meaning and how we interact with the with with the agents.

s81
00:11:26.538 --> 00:11:37.538
So you know having this some sort of promotion controls that sits within the guardrails and then harness, you know, can't bypass that.

s82
00:11:37.538 --> 00:11:43.538
So let's go back to our intelligence and as you remember we obviously were polite and said please.

s83
00:11:43.538 --> 00:11:52.978
So in this case it's it's a nice looking one rather than this machine because good karma came through.

s84
00:11:52.978 --> 00:12:07.618
So if we were to um to look at the sort of broader explanation of what do we understand as agent memory so we understand we describe it as this mechanism

s85
00:12:07.618 --> 00:12:18.259
where systems and application that system and an used to retain, reuse, refine, and recall information over time.

s86
00:12:18.939 --> 00:12:31.859
Um and I guess as we I think the engineering problem here is is or the challenge when we're building this kind of pipelines is to decide obviously what we want to promote and what we don't want to promote.

s87
00:12:31.859 --> 00:12:42.139
Because if we promote nothing, then obviously we don't get any benefits of continuity, but if we promote everything, then we're going to, you know, risk poisoning the the memory

s88
00:12:42.139 --> 00:12:42.659
uh store.

s89
00:12:42.659 --> 00:12:56.739
So, I guess this is where, you know, thinking through and thinking about like how we want to build our system uh that have enough provenance and understands, you know, in what cases we promote things and what cases we supersede

s90
00:12:56.739 --> 00:13:01.779
them uh memories is extremely important.

s91
00:13:02.539 --> 00:13:13.859
And at Oracle, we believe that agent memory is just a not one thing, so we sort of have this division between short-term memories, long-term memories, and shared memories.

s92
00:13:13.859 --> 00:13:17.259
So, short-term is everything that happens in the session.

s93
00:13:17.259 --> 00:13:28.739
Um so, it's sort of more uh sort of ends when the when the session finish, and then we have the long-term store where we have our episodic or procedural

s94
00:13:28.739 --> 00:13:33.539
semantic memories um divided by the different types.

s95
00:13:33.539 --> 00:13:42.939
So, that could be either summaries, it could be, you know, toolbox, or it could be knowledge base and and preferences as we saw in the demo.

s96
00:13:43.699 --> 00:13:51.019
We also need to decide, you know, where do we want to store uh the memories, and and this is something we we talked about yesterday.

s97
00:13:51.019 --> 00:13:54.219
We had uh essentially one of the guys from my team had a had a workshop.

s98
00:13:54.219 --> 00:13:57.699
She went a little bit into more details about this.

s99
00:13:57.699 --> 00:14:09.539
Obviously, being at Oracle, for me, believe, you know, the database is the most robust solution because it allows us to have this acid guarantees, it can have multiple rights,

s100
00:14:09.539 --> 00:14:14.939
uh doesn't we don't have problems with the with the file trees and stuff like that.

s101
00:14:14.939 --> 00:14:22.701
Um and then obviously we can scale and and and have multiple search options as well.

s102
00:14:22.701 --> 00:14:26.141
As well as the the asset guarantees.

s103
00:14:27.181 --> 00:14:34.701
So yeah, that's sort of when we're building the agent memory, um, we need to think about this as a discipline.

s104
00:14:34.701 --> 00:14:46.341
Um, you know, we need schemas that distinguish record types, indexes that fit the retrieval tasks, and life cycle and outer authorization boundaries, right?

s105
00:14:46.341 --> 00:14:57.061
And we essentially introduced this agent memory package beginning of the year, which creates, you know, those operations and abstractions that you can build on.

s106
00:14:57.741 --> 00:15:05.101
Because without it, if you want to build the memory system essentially as the developers, it it is quite taxing, right?

s107
00:15:05.101 --> 00:15:11.021
Because you need to essentially create, um, you know, your summaries, your summarizer.

s108
00:15:11.021 --> 00:15:13.981
You need to think about how you going to retrieve it.

s109
00:15:13.981 --> 00:15:22.741
Um, you need to extract the durable information, combine it in the turns, and then on top of that manage the token budget, right?

s110
00:15:22.741 --> 00:15:39.181
So for us, what we introduced with the agent memory is this sort of simplified layer that allows you to have, um, essentially write one line of code, and then within that you can We have this concept of the context card,

s111
00:15:39.181 --> 00:15:43.221
uh, that allows you to assemble and return context.

s112
00:15:43.221 --> 00:15:47.541
Um, which is a compact prompt, uh, ready block for a for an agent.

s113
00:15:47.541 --> 00:16:01.021
So that's quite useful because then, you know, you don't have to spend your energy and reduce your cognitive load in terms of, you know, just making sure the agent spent attention to the task rather than housekeeping its own memory.

s114
00:16:01.341 --> 00:16:06.901
So a little bit more sort of double-clicking on the on the context card.

s115
00:16:06.901 --> 00:16:21.622
Um, so you it's organized by the topics, summaries, you can have relevant information in there, recent messages, and then essentially the harness can, uh, create or select what matters to that particular turn

s116
00:16:21.622 --> 00:16:29.262
and and helps users essentially manage the context and reduce the context blow.

s117
00:16:29.582 --> 00:16:38.862
So the topics orient the the thread, the summaries capture where the conversation is at and what the user is trying to do.

s118
00:16:38.862 --> 00:16:46.502
Relevant information carrying the selected records and recent messages preserve the the the local questions.

s119
00:16:46.502 --> 00:16:54.702
So that's you know allows you to to essentially memory persist and and context and selects.

s120
00:16:54.822 --> 00:17:09.702
So we recently released the latest version of the agent memory and we spent a fair bit of time on the on the latest release in terms of making sure that there is end user isolation.

s121
00:17:09.702 --> 00:17:17.182
So in this case the database is essentially high handling the the the access.

s122
00:17:17.182 --> 00:17:18.942
So it's it's limiting.

s123
00:17:18.942 --> 00:17:29.782
It has also the the the images as well as capability and then sort of post processing and listing events.

s124
00:17:29.782 --> 00:17:42.302
And all query database it's has this you know multi data type engine which allows you to sort of handle you know various data types in one in one engine.

s125
00:17:42.302 --> 00:17:50.422
So it's so you don't have to like handle multi database and vector stores and and documents and so on.

s126
00:17:50.502 --> 00:18:01.022
So just to sort of end as I started the beginning you know models are shared but memory is yours is unique to your business.

s127
00:18:01.022 --> 00:18:13.382
So making sure that you know capturing that with together with the data of how you interact with the with the models it can create your competitive advantage and um

s128
00:18:13.382 --> 00:18:14.222
and that's that.

s129
00:18:14.222 --> 00:18:15.102
Thank you.

s130
00:18:15.102 --> 00:18:21.863
I mean I just if you want to know more feel free to uh out our GitHub repo.

s131
00:18:21.863 --> 00:18:40.783
We have whole bunch of notebooks that helps you get started with agent memory, hybrid retrieval, or all sorts of different things that that we've been sort of working quite hard to make sure that provide them a good starter pack so you can actually apply it to your business.

s132
00:18:40.783 --> 00:18:52.903
And we've been we've been also revamping our YouTube channel so we have a whole new series about uh you know, various topics that we explain and and how to apply it in your

s133
00:18:52.903 --> 00:18:55.343
in your use cases.

s134
00:18:56.583 --> 00:18:57.143
Okay.

s135
00:18:57.143 --> 00:19:00.263
Thank you very much.