WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=KGFyOtl5ktI

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/KGFyOtl5ktI.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.560 --> 00:00:14.560
Testing.

s3
00:00:14.960 --> 00:00:16.960
Awesome.

s4
00:00:17.360 --> 00:00:18.880
Good morning everybody.

s5
00:00:18.880 --> 00:00:22.000
Uh I know it's the last day of the conference, so super excited to have you here.

s6
00:00:22.000 --> 00:00:27.320
Hopefully hopefully I can keep you engaged for the next uh 15 to 18 minutes or so.

s7
00:00:27.320 --> 00:00:31.000
My name's Jake and I lead our labs team here at LangChain.

s8
00:00:31.000 --> 00:00:42.840
And today I want to talk to you about a topic that I have uh been speaking a lot about with with our team and and with a lot of teams that I speak with who are building agents, and that's how to give your agents memory.

s9
00:00:43.000 --> 00:00:56.240
Whether you're building a coding agent, a support agent, a uh a research agent, most teams arrive at the general same instinct that it should get better from one run to the next.

s10
00:00:56.640 --> 00:00:59.960
So, let me give you an example to hold onto for the rest of this talk today.

s11
00:00:59.960 --> 00:01:11.800
Recently my my the the labs team at LangChain have been working with a company in the financial services sector who's building a uh spending, budgeting, and non-investment advice agent.

s12
00:01:11.800 --> 00:01:18.840
Given the highly regulated nature of the financial services sector, obviously you would imagine that it would need to follow some strict tone guidelines.

s13
00:01:18.840 --> 00:01:32.400
And so, the way that it speaks and and the and its vocabulary and the way that it interacts with different users is obviously the the tell on whether or not it's following and abiding by the correct tone.

s14
00:01:32.560 --> 00:01:42.200
It will derive the way that it generates this tone from the context and from the memories and from the references that it has to its rules and policies.

s15
00:01:42.200 --> 00:01:48.760
And you can imagine in some contexts, and actually in some contexts that we experience, its tone would slip.

s16
00:01:48.760 --> 00:02:04.520
And so, it would move from being advisory and helpful to actually pointed and directive, saying things like users in your similar situation would generally set a cap and it would actually say, "Oh, well, actually it looks like you need to cancel some subscriptions and move some money into savings."

s17
00:02:04.520 --> 00:02:09.479
The company that we're working with thought that this is a violation of tone and there were similar instances of this, too.

s18
00:02:09.479 --> 00:02:12.840
And so, to correct that, the review process is manual.

s19
00:02:12.840 --> 00:02:21.800
You have to have someone understand the traces and understand where its tone slipped, make those changes to the context that it needs to reference when it's giving its tone,

s20
00:02:21.800 --> 00:02:24.160
and then make sure that there are no regressions, as well.

s21
00:02:24.160 --> 00:02:27.600
Obviously, that that doesn't scale, or rather it's challenging to scale.

s22
00:02:27.600 --> 00:02:38.000
And so, the system should be or or the system around the agent should be a place where it can continuously learn and benefit from its interactions and its experiences.

s23
00:02:38.000 --> 00:02:42.840
And that's the version of memory that I want to talk about today.

s24
00:02:43.720 --> 00:02:46.600
To get there, I want to start with a tension that I think a lot of teams have run into.

s25
00:02:46.600 --> 00:02:51.160
I think all of us here today, the reason why we're at this conference is we're curious, we're building agents.

s26
00:02:51.160 --> 00:03:03.239
I think we've gotten very good at observing them, but observing them and then turning those observations and those agent experiences into lessons for future runs are obviously very different things.

s27
00:03:03.760 --> 00:03:10.040
And so, that's where memory And so, that's where memory comes into play.

s28
00:03:10.040 --> 00:03:13.840
Agents are producing more signal and more traces than ever before.

s29
00:03:13.840 --> 00:03:18.360
And this is going to be something that's obviously that will obviously increase as more agents are built.

s30
00:03:18.360 --> 00:03:23.720
That's fantastic for us because every run produces a trace and that trace is stored somewhere.

s31
00:03:23.720 --> 00:03:24.600
That's incredibly useful.

s32
00:03:24.600 --> 00:03:34.000
It means that we can understand the tools that were called, the reasoning that the agent made as it was on its way to making its decision, the artifacts that it referenced.

s33
00:03:34.239 --> 00:03:40.160
Observability is obviously a critical foundation for building very reliable very reliable agents.

s34
00:03:40.160 --> 00:03:48.320
But I think the challenge really is after the trace is stored or after the log or after the transcript of what the agent did is stored.

s35
00:03:48.320 --> 00:04:01.640
If the future context that that agent references never changes, then a misguided skill or instruction will mean that the agent The mistake that the agent makes today will still persist in future runs.

s36
00:04:01.640 --> 00:04:09.800
The goal or the system that we want to work towards is where a trace stops just being a log and becomes a source of signal.

s37
00:04:09.800 --> 00:04:11.080
Memory.

s38
00:04:11.080 --> 00:04:13.040
Agents that are observable tell you what happened.

s39
00:04:13.040 --> 00:04:20.400
Agents that are adaptive are able to change are able to change and guide themselves on the following runs.

s40
00:04:20.400 --> 00:04:21.160
So, I've mentioned memory.

s41
00:04:21.160 --> 00:04:25.400
I think it will really help to sort of define what I mean when I say memory.

s42
00:04:25.400 --> 00:04:31.600
It's the durable context that an agent references that will guide future runs.

s43
00:04:31.600 --> 00:04:36.720
A trace, a transcript, or a log is evidence of what happened.

s44
00:04:36.720 --> 00:04:45.440
It only becomes memory when that lesson is converted into durable context the agent can reference on future runs.

s45
00:04:45.760 --> 00:04:48.360
Now, you'll see six things on this slide behind me.

s46
00:04:48.360 --> 00:04:54.440
Facts, preferences, patterns, previous examples, skills, and instructions.

s47
00:04:54.440 --> 00:05:06.200
I think that it's helpful to group them into three distinct buckets, a taxonomy that we borrow from cognitive science, and actually that applies a lot when we're talking about agents that are language-based.

s48
00:05:07.280 --> 00:05:08.800
First, there's what the agent knows.

s49
00:05:08.800 --> 00:05:11.240
This is often referred to as semantic memory.

s50
00:05:11.240 --> 00:05:17.080
This includes things like the facts and the preferences that guide the way that it responds.

s51
00:05:19.000 --> 00:05:20.240
Oops.

s52
00:05:20.240 --> 00:05:24.280
Then there's the Then there's what the agent has experienced.

s53
00:05:24.280 --> 00:05:25.400
This is the episodic memory.

s54
00:05:25.400 --> 00:05:30.680
This is the learned patterns that it's seen, past interactions, and examples.

s55
00:05:31.760 --> 00:05:33.720
And finally, there's how the agent should behave.

s56
00:05:33.720 --> 00:05:36.320
This is also known as procedural memory.

s57
00:05:36.320 --> 00:05:45.280
And that's the instructions, the skills that you're all probably very familiar with, and the rules that guide how it should how it should behave.

s58
00:05:45.280 --> 00:05:49.080
Procedural memory is where actually most of the visible gains come from.

s59
00:05:49.080 --> 00:05:58.280
So, in that financial assistance that I the example that I gave at the beginning, tone changes that we wanted to to promote to the agent and and make it follow came in the form

s60
00:05:58.280 --> 00:06:00.280
of updating its procedural memory.

s61
00:06:00.280 --> 00:06:04.960
We weren't giving it a new fact or we weren't telling it not to use a bad word.

s62
00:06:04.960 --> 00:06:09.000
We were giving it a rule to follow as it engages with its users.

s63
00:06:09.000 --> 00:06:12.680
That's the procedural memory that guides our agent.

s64
00:06:15.640 --> 00:06:27.680
Another distinction that that I want to make and I think this is very important when thinking about the general agent agent memory paradigm is the separation between working memory or short-term memory and long-term memory.

s65
00:06:27.680 --> 00:06:30.960
Working memory is what the agent is referencing in its current run.

s66
00:06:30.960 --> 00:06:33.480
So, these might be things like intermediate scratch pads.

s67
00:06:33.480 --> 00:06:34.560
It might be tool results.

s68
00:06:34.560 --> 00:06:38.840
It might be the files retrieved as part of its operation.

s69
00:06:38.840 --> 00:06:43.600
Everything that it needs to do to perform its current job.

s70
00:06:43.640 --> 00:06:54.960
Long-term memory is what exists typically in some data structure or some store somewhere else that has instructions and skills and things that it needs to reference as part of

s71
00:06:54.960 --> 00:06:58.680
future turns and and continuous behavior.

s72
00:06:58.680 --> 00:07:09.400
Different agent harnesses will leverage and inject long-term memory in different ways, whether you're building your own SDK or or using a variety of different popular agent frameworks.

s73
00:07:09.400 --> 00:07:11.680
Some inject it directly into the prompt.

s74
00:07:11.680 --> 00:07:13.480
Some will use tools to retrieve that.

s75
00:07:13.480 --> 00:07:17.080
Some will Some will pull it directly from a store.

s76
00:07:17.080 --> 00:07:20.680
Some load it through files or some change the run time state.

s77
00:07:20.680 --> 00:07:24.400
The The implementation can vary, but the distinction is really important.

s78
00:07:24.400 --> 00:07:33.120
Some of the memory or some of the context that an agent references is temporary while some of it is durable and long-term and is useful to guide future runs.

s79
00:07:33.120 --> 00:07:42.840
And I think that distinction is really important when we think about traces and logs and artifacts that the agent produces as it runs.

s80
00:07:42.919 --> 00:07:45.760
A trace isn't just a record of the final output.

s81
00:07:45.760 --> 00:07:50.640
It's a record of how our working memory changed over the course of the run.

s82
00:07:50.640 --> 00:07:53.560
Some part of that should stay as history.

s83
00:07:53.560 --> 00:07:57.960
Some part of that should become durable long-term context.

s84
00:08:00.920 --> 00:08:10.160
And so that's what gives Oh, that's what lends itself to this really helpful mental model that I have or that I and and the team at LangChain and generally I think

s85
00:08:10.160 --> 00:08:16.320
is is a very useful way to sort of think about the relationship between working or short-term memory and long-term memory.

s86
00:08:16.320 --> 00:08:18.080
It's a read-write cycle.

s87
00:08:18.080 --> 00:08:21.680
And so you might have heard of sleep-time compute or dreaming.

s88
00:08:21.680 --> 00:08:33.960
Those terms refer to the way that memory is updated or pulled from long-term into short-term whilst the agent is doing work or actually in some cases whilst the agent is also not doing work.

s89
00:08:33.960 --> 00:08:36.800
That's the sleep-time compute.

s90
00:08:36.880 --> 00:08:43.960
If we start somewhere at the beginning of an agent run, it needs to read relevant context from long-term memory into its short-term memory.

s91
00:08:43.960 --> 00:08:50.920
So this is anything This is anything important for the current run that it has It might be skills, it might be instructions.

s92
00:08:50.920 --> 00:08:55.960
You've probably seen progressive disclosure with agents that actually pull in the relevant skills for a certain task.

s93
00:08:55.960 --> 00:09:00.360
That's that phase moving from long-term memory into short-term memory.

s94
00:09:00.360 --> 00:09:03.240
Then when the agent runs, it produces a trace.

s95
00:09:03.240 --> 00:09:16.000
This evidence is really important to us because now we can understand how it retrieves context, the the tool calls that it made or the decisions or the sub-agents or anything that's part of its working job.

s96
00:09:16.080 --> 00:09:18.800
As I mentioned, most evidence should stay as history.

s97
00:09:18.800 --> 00:09:26.600
If we turned everything that was from a If we turned all of that evidence into long-term memory, then it would make the agent struggle and it would make it harder to reason over future tasks.

s98
00:09:26.600 --> 00:09:31.160
And so what I think is really interesting is the filter step.

s99
00:09:31.160 --> 00:09:39.800
How do you decide How do you decide what is useful signal and what should become durable long-term context and how do you decide what should stay as history.

s100
00:09:39.800 --> 00:09:42.280
Once you've made that decision and we'll get into that in a moment.

s101
00:09:42.280 --> 00:09:48.880
Once you've made that decision, that is that signal is then written back into durable long-term context.

s102
00:09:48.880 --> 00:09:58.200
So, in the case of our financial assistant, that was be measured and not pointed or not directive, so that it can update the way that it behaves on future turns.

s103
00:09:58.200 --> 00:09:59.200
And that's the core loop.

s104
00:09:59.200 --> 00:10:01.400
Memory informs the run.

s105
00:10:01.400 --> 00:10:03.240
The run makes the evidence.

s106
00:10:03.240 --> 00:10:08.040
We filter that evidence to make signal, and then we use that signal to update memory.

s107
00:10:08.040 --> 00:10:17.920
If you can build that flywheel, then we trend towards a place where we can build very reliable and very powerful powerful agents that just feel like they get better with time.

s108
00:10:18.280 --> 00:10:27.440
And so, once we have that mental model, I think, you know, an even more simple way to break this down or to to abstract this away is is capturing what happened,

s109
00:10:27.440 --> 00:10:33.960
analyzing what might matter, and then updating the context so that future runs can use this.

s110
00:10:33.960 --> 00:10:43.320
The implementations around how different how how different people building different frameworks are still evolving, but this three-part shape is a really useful starting point.

s111
00:10:44.400 --> 00:10:55.280
One implementation of this, and I won't I won't claim it's the only implementation, but just as just a kind of like put some concrete structure to what I'm talking about is the way that we do this with LangSmith.

s112
00:10:55.480 --> 00:11:04.520
The capture step, to be able to understand and see the trajectory and the things that the agent did, is where our observability comes in.

s113
00:11:04.520 --> 00:11:05.760
You're able to see the tool calls.

s114
00:11:05.760 --> 00:11:08.000
You're you're able to see the way that the agents called sub-agents.

s115
00:11:08.000 --> 00:11:11.440
You're you're able to see the context that was retrieved.

s116
00:11:11.600 --> 00:11:19.440
The analyze step in the middle is something new that I'm really excited about, but basically, it is that intelligent process that's doing the background analysis and doing the background work

s117
00:11:19.440 --> 00:11:32.160
to extract signal from traces based on the things that matter to you, and then promoting that signal to the memory store or the point of reference that the agent will use for future runs, and that is LangSmith's context hub.

s118
00:11:32.160 --> 00:11:45.720
So, you can imagine now if there is a place where this analysis can be done, signal can be in track extracted, a remote store where you can now go and update what's in that so that the agent can pull that down the next time it runs.

s119
00:11:45.720 --> 00:11:48.640
That is what LangChain's context hub does.

s120
00:11:48.640 --> 00:11:50.840
And the abstract loop it maps cleanly.

s121
00:11:50.840 --> 00:11:58.040
You capture the experience, you extract the signal, and then you store that durable context for future runs.

s122
00:11:58.720 --> 00:12:11.640
To kind of put that in some more detail and and also to frame it in the in the the context of the financial assistant that I was speaking about at the beginning and and sort of the way that we we built this loop.

s123
00:12:11.640 --> 00:12:21.920
When that agent runs, its context is loaded from the context hub and so that might include its instructions, its skills, relevant markdown files that matter to it, policies, and rules.

s124
00:12:21.920 --> 00:12:34.640
When it functions or when we invoke that agent, then the a trace is produced that shows us the the uh failed tool calls or user corrections or in this case it flagged where our tone strayed from what we wanted it to be.

s125
00:12:34.640 --> 00:12:46.800
Engine then looks for patterns and then identifies where different static skill markdown files that are based in context hub need to be changed or promoted, and it'll make that change.

s126
00:12:47.839 --> 00:12:56.440
And then the next time our agent runs, we're able to actually go and and understand and see that its tone changes when it interacts with engages with users.

s127
00:12:59.880 --> 00:13:09.560
So, some things that I and my team have currently uh or kind of always think about when when designing memory and and this should also as well ideally guide

s128
00:13:09.560 --> 00:13:16.720
the the way that you design and build some of these abstractions is that agents produce so much we call it exhaust or we call it kind of like feedback.

s129
00:13:16.720 --> 00:13:18.800
Not everything should become a memory update.

s130
00:13:18.800 --> 00:13:26.480
And actually the devil is in the details around what should become one as as we've sort of discussed over the course of this presentation.

s131
00:13:26.800 --> 00:13:29.280
It's understanding and deciding what matters to you.

s132
00:13:29.280 --> 00:13:32.760
You know, most trace data actually should stay as history as referencable history.

s133
00:13:32.760 --> 00:13:37.120
Some some might become data sets to do offline evaluations and things like that.

s134
00:13:37.120 --> 00:13:44.800
A small a small segment of that or or subset should end up becoming what changes your agent's memory.

s135
00:13:44.800 --> 00:13:55.400
Now, this second one is sort of like a gotcha that I I ran into a lot and I think that you sort of tread the fine line of trying to build a system that's optimized, but then also one that has

s136
00:13:55.400 --> 00:13:58.600
a memory store or updates available to it in the hot path.

s137
00:13:58.600 --> 00:14:03.720
And what I found was uh for particularly for long-running background agents, right?

s138
00:14:03.720 --> 00:14:13.600
Imagine if you have an agent that's running over long time horizons and you make some memory updates in the beginning of its operation, but because of the run time state and because of the way that it's accessing memory, you don't actually

s139
00:14:13.600 --> 00:14:16.880
access that or that's not available to it for future runs.

s140
00:14:16.880 --> 00:14:31.560
And so, you know, understanding what you can cache, what you can't cache to be able to make sure that memory updates are available for future runs, very important uh and sort of I was I was banging my head on a wall for a while while while I was trying to sort of understand why my behavior wasn't changing

s141
00:14:31.560 --> 00:14:36.200
and I needed to recognize and think about that second principle.

s142
00:14:36.280 --> 00:14:37.200
And then the third one as well.

s143
00:14:37.200 --> 00:14:47.240
I think that uh open core and uh you know, Hermes agent and and a lot of agents that are uh becoming widely popular that sort of do this self-update process,

s144
00:14:47.240 --> 00:14:47.440
right?

s145
00:14:47.440 --> 00:14:54.960
I think that it's another really important consideration of like what do you make self-updateable as opposed to what do you make something that needs human review?

s146
00:14:54.960 --> 00:15:04.440
And so, if we're talking about procedural memory with instructions and policies and tone guidelines, those are probably going to drive about 75% of the behavior of my agent.

s147
00:15:04.440 --> 00:15:08.480
And so, that actually probably should have some level of human in the loop and human interaction.

s148
00:15:08.480 --> 00:15:10.360
You should have somewhere to surface it.

s149
00:15:10.360 --> 00:15:21.560
The changes that you make to the parts of the agent's memory that are procedural before actually committing those and and putting those in the in the in the agent's sort of like hot reload path.

s150
00:15:21.600 --> 00:15:31.040
So, protecting important behavior with the is, I guess, another really important third third kind of like level to this that I find really important.

s151
00:15:31.040 --> 00:15:33.280
So, I'm going to come back to this initial refrain.

s152
00:15:33.280 --> 00:15:35.200
The next run should be better.

s153
00:15:35.200 --> 00:15:36.960
Memory allows us to do this.

s154
00:15:36.960 --> 00:15:39.760
The word memory is thrown around a lot.

s155
00:15:39.760 --> 00:15:48.400
I think in the context of building agents and working with harnesses and working with like providing context to your agent, memory really does give us the ability to do this.

s156
00:15:48.400 --> 00:15:51.640
And it's more than just a place to store information.

s157
00:15:51.640 --> 00:15:54.560
It's how experience becomes context.

s158
00:15:54.560 --> 00:16:00.480
And then that context is how our next runs get the chance to improve.

s159
00:16:00.480 --> 00:16:09.760
If you're curious about an actual implementation of this, by the way, that QR code has a video of me walking through how I built an agent that does exactly what I've just described.

s160
00:16:09.760 --> 00:16:15.400
Uh hopefully that'll kind of like make it tangible and and will sort of connect some of those dots.

s161
00:16:15.400 --> 00:16:18.320
Uh and if you want to have any further discussions about this, please come and see me.

s162
00:16:18.320 --> 00:16:20.760
We're at booth Eugene 19.

s163
00:16:20.760 --> 00:16:23.840
Uh I'd love to