WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=RGSFUqzqErE

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/RGSFUqzqErE.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.760 --> 00:00:20.600
Now taking the stage is CVP and distinguished engineer at Microsoft, Pablo Castro.

s3
00:00:22.720 --> 00:00:25.080
Hello everyone.

s4
00:00:25.560 --> 00:00:26.280
Hello everyone.

s5
00:00:26.280 --> 00:00:28.240
Good morning.

s6
00:00:28.240 --> 00:00:33.320
It's great to be back here at the AI Engineer World's Fair.

s7
00:00:33.320 --> 00:00:40.280
Now, my job at Microsoft is to connect the dots between AI and knowledge.

s8
00:00:40.280 --> 00:00:43.480
As an information retrieval nerd, like that's great for me.

s9
00:00:43.480 --> 00:00:50.200
Like I spend a lot of time on looking at knowledge representation, extraction, search, and whatnot.

s10
00:00:50.200 --> 00:00:56.640
And thinking about agents and knowledge really invites to reflect on, you know, what it means to know something.

s11
00:00:56.640 --> 00:01:02.840
And uh, you know, the the the nature of how do we get things done based on what we know.

s12
00:01:02.840 --> 00:01:05.040
Next slide.

s13
00:01:09.240 --> 00:01:11.360
All right.

s14
00:01:14.440 --> 00:01:15.240
There.

s15
00:01:15.240 --> 00:01:23.600
So, this morning what I thought we would do is spend a little bit of time talking about the nature of knowledge and split it into these three categories of intrinsic,

s16
00:01:23.600 --> 00:01:26.720
extrinsic, and learned.

s17
00:01:27.160 --> 00:01:30.080
Intrinsic knowledge is just the knowledge that comes with the models.

s18
00:01:30.080 --> 00:01:38.440
You know, it's what we uh train the models on, the training data, and what um uh stored in the models' kind of parametric memory.

s19
00:01:38.440 --> 00:01:45.520
And while it's kind of the obvious thing, I would argue this is the knowledge that actually threw us into the exponential we are in today.

s20
00:01:45.520 --> 00:01:51.480
It's what started many of the scenarios that then grew on all the things we're doing with agents today.

s21
00:01:52.320 --> 00:01:54.120
Let me give you an example with code.

s22
00:01:54.120 --> 00:01:59.160
So, I wrote these two pieces of code about 25 years apart.

s23
00:01:59.160 --> 00:02:04.000
And yet, the process to put this thing together was surprisingly similar.

s24
00:02:04.000 --> 00:02:11.040
Like I had to sit down with what I knew or what I had to go look up and then just write it up.

s25
00:02:11.040 --> 00:02:19.360
And while, you know, I'm illustrating this with knowledge, you could say the same thing about, you know, writing an email or creating a summary of a document.

s26
00:02:19.360 --> 00:02:25.000
Now, you can see this exponential at play in tasks like these where, you know, I'm sure you can go further back.

s27
00:02:25.000 --> 00:02:30.560
But an interesting point in time to start looking at this would be when Microsoft introduced IntelliSense.

s28
00:02:30.560 --> 00:02:32.000
That was in '96.

s29
00:02:32.000 --> 00:02:32.959
And, you know, it was great.

s30
00:02:32.959 --> 00:02:36.680
You didn't have to remember function signatures anymore and whatnot.

s31
00:02:36.680 --> 00:02:47.520
It takes 22 years from there to go for the next step where machine learning helps us actually rank the options we give you in IntelliSense, so it's quicker to pick the right choice.

s32
00:02:47.520 --> 00:02:51.360
Just 3 years after that, GitHub Copilot launches.

s33
00:02:51.360 --> 00:02:54.200
And that was one key inflection point.

s34
00:02:54.200 --> 00:02:57.280
This was even before ChatGPT was announced.

s35
00:02:57.280 --> 00:03:06.920
And, you know, I would argue that GitHub Copilot, ChatGPT, that sort of experiences were heavily grounded on this intrinsic memory, what the models already knew.

s36
00:03:06.920 --> 00:03:08.840
From there, of course, things shifted.

s37
00:03:08.840 --> 00:03:18.160
You know, a couple of years later, Cursor launches, GitHub Copilot X launches, and how we do things kind of evolved really quick, which takes us to kind of late last year.

s38
00:03:18.160 --> 00:03:19.480
Opus 4.5 ships.

s39
00:03:19.480 --> 00:03:25.040
And then rapid succession, you know, GPT, Opus, and other models keep getting better and better at coding.

s40
00:03:25.040 --> 00:03:35.880
Which takes us to early this year, where incredibly successful software like like Open Claw comes out to existence with not a single line of code written by hand.

s41
00:03:35.880 --> 00:03:38.080
So, this is the shape of the exponential we're in.

s42
00:03:38.080 --> 00:03:46.320
And a lot of this was powered by the by the intrinsic knowledge in models and of course their ability to reason.

s43
00:03:47.239 --> 00:03:54.480
Now, in the context of Microsoft, we want to make available all these models and make it easy for you to integrate them into the agents you're building.

s44
00:03:54.480 --> 00:04:00.360
We do this from our agent platform that starts in GitHub where we all go and build.

s45
00:04:00.360 --> 00:04:03.920
It has a contextualization system so you can ground your agents.

s46
00:04:03.920 --> 00:04:10.440
And when it comes to agent hosting, observability, and management, we all do all of these in Foundry.

s47
00:04:10.440 --> 00:04:19.640
Microsoft Foundry is also where we offer thousands of models in our model catalog so you can pick whatever is the right model for the right task.

s48
00:04:19.640 --> 00:04:21.519
And we keep adding more every day.

s49
00:04:21.519 --> 00:04:34.120
In fact, just yesterday we announced that Claude in Microsoft Foundry is generally available so you can use all the capabilities of Claude in the context of the unified experience in Foundry.

s50
00:04:34.120 --> 00:04:37.320
So you get best of both worlds.

s51
00:04:38.320 --> 00:04:48.120
Now, an interesting model got us here, but it only gets you so far if you're building a system that or an agent that needs to participate in what's happening in an organization

s52
00:04:48.120 --> 00:04:49.400
or a company.

s53
00:04:49.400 --> 00:04:56.440
And you know, as an industry we realized this early and we, you know, we saw the the rag pattern emerge.

s54
00:04:56.440 --> 00:05:12.120
That started as a pretty low-tech technique, but quickly evolved and what we do today with context engineering and you know, it became a pretty sophisticated system for connecting agents and the knowledge they need to get their job done.

s55
00:05:12.840 --> 00:05:17.919
Of the many dimensions of of which this got kind of complicated, I'm going to pick on two.

s56
00:05:17.919 --> 00:05:25.280
One is kind of the evolution from simple and isolated data sets to whole company-wide grounding.

s57
00:05:25.280 --> 00:05:33.919
And the other one is how we started with simple vector search and whatnot and we really saw this evolve into fairly complicated retrieval systems.

s58
00:05:33.919 --> 00:05:36.480
So let's start with company grounding.

s59
00:05:36.480 --> 00:05:48.000
Like at Microsoft, you know, spending time with customers, one of the things we saw early was that whenever you build an agent, you you always have the knowledge you care about for that agent and you'll manage that yourself,

s60
00:05:48.000 --> 00:05:54.480
but you also need to ground the agent often on the kind of ambient data of your organization, you know, whenever the agent leaves.

s61
00:05:54.480 --> 00:06:03.600
This includes maybe your documents, your emails, your chat threads, or the information in your data warehouse and whatnot.

s62
00:06:03.600 --> 00:06:14.120
So, we built Microsoft IQ as a way to give you a single entry point into all these kind of ambient data that agents need to get the job done in addition to the specific information

s63
00:06:14.120 --> 00:06:16.680
that you build into the agent.

s64
00:06:16.680 --> 00:06:26.480
Microsoft IQ is not one feature, it's more like a set of capabilities that goes from work IQ that connects your agents to all the documents in say SharePoint, all the emails, calendar,

s65
00:06:26.480 --> 00:06:37.720
your chats, and the connections between people to fabric IQ that gives you access to all the all your analytics assets, you know, from data warehouses and data lakes to Power BI reports,

s66
00:06:37.720 --> 00:06:43.919
and foundry IQ, which is what you use for your all agents where you can push your own data and then use it for grounding.

s67
00:06:43.919 --> 00:06:55.919
And of course, sometimes you have your agents need to go out to the web to ground on data maybe not yours, it's public information, but but you need to use it to complete

s68
00:06:55.919 --> 00:07:01.440
the picture of what the agent world's view is, and for that we have web IQ.

s69
00:07:01.440 --> 00:07:06.640
Now, this first part allows agents to ground on the kind of these ambient data.

s70
00:07:06.640 --> 00:07:12.760
Now, the second dimension I mentioned before is the evolution of the actual retrieval systems.

s71
00:07:12.760 --> 00:07:25.720
You know, when drive first emerged, I think, you know, what we saw is like an initial adoption for vector databases that really unblocked us from getting a lot of these systems off the ground, and that was great.

s72
00:07:25.720 --> 00:07:35.560
Um, I think, you know, for a hot second as an industry, we thought that if we could get really, really good at computing cosine similarity between vectors, we were all set for retrieval.

s73
00:07:35.560 --> 00:07:38.760
It turns out, you know, things never are are never that easy.

s74
00:07:38.760 --> 00:07:46.640
Uh, so, you know, what evaluations show over and over again is how, you know, if you combine methods, you just get better results.

s75
00:07:46.640 --> 00:07:52.120
Like in this case, this is an evaluation from actually I search, the search technology behind Foundry IQ.

s76
00:07:52.120 --> 00:08:01.320
And you can see how individual methods don't do as well as combined methods, particularly when you apply them to real-world customer scenarios.

s77
00:08:01.320 --> 00:08:08.800
Now, the trick is how you build a platform that allows you to combine all these building blocks without putting the complexity right in front of you.

s78
00:08:08.800 --> 00:08:15.919
You like let you opt into it when you need control, but when you have a scenario that is clear, then you can have an easy system.

s79
00:08:15.919 --> 00:08:19.400
So, in Foundry IQ, that was one of our core design goals.

s80
00:08:19.400 --> 00:08:22.520
And the way we do this is we actually layer the system.

s81
00:08:22.520 --> 00:08:28.800
So, you can start at the top, you can go to Foundry and say, "Hey, I have a bunch of, I don't know, PDFs or pictures over there.

s82
00:08:28.800 --> 00:08:29.960
Just deal with them."

s83
00:08:29.960 --> 00:08:37.960
And then we'll do everything under the covers, do like, I don't know, chunking, vectorization, deal with relevance and ranking, deal with agentic retrieval and whatnot.

s84
00:08:37.960 --> 00:08:41.240
Now, if you're an expert and you want control, you can also do that.

s85
00:08:41.240 --> 00:08:48.839
You can go to the bottom of the stack, you want to build vector indexes and tell us how to quantize the vectors or control lexical retrieval and whatnot.

s86
00:08:48.839 --> 00:08:56.160
You can do all of that and you can do it in the same stack, which means you can go up and down as you as your needs change.

s87
00:08:56.240 --> 00:09:10.960
Now, on on top of the core retrieval system, we also introduced an agentic retrieval stack because we see that for easy cases, like, you know, quick single-shot retrieval is great, but for more sophisticated cases, you do want

s88
00:09:10.960 --> 00:09:23.640
a system that can reflect on on what's in the data set and decide whether or not we've satisfied the information need as stated in the input before we come back with results.

s89
00:09:24.160 --> 00:09:29.520
Of course, we see a lot of patterns like this emerge and they always the question is, is this actually useful?

s90
00:09:29.520 --> 00:09:31.560
Like, are the results better?

s91
00:09:31.560 --> 00:09:38.600
Our experience in our own evaluations is for for difficult cases, agentic retrieval can a difference.

s92
00:09:38.600 --> 00:09:52.440
You across the many metrics that we we track, you know, things like um the actual actual evidence recall or answer completeness, we see like the agentic retrieval approach continuously does better than than simple that's

s93
00:09:52.440 --> 00:09:54.440
uh individual simple parts.

s94
00:09:54.440 --> 00:09:59.560
Now, let me show you some of these in action if you can go to laptop.

s95
00:10:06.280 --> 00:10:08.800
Can we switch to the laptop?

s96
00:10:12.400 --> 00:10:14.120
There you go.

s97
00:10:14.120 --> 00:10:26.000
So, here I'm in I'm in Foundry, and uh Foundry is where you you manage your agents, manage models, but also the place where you can manage all the knowledge that you give your agents in order to

s98
00:10:26.000 --> 00:10:27.600
uh do their jobs.

s99
00:10:27.600 --> 00:10:34.840
Um when here, you can you can create knowledge bases as the kind of the entry point of energy agents into the knowledge you care about.

s100
00:10:34.840 --> 00:10:36.280
In this case, I'll create a knowledge base.

s101
00:10:36.280 --> 00:10:38.160
I have a data set about movies.

s102
00:10:38.160 --> 00:10:43.120
These are agentic retrieval systems, so I'll give it a model to power the retrieval workflow.

s103
00:10:43.120 --> 00:10:48.040
And I can say how much effort you want the model to uh to make or the system to make.

s104
00:10:48.040 --> 00:10:51.760
And this is effectively a trade-off between latency and quality.

s105
00:10:51.760 --> 00:10:57.920
I can configure a number of other things, but critically, I want to say where the data I want to ground is coming from.

s106
00:10:57.920 --> 00:11:05.360
And uh I can start from scratch, or in this case, I have a bunch of unstructured data like PDFs and whatnot in in blob storage.

s107
00:11:05.360 --> 00:11:10.880
I have structured, you know, parquet tables with statistics, and I also want to ground on the web.

s108
00:11:10.880 --> 00:11:20.839
So, if I take these three steps, and then I save this knowledge base, now I have this asset, this knowledge base that I can connect to an uh Foundry agent right here, and it'll take a second.

s109
00:11:20.839 --> 00:11:32.960
Uh but also it's a standalone asset that if I have already a harness that I'm using in other in other places, every knowledge base is an MCP server, so you can just connect to it uh without having to write any glue code in the middle.

s110
00:11:32.960 --> 00:11:37.160
Now, a knowledge base like this has uh you know, has a bunch of parts.

s111
00:11:37.160 --> 00:11:45.800
Some of them, like uh for example, this uh storage uh content, you usually build indexes uh and you you know, vectorize these things and whatnot.

s112
00:11:45.800 --> 00:11:49.280
Uh and if you want control over that, like if I if you don't, you can just use it here.

s113
00:11:49.280 --> 00:11:55.160
But, if you do, let me just switch to Azure and show you the service behind that particular instance.

s114
00:11:55.160 --> 00:11:59.960
Where if I go to knowledge bases, this is the knowledge base we just created a second ago.

s115
00:11:59.960 --> 00:12:01.200
And I can go peek inside.

s116
00:12:01.200 --> 00:12:06.440
For example, I can go fish out the indexes that back this particular uh piece of content.

s117
00:12:06.440 --> 00:12:09.960
And in that index, I can see what is the structure of the index.

s118
00:12:09.960 --> 00:12:18.400
Uh if I'm opinionated about, I don't know, maybe the quantization uh approach I want to use or which indexing algorithm I want for my vectors, I can say all of that.

s119
00:12:18.400 --> 00:12:21.600
And of course, I can actually go and explore the data.

s120
00:12:21.600 --> 00:12:26.200
And you know, see what's inside, how chunks were organized, and and whatnot.

s121
00:12:26.200 --> 00:12:36.640
So, the goal of this is to again give you high product a highly productive environment when you need uh when you don't need uh the sophistication, and when you need it to make sure you have it

s122
00:12:36.640 --> 00:12:37.880
to get your job done.

s123
00:12:37.880 --> 00:12:40.360
We go back to slides.

s124
00:12:45.680 --> 00:12:51.680
And of course, the other aspect of this is, you know, top of mind these days for all of us is token uh is token efficiency.

s125
00:12:51.680 --> 00:13:02.720
And uh so, uh we carefully evaluate this system to make sure that we give you the most information dense answer that has the fewest tokens uh so that you you know, the

s126
00:13:02.720 --> 00:13:08.200
the your consumption of tokens has a high value when it comes to all retrieval tasks.

s127
00:13:08.920 --> 00:13:12.960
The last category of knowledge I wanted to talk about is learned knowledge.

s128
00:13:12.960 --> 00:13:19.440
Now, learned knowledge is the result of us doing the work we do as individuals and as organizations every day.

s129
00:13:19.440 --> 00:13:34.400
And the the idea that we can actually observe the processes and get better at them by reflecting and improving every step of it is something that is really uh changed now that we have agents doing the work and we can go tune the agents automatically.

s130
00:13:34.800 --> 00:13:44.640
Satya wrote about this recently and reflected on the fact that people and agents can really compound in in how they do the work and how they can create this learning loop

s131
00:13:44.640 --> 00:13:54.840
that effectively captures what's unique about the company or or the organization you're working on and inputs that to work to differentiate the work that you do.

s132
00:13:54.840 --> 00:14:00.640
Now in Foundry we wanted to offer like a material a materialized version of this that you can use today.

s133
00:14:00.640 --> 00:14:13.360
So we built a component called the agent optimizer that effectively goes through this process and allows you to evaluate a baseline, generate candidates, and then you know, evaluate the new candidates and we have a strong result,

s134
00:14:13.360 --> 00:14:15.120
then deploy that to production.

s135
00:14:15.120 --> 00:14:20.600
Let me give you a kind of a quick flavor of what this looks like if we can switch back to the laptop.

s136
00:14:25.040 --> 00:14:25.720
All right.

s137
00:14:25.720 --> 00:14:30.839
So here I'm I mean VS Code, I have the Foundry toolkit installed.

s138
00:14:30.839 --> 00:14:40.480
And I have a simple agent, it doesn't matter how you write your agent as long as you externalize configuration like you know, your instructions, tool definitions, skills, and whatnot.

s139
00:14:40.480 --> 00:14:44.280
So once you have one of those, it takes two key steps to do this.

s140
00:14:44.280 --> 00:14:48.280
So first oops.

s141
00:14:50.120 --> 00:15:06.280
Um I can actually so usually you have an evaluation already, but if you don't, you can actually say eval generate and what we'll do is we'll look at what we know about the agent like traces and instructions and whatnot and we'll produce a task adherence focused evaluation for you.

s142
00:15:06.280 --> 00:15:08.800
In this case I run this a little bit earlier.

s143
00:15:08.800 --> 00:15:17.280
So just to give you a flavor of what this looks like, you you have a bunch of tasks and then you know, the questions and the criteria and whatnot.

s144
00:15:17.560 --> 00:15:24.280
Once you have a dataset you can evaluate then next step is you can say uh optimize.

s145
00:15:24.280 --> 00:15:36.880
And uh I could just run optimize on its own and that will run In this case, this run for maybe 45 minutes or so and you get an optimized version by effectively hill climbing the metric that's established from by evaluation.

s146
00:15:36.880 --> 00:15:42.040
Um so I run this earlier and so let me show you the output for this particular one.

s147
00:15:42.040 --> 00:15:54.160
Where you can see that, you know, we established the baseline first and then we kept iterating on candidates uh using different combinations using a J power style kind of loop uh and uh looking for options that perform better given

s148
00:15:54.160 --> 00:15:55.880
uh the rubric that we have.

s149
00:15:55.880 --> 00:16:05.000
And uh the interesting thing is that once you found one that is that is better, then you can simply just say optimize apply and what this does is since you externalize the configuration,

s150
00:16:05.000 --> 00:16:07.960
it allows you to swap one configuration for the other.

s151
00:16:07.960 --> 00:16:13.520
Um if I if if we look here, you can see that for example, I have baseline and the one we just applied.

s152
00:16:13.520 --> 00:16:18.840
And just to pick on instructions, these are just the uh trivial instructions for this example agent.

s153
00:16:18.840 --> 00:16:29.760
But if I look at the optimized one, then you can see like a bunch of instructions that are not handwritten but that that they emerged out of the hill climbing process to get to make this particular

s154
00:16:29.760 --> 00:16:41.440
um agent better given what we have in terms of instructions and skills and tools, but also based on reflecting on the actual uh traces from the agent as users are using it.

s155
00:16:41.440 --> 00:16:45.680
So this is a real learning loop materialized in practice.

s156
00:16:45.680 --> 00:16:48.160
You can go back to slides.

s157
00:16:48.960 --> 00:16:59.600
So this was like a very quick overview about how do we think about knowledge in the context of AI and how do how we think we can enable these learning loops that will capture,

s158
00:16:59.600 --> 00:17:05.199
you know, these differentiated capability that lives in each one of the companies and organizations we work on.

s159
00:17:05.199 --> 00:17:11.680
If you want to try anything of what I talked about or showed today, you can head to ai.azure.com and get going.

s160
00:17:11.680 --> 00:17:13.959
And with that, thank you all for listening this morning.

s161
00:17:13.959 --> 00:17:15.959
I hope you have a great rest of the event.

s162
00:17:15.959 --> 00:17:17.959
Thanks.