WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=jt1Pbr_n6oU

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/jt1Pbr_n6oU.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.800 --> 00:00:13.040
Yes.

s3
00:00:13.040 --> 00:00:16.800
So my talk today is about the title your data models remote.

s4
00:00:16.800 --> 00:00:21.439
We have a enterprisewide platform that we had just rolled out here this past month.

s5
00:00:21.439 --> 00:00:24.080
And so I'll go into details on this.

s6
00:00:24.080 --> 00:00:35.440
I'll give you some hopefully some practical lessons here and why we made decisions we made for this uh how how you could picture your processes within a similar type framework.

s7
00:00:35.440 --> 00:00:38.320
So first just a quick introduction.

s8
00:00:38.320 --> 00:00:48.320
So this gets into the the title here the talk and the the framing of you know what I hope you take from this but with AI moving very fast at the frontier

s9
00:00:48.320 --> 00:01:01.840
what what's defensible you know you can move you can build things very quickly with clogged code um but once you push things to production there's constraints you find how much of your your

s10
00:01:01.840 --> 00:01:11.600
uh deployed stack do you want to actually own you know there's monitoring there's upkeep there's uh you know people building dependencies off your stack that you have to be prepared to handle.

s11
00:01:11.600 --> 00:01:15.040
How much uh appetite do users have for decentralized access?

s12
00:01:15.040 --> 00:01:21.439
This gets into I'll show you what we built, but you know what's what's the access point for users?

s13
00:01:21.439 --> 00:01:23.040
You know, is it another chat app?

s14
00:01:23.040 --> 00:01:24.159
Is it clawed?

s15
00:01:24.159 --> 00:01:25.920
Is it chat GPT?

s16
00:01:25.920 --> 00:01:28.000
Uh is it something else?

s17
00:01:28.000 --> 00:01:32.560
What's your product differentiation from from those different SAS products?

s18
00:01:32.560 --> 00:01:42.400
And so our team then you know with this context in mind you know thought through here you know what's our skill set here what's our competitive advantage in this environment

s19
00:01:42.400 --> 00:01:53.040
and this is what I you really hope that you you take from this talk and you picture yourself in this but our moat here was our understanding of our internal processes

s20
00:01:53.040 --> 00:02:05.360
the tacet knowledge that you need to to run successful AI and this is true I think no matter how good AI gets how good models get and new releases that different companies put out when when

s21
00:02:05.360 --> 00:02:08.239
Mythos comes out or when there's a new app from Claude.

s22
00:02:08.239 --> 00:02:15.920
Yeah, I'm not I'm not worried because the part that we've built is the defensible part that that's that's durable.

s23
00:02:15.920 --> 00:02:26.800
So, these are the I I'll tell you what this means here in more detail, but these are the the processes tacet knowledge that we've modeled into what we call the strategic intelligence platform or SIP.

s24
00:02:26.800 --> 00:02:32.480
and it rolled out here this past month in production for enterprise use across uh the Gates Foundation.

s25
00:02:32.480 --> 00:02:35.360
So about 4,000 people.

s26
00:02:36.080 --> 00:02:51.280
So first I know this is an engineering talk but the the the scope of this talk gets into data modeling internal operations processes and so I want to give very quick background here over what the Gates Foundation does because then this is what we're we're modeling.

s27
00:02:52.160 --> 00:03:02.159
So, as you're probably familiar, the the Gates Foundation has a has a very wide scope and it's a very ambitious work that we've been doing for the past 25 plus years.

s28
00:03:02.159 --> 00:03:14.080
And there's all kinds of, you know, broad initiatives that we're doing, you know, whether it's for uh child mortality, whether it's for nutrition, agriculture, uh education.

s29
00:03:14.080 --> 00:03:19.360
And these are kind of broadly the the different buckets that these different initiatives fit fit into.

s30
00:03:19.360 --> 00:03:24.720
creating market incentives, spurring an innovation, collaboration between public and private sectors.

s31
00:03:24.720 --> 00:03:30.159
And then the fourth one here kind of gets into the the lens that we're building here.

s32
00:03:30.159 --> 00:03:38.959
You know, high quality data trying to derive datadriven insights from the from the actual investments, the the grants that we've put out.

s33
00:03:38.959 --> 00:03:41.680
And over 25 years, there's a ton of structure.

s34
00:03:41.680 --> 00:03:43.760
There's a ton of data that's developed.

s35
00:03:43.760 --> 00:03:48.400
And trying to extract those insights at scale is difficult.

s36
00:03:48.400 --> 00:03:51.519
And that's what we're trying to solve.

s37
00:03:52.400 --> 00:04:01.599
So this slide here is a snapshot of the of some of the different uh of the work that went out in 2023 within the foundation.

s38
00:04:01.599 --> 00:04:02.480
This gives you an idea.

s39
00:04:02.480 --> 00:04:08.239
I just put this here to to show some of the structured the structure that we have that we're working across.

s40
00:04:08.239 --> 00:04:11.680
So you have over 2,000 grants in one year.

s41
00:04:11.680 --> 00:04:21.040
Many of these are 5 million plus uh many 100 plus countries that uh that that are targeted with these grants uh alumni.

s42
00:04:21.040 --> 00:04:24.400
So there's 4,000 different employees of the foundation.

s43
00:04:24.400 --> 00:04:37.120
Um you know many different strategies within the foundation the US within the US across almost all the states grantees the total annual dispersement over 7 billion dollars.

s44
00:04:37.120 --> 00:04:40.880
And so this gives you some idea of structure that we're we're working with.

s45
00:04:40.880 --> 00:04:45.120
And this one just finally here when I show the data model this will make more sense.

s46
00:04:45.120 --> 00:04:49.440
But we have different divisions that that funding goes out through different divisions.

s47
00:04:49.440 --> 00:04:52.639
And so this breaks down some of those divisions.

s48
00:04:52.639 --> 00:04:56.880
So you can see different priorities and it'll make more sense in a second here.

s49
00:04:56.880 --> 00:05:05.120
But global development, global health, uh gender equality, USP are just a sample of the different divisions.

s50
00:05:05.120 --> 00:05:05.600
Okay.

s51
00:05:05.600 --> 00:05:17.600
So the the fun stuff here now I hope the uh strategic intelligence platform so in a in a in a nutshell here structuring operational data for agentic retrieval.

s52
00:05:17.600 --> 00:05:27.280
So we're building a knowledge graph with the idea of the agent consumer and here is an endto-end look of what this looks like.

s53
00:05:27.280 --> 00:05:42.560
So we we have different systems of record structured unstructured these have been siloed traditionally the so part of our team here the work has been to create what's essentially a data lakehouse

s54
00:05:42.560 --> 00:05:45.120
putting everything under one roof.

s55
00:05:45.120 --> 00:05:47.759
This is our internal enterprisewide data.

s56
00:05:47.759 --> 00:05:54.160
It's also different different programmatic data that are uh outputs of different investments.

s57
00:05:54.160 --> 00:05:56.080
Once it's there it's easy for us to consume.

s58
00:05:56.080 --> 00:06:12.080
So we have a data curation layer that does different processing to it and then finally SIP here at the end with aentic chat agentic workflow as the as the the UX you how users are consuming our platform and so it's a cross system semantic graph layer that agents can res

s59
00:06:12.080 --> 00:06:23.440
across okay uh so some of this I'll try to speed through here just for the sake of time but this one is critical when you're dealing with systems of record with

s60
00:06:23.440 --> 00:06:26.639
lots of complexity engagement ment is critical.

s61
00:06:26.639 --> 00:06:28.800
This is something that we've we've found here repeatedly.

s62
00:06:28.800 --> 00:06:33.280
We have to engage data owners to understand, you know, this tacet knowledge we're trying to to model.

s63
00:06:33.280 --> 00:06:38.080
What's the full meaning of different fields, the structure of the data set, how do we join things together?

s64
00:06:38.080 --> 00:06:45.919
How do we uh understand limitations, systematics of the data, safeguards, security trimmings, uh reporting conventions?

s65
00:06:45.919 --> 00:06:48.720
You know, it's not enough just to answer it a question a certain way.

s66
00:06:48.720 --> 00:06:52.240
You have to answer it the way that it's been answered in the past.

s67
00:06:52.240 --> 00:06:55.199
And so, this is the comes back to the moat here.

s68
00:06:55.199 --> 00:07:09.039
This is the procedural understanding tacet knowledge that AI needs and it's yeah it's the part that we that we own that's you know that's ours that um and that's what we're modeling here.

s69
00:07:09.120 --> 00:07:09.440
Okay.

s70
00:07:09.440 --> 00:07:17.199
So going back here just very quickly for this one this is the a snapshot here of different data curation considerations that we're that go into this pipeline.

s71
00:07:17.199 --> 00:07:18.854
So you have

s72
00:07:18.854 --> 00:07:19.120
[sighs]

s73
00:07:19.120 --> 00:07:31.120
for different data sets whether it's structured unstructured there's different pre-processing filtering dduplication there's an order to different documents there can be uh inconsistencies across documents those need to be uh handled up front

s74
00:07:31.120 --> 00:07:42.479
there's extraction so structured field extraction semantic chunking for unstructured documents if you have figures you need to convert this into text in some way so you can do retrieval across this

s75
00:07:42.479 --> 00:07:49.360
uh various forms of tagging that these can form connections in your graph structure extra metadata that you create during this pipeline.

s76
00:07:49.360 --> 00:07:52.720
Then that becomes different properties in your graph.

s77
00:07:52.720 --> 00:07:54.720
And then the third bucket here, governance.

s78
00:07:54.720 --> 00:08:03.360
This is a important one that I think AI makes more acute things that were that were accessible previously.

s79
00:08:03.360 --> 00:08:06.800
They're much more accessible now with with AI.

s80
00:08:06.800 --> 00:08:08.000
And so you have to consider this.

s81
00:08:08.000 --> 00:08:10.240
Your risk sphere is is larger.

s82
00:08:10.240 --> 00:08:12.319
So things like PII need to be masked.

s83
00:08:12.319 --> 00:08:22.800
you need to reconsider different uh sensitive data classifying this um making sure that there's the right entitlements for each user who's accessing your system.

s84
00:08:23.440 --> 00:08:24.960
Okay, so that's the overview here.

s85
00:08:24.960 --> 00:08:27.919
The this the the data model itself.

s86
00:08:27.919 --> 00:08:30.720
Now, this is the part I'll walk through here.

s87
00:08:30.720 --> 00:08:38.880
There's a a nice animation here, but hopefully the takeaway is you can picture your own your own organization story within what I show here.

s88
00:08:38.880 --> 00:08:48.080
I'll get somewhat technical but it's only to hope hope hopefully to give you an idea of how we how we solved our problem and then you can hopefully uh model this to yours as well.

s89
00:08:48.080 --> 00:08:53.600
Graph is very flexible um practical representation of a physical model.

s90
00:08:53.831 --> 00:08:54.240
[snorts]

s91
00:08:54.240 --> 00:08:54.480
Okay.

s92
00:08:54.480 --> 00:09:02.080
So I'll zoom through a few of these here but the just the the entry point here we have over 80 different strategy teams.

s93
00:09:02.080 --> 00:09:04.320
These teams have annual reviews that happen.

s94
00:09:04.320 --> 00:09:07.040
This is how the budgeting for each year is derived.

s95
00:09:07.040 --> 00:09:08.720
And then so we model this here in the graph.

s96
00:09:08.720 --> 00:09:18.880
The the the meetings are where unstructured documents uh enter into this system from but then there's they have a structured connection to your other systems of record.

s97
00:09:18.880 --> 00:09:21.360
Um what I show here is a conceptual data model.

s98
00:09:21.360 --> 00:09:22.320
So it's flat.

s99
00:09:22.320 --> 00:09:24.720
So you're not seeing the instantiation.

s100
00:09:24.720 --> 00:09:27.040
The actual graph there's you know many different nodes.

s101
00:09:27.040 --> 00:09:32.560
Cardonality is it one one to n. So the actual graph it's you know even more complicated.

s102
00:09:32.560 --> 00:09:37.760
But for the data model itself, let's let me let me show you the first different hier.

s103
00:09:37.760 --> 00:09:43.760
So we have multiple hierarchies that exist within what we've modeled the there's different types of hierarchies you can have.

s104
00:09:43.760 --> 00:09:52.399
In this case, this is a hopefully you can see all this very well, but it's a it's an it's a um additive DAG.

s105
00:09:52.399 --> 00:09:59.040
So there's a all five levels here of this hierarchy from the top to the bottom matter.

s106
00:09:59.040 --> 00:10:01.360
So you have to consider everything together.

s107
00:10:01.360 --> 00:10:07.120
And so then there's different rollup patterns you can do to work across this this sort of pattern.

s108
00:10:07.120 --> 00:10:12.560
In our case, we have a in path shortcut here that connects the funding path.

s109
00:10:12.560 --> 00:10:20.800
Uh funds to bow is where we have the um the budget for each of these different funding teams that that's stored.

s110
00:10:24.160 --> 00:10:28.480
So we have funding what's the the internal funding teams have portfolios.

s111
00:10:28.480 --> 00:10:31.440
These portfolios then go towards different investments.

s112
00:10:31.440 --> 00:10:34.560
Multiple funding teams fund an individual investment.

s113
00:10:34.560 --> 00:10:38.399
So it's a endtoend relationship there.

s114
00:10:38.399 --> 00:10:42.320
The investments are the thing that are our product.

s115
00:10:42.320 --> 00:10:44.399
It's our it's our our business.

s116
00:10:44.399 --> 00:10:48.880
But internally we have funds that then prioritize different different types of investments.

s117
00:10:48.880 --> 00:10:50.480
That's what's shown here.

s118
00:10:50.480 --> 00:10:59.360
And so you can take this down to the transaction level or you can have different uh annualbased aggregations that you map here as well.

s119
00:10:59.592 --> 00:10:59.680
[snorts]

s120
00:10:59.680 --> 00:11:02.560
And then from investment, there's a lot of interesting things you can do.

s121
00:11:02.560 --> 00:11:11.600
You can map to all the different organizations and you can have different types of organizations and there's actually a lot here that is still kind of green space that we want to fill in.

s122
00:11:11.600 --> 00:11:16.240
We have all these different observables that people have produced in the investments that we want to model here.

s123
00:11:16.240 --> 00:11:25.200
So publish reports, products, you know, all this stuff is structured and connects to the entire uh organizational picture.

s124
00:11:26.447 --> 00:11:27.040
[snorts]

s125
00:11:27.040 --> 00:11:28.640
So I mentioned that there's different hierarchies.

s126
00:11:28.640 --> 00:11:30.480
This is the second type of hierarchy.

s127
00:11:30.480 --> 00:11:35.360
At this hierarchy, each level matters in and of itself.

s128
00:11:35.360 --> 00:11:37.279
And so it's not a a DAG necessarily.

s129
00:11:37.279 --> 00:11:44.160
And so you can actually do things like precomputing the the some of these these different shortcuts.

s130
00:11:44.160 --> 00:11:48.880
So the hierarchy it goes from the top to the bottom contains connects.

s131
00:11:48.880 --> 00:11:54.000
It this is showing the investment management side of the of the organization.

s132
00:11:54.000 --> 00:11:56.959
And there's concepts of direct team management.

s133
00:11:56.959 --> 00:12:01.440
So one team at like team level two manages the investment.

s134
00:12:01.440 --> 00:12:03.839
But then there's also a concept of indirect management.

s135
00:12:03.839 --> 00:12:09.920
So the uh children below team level two still should be attributed to the team level two.

s136
00:12:09.920 --> 00:12:15.120
And so there's different things you can different games you can play with these sort of rollups to precomputee.

s137
00:12:15.120 --> 00:12:25.360
I don't know if you can see this, but rollup manages m is a is a a derived edge that we that we create after we create the contains and manage manages

s138
00:12:25.360 --> 00:12:27.360
edge.

s139
00:12:27.839 --> 00:12:30.880
So I've shown two different two different lenses for one investment.

s140
00:12:30.880 --> 00:12:34.160
There's the funding lens, the management lens and you can model both of these here.

s141
00:12:34.160 --> 00:12:40.880
Then within within the graph, a third hierarchy here is people.

s142
00:12:40.880 --> 00:12:49.040
You have organizations, you have org charts, and you have people who are owners, you have people who are attendees of meetings, you have people who are uh directors.

s143
00:12:49.040 --> 00:12:50.560
There's all kinds of different roles they have.

s144
00:12:50.560 --> 00:12:51.920
You can model these here.

s145
00:12:51.920 --> 00:12:57.040
You can have their their uh you know, who they report to, what their uh team structure is.

s146
00:12:57.040 --> 00:13:00.480
And these all are structured data that connects across systems.

s147
00:13:00.480 --> 00:13:08.639
Traditionally, they existed in a just a HR source system, but they're relevant for the context of the the full story.

s148
00:13:11.680 --> 00:13:13.839
And then that leads to this connectedness.

s149
00:13:13.839 --> 00:13:16.880
So we have different source systems that were siloed.

s150
00:13:16.880 --> 00:13:30.959
We to understand the entire picture for the agent to understand correctly across the structure you need to find these common B these common uh these common uh entities that you stitch together.

s151
00:13:30.959 --> 00:13:32.320
And so that's what's shown here.

s152
00:13:32.320 --> 00:13:37.360
These are different source systems but they're related quantity entities that exist there.

s153
00:13:37.360 --> 00:13:45.200
And now the agent can traverse here and understand this pretty complicated organ organizational structure.

s154
00:13:49.839 --> 00:13:53.040
One last part here that I haven't shown yet is the the document part.

s155
00:13:53.040 --> 00:13:56.079
So, and this is still there's there's a lot more we can do to this part.

s156
00:13:56.079 --> 00:14:04.480
We've just been uh ingesting one different document source so far, but this is where you combine unstructured and structured.

s157
00:14:04.480 --> 00:14:08.959
And this gets into part of the magic here that you can model with Neo4j.

s158
00:14:08.959 --> 00:14:12.560
But we have meetings that have documents.

s159
00:14:12.560 --> 00:14:19.120
Documents then have different semantic sections that you can or chunks that you can uh that you can model here.

s160
00:14:19.120 --> 00:14:26.240
You can put full text indexes across these to to aid in the different uh search and retrieval approaches for the agent.

s161
00:14:26.240 --> 00:14:30.079
There could also just be a pure graph retrieval that that the agent does.

s162
00:14:30.079 --> 00:14:36.000
And then all these things then connect back to your your your main organizational structure.

s163
00:14:38.399 --> 00:14:41.120
So then as a whole this is what the data model looks like.

s164
00:14:41.120 --> 00:14:42.720
So I've been zooming in here now.

s165
00:14:42.720 --> 00:14:55.120
You can see the the full interconnectedness of this four different systems one graph uh one semantic layer that's exposed through an MCP then to the to the agents.

s166
00:14:56.720 --> 00:15:04.959
And so this is the so if you think of the agent's perspective, this is the the structure that it can dynamically discover and reason across at query time.

s167
00:15:04.959 --> 00:15:12.720
And for the developer, it's also a very cool thing because it exposes, you know, what you don't know about your your the thing you're modeling.

s168
00:15:12.720 --> 00:15:19.680
You very soon you find out that there's a gap in your understanding or there's some data set that you're not, you know, fully including.

s169
00:15:19.680 --> 00:15:25.040
And so this this process in in of it in and of itself is very valuable.

s170
00:15:26.720 --> 00:15:29.040
Okay, let me give you a sense here what we do with this now.

s171
00:15:29.040 --> 00:15:34.160
So this is the I showed you the platform, the graph, but then how does this relate to AI?

s172
00:15:34.160 --> 00:15:43.920
So we've we've connected this through MCP and I you know I discussed earlier what the what's durable, what's defensible to us.

s173
00:15:43.920 --> 00:15:54.480
What was not defensible was the was the the chat interface was the UI and even in some cases the the general chat cases the um you know the agent interaction

s174
00:15:54.480 --> 00:16:07.680
and so we users themselves are included already or chat GPT and so we serve the platform where they are and so it's served here now through MCP here's an example just kind of a innocuous

s175
00:16:07.680 --> 00:16:15.759
uh question here but Neoforj has some off-the-shelf uh MCP servers Here we've actually modified these quite a bit here.

s176
00:16:15.759 --> 00:16:16.320
We forked it.

s177
00:16:16.320 --> 00:16:19.600
And then there's various updates to the schema.

s178
00:16:19.600 --> 00:16:25.360
Uh things to to pass state back to the to to our system.

s179
00:16:25.360 --> 00:16:34.880
You know, the conversation uh ids, the the message uh numbers, stuff like this we we've we've modified in these MCP tools.

s180
00:16:34.880 --> 00:16:36.480
But so that's a general chat experience.

s181
00:16:36.480 --> 00:16:37.600
That's one entry point.

s182
00:16:37.600 --> 00:16:43.279
The other part that we're building right now too that's very exciting is more constrained workflow experiences.

s183
00:16:43.279 --> 00:16:48.480
And so these can also be offered through things like co-work uh clawed chat.

s184
00:16:48.480 --> 00:17:00.000
And you can do things like um you can have your you can have MCP apps be the the you the standard entry way that users access you know different UIs that are

s185
00:17:00.000 --> 00:17:07.839
uh ported into your your your your chat experience and you can have different sandbox based agents that then run the the workflow.

s186
00:17:07.839 --> 00:17:09.919
And so these are these are active things that we're working on.

s187
00:17:09.919 --> 00:17:20.559
It helps to constrain the experience compared to chat, but it at the same time it pulls from that same knowledge graph-based uh back-end platform.

s188
00:17:23.439 --> 00:17:24.880
Okay, I've got a couple minutes.

s189
00:17:24.880 --> 00:17:34.640
I'll kind of speed through this, but the the way eval relate to data modeling is that as you're doing eval, you find you find gaps.

s190
00:17:34.640 --> 00:17:36.640
You find ambiguities in your data model.

s191
00:17:36.640 --> 00:17:47.760
you find ways in which users are asking questions that uh that are ambiguous or it's um you not it's not returning things that conform with the reporting standards.

s192
00:17:47.760 --> 00:17:50.799
So what we've done here then is we've worked with data owners.

s193
00:17:50.799 --> 00:17:59.200
We've we've uh built targeted uh eval questions that that they that match their reporting standards.

s194
00:17:59.200 --> 00:18:02.320
We've separated these into different complexity tiers.

s195
00:18:02.320 --> 00:18:05.679
One challenge is that the the structured data is constantly changing.

s196
00:18:05.679 --> 00:18:17.280
So we have to have the graph query itself that we that we create for each of these different questions and then at runtime for the eval we we pull from the live graph and then we compare that to what the agent

s197
00:18:17.280 --> 00:18:23.120
is delivering for that question and so that's what's shown here then there's a feedback loop that you can do for this.

s198
00:18:23.120 --> 00:18:32.000
So as you're running an eval pipeline, an eval structure pipeline, you have an LLM as a judge, we've modeled things like pass at one uh stability.

s199
00:18:32.000 --> 00:18:40.000
So if you ask the same question multiple times, you get the same answer back, you can use LLM as a judge to to to measure this.

s200
00:18:40.000 --> 00:18:44.559
And then there's a feedback loop here that you you can update then your your data model.

s201
00:18:44.559 --> 00:18:57.600
you can update your uh your domain rules, your uh schema descriptions to help to help uh fill those those gaps that you that you find.

s202
00:18:58.799 --> 00:19:08.000
Then after you do this, this is a this is just some some eval reporting here that we uh we show the pass at one and the the stability for our system.

s203
00:19:08.000 --> 00:19:10.880
So we we've gotten this very very strong.

s204
00:19:10.880 --> 00:19:15.760
the the questions that we that we end up do missing, it tends to be things that are ambiguous in some way.

s205
00:19:15.760 --> 00:19:22.720
And so it's not wrong, it's just that it's things that might be right but not what the user intended.

s206
00:19:22.720 --> 00:19:27.919
So that that's kind of the constant struggle that we that we that we're working around 30 seconds here.

s207
00:19:27.919 --> 00:19:28.960
What's ahead for SIP?

s208
00:19:28.960 --> 00:19:33.280
So we're continue continuing to fill out our existing uh data from systems of records.

s209
00:19:33.280 --> 00:19:36.000
So things that fit into our current data model.

s210
00:19:36.000 --> 00:19:41.280
We want to expand the primary graph to additional enterprisewide data sets.

s211
00:19:41.280 --> 00:19:45.440
There's a lot of there's a lot of demand for a federated graph experience.

s212
00:19:45.440 --> 00:19:51.280
So, we have a main enterprise system, but we have specific teams that have their own data that they want to link to this.

s213
00:19:51.280 --> 00:19:58.400
And so, we're working on how to do this uh different agentic experiences like I mentioned as well.

s214
00:19:58.400 --> 00:19:59.120
And that's it.

s215
00:19:59.120 --> 00:20:08.480
Uh, so yeah, please if you want to ask questions, if there's things that you want to talk about, I'll be out back or you can add me on LinkedIn here and you know, keep the conversation going.

s216
00:20:08.480 --> 00:20:10.453
Thank you.

s217
00:20:10.453 --> 00:20:12.453
[applause]

s218
00:20:27.484 --> 00:20:29.484
[music]