Thinner Agents on a Smarter Substrate: The Ontology-based Semantic Layer — Emil Eifrem, Neo4j
https://www.youtube.com/watch?v=VGN22pPpb-8
AI Engineer · 11 min · 143 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] All right.
[00:14] At Neo4j, we work with some of the largest companies in the world to help make their data ready for AI agents.
[00:21] And today I want to talk to you about a problem that we saw emerging over the last, call it, 6 to 9 months and propose a solution blueprint for that.
[00:31] So, let's say that we work at a big organization, a big bank, and we want to write an agent.
[00:36] Let's say that agent is helping automate the opening of a bank account.
[00:42] Right?
[00:42] You can imagine that's very ripe for automation.
[00:45] You want to be able to orchestrate that process.
[00:47] And I'm going to use the powers bestowed upon me by a short keynote slot to grossly simplify what that agent looks like.
[00:55] I'm going to say there's two pieces.
[00:57] The first one is, let's call it, the business logic.
[01:00] Some version of interpreting intent and plan, act, and we loop around that.
[01:04] It's what your agent does.
[01:06] And we know that when an agent act, it doesn't always operate on data, but we equally know that in order for agents to be successful, a huge part of that is giving it access
[01:15] to the right data at the right time.
[01:18] So, the second big bucket is, let's call it, the data sources.
[01:22] Need to identify, figure out, okay, in order to solve my problem, I need access to these few things, and wire them up and make them available to the agent.
[01:31] In the example of our account opening agent, maybe we can imagine that we need to be able to validate identity.
[01:37] And so, we might look at two data sources for that, the Department of Motor Vehicles, the DMV registry, and maybe some kind of passport verification service.
[01:46] So, we wire that up into our agent, and it works.
[01:49] It's great.
[01:49] It's fantastic.
[01:50] And at the same time, you and other teams in your organization are building other agents, and conceptually, they look very similar.
[01:58] That's great.
[01:59] It's fantastic.
[02:00] It works.
[02:02] But, it has a few problems.
[02:03] So, first of all, every single time a team has to build an agent, they have to figure out from scratch where the data that they require for that agent to operate, where it sits,
[02:13] which if you work at a startup and you have one application, it sits on top of one Postgres database, that's not hard.
[02:20] The data is in that Postgres database.
[02:22] But, in an enterprise ecosystem, you don't have one database, you have a hundred databases, and you have Snowflake and Databricks, probably, and you have S3 buckets, and so on and so forth.
[02:32] You have to do that work manually from scratch every single time.
[02:36] And then, when you found the data sources, you know, in an enterprise, there's lots of duplication of data.
[02:41] So, then you need to figure out like, is this the right data?
[02:43] Is it the right version?
[02:44] Can I trust it?
[02:45] Am I allowed to access it?
[02:47] So on and so forth.
[02:49] It also violates one of the core principles of software engineering, the DRY principle, don't repeat yourself.
[02:55] So, when something change, that cascades across all of your agents.
[02:59] You have to kind of manually rewire all of them all the time, which works, but it's just a a lot of work.
[03:06] And then, finally, there's no learning around the data sources and how your agents operate on them.
[03:11] So, when your agent wake up wakes up tomorrow, it's not smarter than it was today, and there certainly isn't any cross-agent learning because all of that wiring between business intent and the data sources is encoded in a combination of code
[03:24] and prompts.
[03:27] So, I know what you're all thinking.
[03:28] Markdown files, skills to the rescue.
[03:31] And yes and no. Um you can come talk to me afterwards for kind of the full version of this, but we've seen a ton of team that tried to solve this problem
[03:40] using just Markdown files.
[03:41] And the summary is it is part of the solution, but it is not the solution.
[03:47] Uh but, don't take it from me, take it from Swyx.
[03:50] A week ago on the Latent Space Spot podcast, I said, "Hey guys, you got to learn your databases.
[03:55] You cannot vibe code with just markdown files."
[04:00] So, we've been solving this problem at scale for some really massive organizations recently, including a Fortune 20 global bank, a massive tech platform company based here in the Bay Area, and a leading fintech company.
[04:16] And the pattern that is emerging is that in order to do things at scale, we need thin agents on a smarter shared substrate.
[04:27] Thin agents on a smarter shared substrate.
[04:31] And what does that look like in practice?
[04:32] There are three pillars to that.
[04:34] The first pillar is a business-facing ontology.
[04:37] And the word ontology, like I grew up in this world, people talked about ontologies forever.
[04:43] More recently, it's become very hype, probably thanks to Palantir, but also the rise of AI.
[04:48] And there's a lot of people who want to make ontologies really complex.
[04:52] But the core concepts are actually super simple.
[04:54] What are the key concepts in your organization?
[04:57] In our banking example, customers, accounts, um debit cards, checks, transactions, and how do they all relate?
[05:07] But very importantly, they are expressed in a way that makes sense to all the human beings working in your universe, right?
[05:14] All the people working in your company, it's expressed in that name, in that way.
[05:18] In other words, you don't say if underscore name.
[05:22] No, you have a customer and they have a first name.
[05:25] So, that's the first, a business-facing ontology.
[05:27] The second pillar is a technical ontology.
[05:30] This is all the metadata of all the data sources and data assets in your enterprise ecosystem.
[05:36] I have 14 Oracle databases, I have 15 Neo4j databases, I have Snowflake and Databricks and I have S3 buckets and all all that kind of stuff.
[05:46] Where do they sit?
[05:47] What are the schemas?
[05:48] All of that kind of good stuff.
[05:49] You can You construct that technical ontology in three key ways that we can talk about later, though not in this in this talk.
[05:57] And then you have a mapping between the two.
[05:58] So that customer that has a first name, that first name has a system of record and over there there's an Oracle database with a column called F_name.
[06:08] The mapping between the two.
[06:10] And then the third pillar is the run time signals out of your agents.
[06:14] When they walk this graph and they execute, they leave the traces around.
[06:20] What have I tried?
[06:21] Was I successful?
[06:22] What was the outcome?
[06:24] The execution traces.
[06:25] Those three pillars.
[06:27] Okay, so let's look at that in the context of our bank account opening agent.
[06:32] This is a simplified view, but you can see this graph here.
[06:34] It has a combination of business concepts like checks and accounts and credit history and stuff like that.
[06:40] This is a process following agent or a process guided agent.
[06:45] We want this type of agent to actually follow a process.
[06:48] We've also encoded that in the ontology, a business process.
[06:52] And then if you look at the node that is surrounded by green, the check compliance one, we flip to the technical ontology and we've put in the graph here.
[07:02] We've discovered and encoded that in order to do a compliance check, you might imagine that you need to resolve a government issued ID.
[07:09] And then we say that in this particular organization, there are two data sources that can help us with that.
[07:15] It's the motor vehicle records and the passport verification one.
[07:18] Which is that's really great.
[07:20] So then when our agents come in here and they realize I'm going to check compliance, I need a government issued ID.
[07:26] Here are the two ways that I can resolve that.
[07:29] When they execute and they try that, they leave the third pillar, the execution traces for that.
[07:35] And they're more sophisticated than what's on this simplified slide, but involves things like, okay, where was I?
[07:41] What did I do?
[07:42] What is my context?
[07:44] And was I successful?
[07:45] And ultimately it leads out to some kind of a score.
[07:48] And you use that as input.
[07:50] It's like, okay, I've been very successful using the DMV lookup, for example, then I'm more likely to choose one if I'm in the right context in my next invocation.
[08:01] Three pillars of the ontology-based semantic layer, a business ontology, a technical ontology, the execution traces taken together, they solve all four of the problems.
[08:11] We now have a very easy way to discover the data sources.
[08:14] We know if they're trustworthy or not.
[08:16] We know that top-down by some kind of human curated knowledge, right?
[08:20] An administrator of some sort saying it.
[08:22] We also know it bottom-up through the execution traces.
[08:25] This is what actually worked in reality, in practice.
[08:29] We have a single governed place that maps business intent and the concepts to those data sources.
[08:34] So, we don't repeat ourselves.
[08:35] If something changes, that cascades across all my agents, right?
[08:40] And we have self-learning.
[08:42] So, my agent that wakes up tomorrow is slightly smarter than it was today.
[08:46] And not just self-learning on an individual agent, but across agents as well.
[08:52] So, we're moving from this world, a world of thick agents with manually wired data sources, into this world where we have thin agents on a smarter shared ontology-based semantic
[09:06] layer.
[09:06] And this allows us to do a ton more agents without having to re-engineer them every time.
[09:15] Thin agents on top of a smarter shared substrate.
[09:20] If you think this is interesting, there's a documentation webpage that outlines more information about this.
[09:26] If you see the QR code here, you can also come and talk to us at the booth.
[09:30] We have a big booth here at the Expo, P3.
[09:33] We love talking about this this kind of stuff.
[09:35] But not just that, this is one pattern, a very exciting pattern that we see a lot of traction around right now for using graphs in AI.
[09:44] But there's hundreds of more interesting patterns that combines graphs and AI.
[09:49] 10 of them is actually in the graph track that is kicking off right now in room 2005.
[09:56] And you have some really amazing talks from organizations like the Gates Foundation, monday.com, JP Morgan Chase, Berkeley, New York Times, and so on and so forth.
[10:08] So go check out that thing.
[10:10] And then finally, this was primarily centered around organizations where you deal with many data sources and many agents.
[10:18] But if you're a startup building on Neo4j, love you.
[10:22] There is a startup program for Neo4j that is phenomenal.
[10:25] You get access to free credit, but more importantly, we've built up a dedicated solution engineering team that spent every day working with startups for free, helping them model their data in Neo4j,
[10:37] tune it for performance, and so on and so forth.
[10:40] So please sign up for our startup program.
[10:43] Thank you very much.
[10:44] Enjoy the conference.
[10:46] Have a good day, everyone.