Thinner Agents on a Smarter Substrate: The Ontology-based Semantic Layer — Emil Eifrem, Neo4j

AI Engineer · 11 min · 143 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

  1. 00:01[music]
  2. 00:12All right.
  3. 00:14At Neo4j, we work with some of the largest companies in the world to help make their data ready for AI agents.
  4. 00:21And today I want to talk to you about a problem that we saw emerging over the last, call it, 6 to 9 months and propose a solution blueprint for that.
  5. 00:31So, let's say that we work at a big organization, a big bank, and we want to write an agent.
  6. 00:36Let's say that agent is helping automate the opening of a bank account.
  7. 00:42Right?
  8. 00:42You can imagine that's very ripe for automation.
  9. 00:45You want to be able to orchestrate that process.
  10. 00:47And I'm going to use the powers bestowed upon me by a short keynote slot to grossly simplify what that agent looks like.
  11. 00:55I'm going to say there's two pieces.
  12. 00:57The first one is, let's call it, the business logic.
  13. 01:00Some version of interpreting intent and plan, act, and we loop around that.
  14. 01:04It's what your agent does.
  15. 01:06And we know that when an agent act, it doesn't always operate on data, but we equally know that in order for agents to be successful, a huge part of that is giving it access
  16. 01:15to the right data at the right time.
  17. 01:18So, the second big bucket is, let's call it, the data sources.
  18. 01:22Need to identify, figure out, okay, in order to solve my problem, I need access to these few things, and wire them up and make them available to the agent.
  19. 01:31In the example of our account opening agent, maybe we can imagine that we need to be able to validate identity.
  20. 01:37And so, we might look at two data sources for that, the Department of Motor Vehicles, the DMV registry, and maybe some kind of passport verification service.
  21. 01:46So, we wire that up into our agent, and it works.
  22. 01:49It's great.
  23. 01:49It's fantastic.
  24. 01:50And at the same time, you and other teams in your organization are building other agents, and conceptually, they look very similar.
  25. 01:58That's great.
  26. 01:59It's fantastic.
  27. 02:00It works.
  28. 02:02But, it has a few problems.
  29. 02:03So, first of all, every single time a team has to build an agent, they have to figure out from scratch where the data that they require for that agent to operate, where it sits,
  30. 02:13which if you work at a startup and you have one application, it sits on top of one Postgres database, that's not hard.
  31. 02:20The data is in that Postgres database.
  32. 02:22But, in an enterprise ecosystem, you don't have one database, you have a hundred databases, and you have Snowflake and Databricks, probably, and you have S3 buckets, and so on and so forth.
  33. 02:32You have to do that work manually from scratch every single time.
  34. 02:36And then, when you found the data sources, you know, in an enterprise, there's lots of duplication of data.
  35. 02:41So, then you need to figure out like, is this the right data?
  36. 02:43Is it the right version?
  37. 02:44Can I trust it?
  38. 02:45Am I allowed to access it?
  39. 02:47So on and so forth.
  40. 02:49It also violates one of the core principles of software engineering, the DRY principle, don't repeat yourself.
  41. 02:55So, when something change, that cascades across all of your agents.
  42. 02:59You have to kind of manually rewire all of them all the time, which works, but it's just a a lot of work.
  43. 03:06And then, finally, there's no learning around the data sources and how your agents operate on them.
  44. 03:11So, when your agent wake up wakes up tomorrow, it's not smarter than it was today, and there certainly isn't any cross-agent learning because all of that wiring between business intent and the data sources is encoded in a combination of code
  45. 03:24and prompts.
  46. 03:27So, I know what you're all thinking.
  47. 03:28Markdown files, skills to the rescue.
  48. 03:31And yes and no. Um you can come talk to me afterwards for kind of the full version of this, but we've seen a ton of team that tried to solve this problem
  49. 03:40using just Markdown files.
  50. 03:41And the summary is it is part of the solution, but it is not the solution.
  51. 03:47Uh but, don't take it from me, take it from Swyx.
  52. 03:50A week ago on the Latent Space Spot podcast, I said, "Hey guys, you got to learn your databases.
  53. 03:55You cannot vibe code with just markdown files."
  54. 04:00So, we've been solving this problem at scale for some really massive organizations recently, including a Fortune 20 global bank, a massive tech platform company based here in the Bay Area, and a leading fintech company.
  55. 04:16And the pattern that is emerging is that in order to do things at scale, we need thin agents on a smarter shared substrate.
  56. 04:27Thin agents on a smarter shared substrate.
  57. 04:31And what does that look like in practice?
  58. 04:32There are three pillars to that.
  59. 04:34The first pillar is a business-facing ontology.
  60. 04:37And the word ontology, like I grew up in this world, people talked about ontologies forever.
  61. 04:43More recently, it's become very hype, probably thanks to Palantir, but also the rise of AI.
  62. 04:48And there's a lot of people who want to make ontologies really complex.
  63. 04:52But the core concepts are actually super simple.
  64. 04:54What are the key concepts in your organization?
  65. 04:57In our banking example, customers, accounts, um debit cards, checks, transactions, and how do they all relate?
  66. 05:07But very importantly, they are expressed in a way that makes sense to all the human beings working in your universe, right?
  67. 05:14All the people working in your company, it's expressed in that name, in that way.
  68. 05:18In other words, you don't say if underscore name.
  69. 05:22No, you have a customer and they have a first name.
  70. 05:25So, that's the first, a business-facing ontology.
  71. 05:27The second pillar is a technical ontology.
  72. 05:30This is all the metadata of all the data sources and data assets in your enterprise ecosystem.
  73. 05:36I have 14 Oracle databases, I have 15 Neo4j databases, I have Snowflake and Databricks and I have S3 buckets and all all that kind of stuff.
  74. 05:46Where do they sit?
  75. 05:47What are the schemas?
  76. 05:48All of that kind of good stuff.
  77. 05:49You can You construct that technical ontology in three key ways that we can talk about later, though not in this in this talk.
  78. 05:57And then you have a mapping between the two.
  79. 05:58So that customer that has a first name, that first name has a system of record and over there there's an Oracle database with a column called F_name.
  80. 06:08The mapping between the two.
  81. 06:10And then the third pillar is the run time signals out of your agents.
  82. 06:14When they walk this graph and they execute, they leave the traces around.
  83. 06:20What have I tried?
  84. 06:21Was I successful?
  85. 06:22What was the outcome?
  86. 06:24The execution traces.
  87. 06:25Those three pillars.
  88. 06:27Okay, so let's look at that in the context of our bank account opening agent.
  89. 06:32This is a simplified view, but you can see this graph here.
  90. 06:34It has a combination of business concepts like checks and accounts and credit history and stuff like that.
  91. 06:40This is a process following agent or a process guided agent.
  92. 06:45We want this type of agent to actually follow a process.
  93. 06:48We've also encoded that in the ontology, a business process.
  94. 06:52And then if you look at the node that is surrounded by green, the check compliance one, we flip to the technical ontology and we've put in the graph here.
  95. 07:02We've discovered and encoded that in order to do a compliance check, you might imagine that you need to resolve a government issued ID.
  96. 07:09And then we say that in this particular organization, there are two data sources that can help us with that.
  97. 07:15It's the motor vehicle records and the passport verification one.
  98. 07:18Which is that's really great.
  99. 07:20So then when our agents come in here and they realize I'm going to check compliance, I need a government issued ID.
  100. 07:26Here are the two ways that I can resolve that.
  101. 07:29When they execute and they try that, they leave the third pillar, the execution traces for that.
  102. 07:35And they're more sophisticated than what's on this simplified slide, but involves things like, okay, where was I?
  103. 07:41What did I do?
  104. 07:42What is my context?
  105. 07:44And was I successful?
  106. 07:45And ultimately it leads out to some kind of a score.
  107. 07:48And you use that as input.
  108. 07:50It's like, okay, I've been very successful using the DMV lookup, for example, then I'm more likely to choose one if I'm in the right context in my next invocation.
  109. 08:01Three pillars of the ontology-based semantic layer, a business ontology, a technical ontology, the execution traces taken together, they solve all four of the problems.
  110. 08:11We now have a very easy way to discover the data sources.
  111. 08:14We know if they're trustworthy or not.
  112. 08:16We know that top-down by some kind of human curated knowledge, right?
  113. 08:20An administrator of some sort saying it.
  114. 08:22We also know it bottom-up through the execution traces.
  115. 08:25This is what actually worked in reality, in practice.
  116. 08:29We have a single governed place that maps business intent and the concepts to those data sources.
  117. 08:34So, we don't repeat ourselves.
  118. 08:35If something changes, that cascades across all my agents, right?
  119. 08:40And we have self-learning.
  120. 08:42So, my agent that wakes up tomorrow is slightly smarter than it was today.
  121. 08:46And not just self-learning on an individual agent, but across agents as well.
  122. 08:52So, we're moving from this world, a world of thick agents with manually wired data sources, into this world where we have thin agents on a smarter shared ontology-based semantic
  123. 09:06layer.
  124. 09:06And this allows us to do a ton more agents without having to re-engineer them every time.
  125. 09:15Thin agents on top of a smarter shared substrate.
  126. 09:20If you think this is interesting, there's a documentation webpage that outlines more information about this.
  127. 09:26If you see the QR code here, you can also come and talk to us at the booth.
  128. 09:30We have a big booth here at the Expo, P3.
  129. 09:33We love talking about this this kind of stuff.
  130. 09:35But not just that, this is one pattern, a very exciting pattern that we see a lot of traction around right now for using graphs in AI.
  131. 09:44But there's hundreds of more interesting patterns that combines graphs and AI.
  132. 09:4910 of them is actually in the graph track that is kicking off right now in room 2005.
  133. 09:56And you have some really amazing talks from organizations like the Gates Foundation, monday.com, JP Morgan Chase, Berkeley, New York Times, and so on and so forth.
  134. 10:08So go check out that thing.
  135. 10:10And then finally, this was primarily centered around organizations where you deal with many data sources and many agents.
  136. 10:18But if you're a startup building on Neo4j, love you.
  137. 10:22There is a startup program for Neo4j that is phenomenal.
  138. 10:25You get access to free credit, but more importantly, we've built up a dedicated solution engineering team that spent every day working with startups for free, helping them model their data in Neo4j,
  139. 10:37tune it for performance, and so on and so forth.
  140. 10:40So please sign up for our startup program.
  141. 10:43Thank you very much.
  142. 10:44Enjoy the conference.
  143. 10:46Have a good day, everyone.