WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=IN-rb-9WmiY NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/IN-rb-9WmiY.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:13.679 --> 00:00:17.679 All right, welcome everyone. s3 00:00:18.160 --> 00:00:26.400 I want to tell you a little bit about uh the new the new uh knowledge layer and I'll start with telling you a little bit about theory of mind. s4 00:00:26.400 --> 00:00:31.039 Uh how many of you uh know what theory of mind is? s5 00:00:31.039 --> 00:00:33.120 All right, so a few folks. s6 00:00:33.120 --> 00:00:42.800 It's a uh term in psychology about uh our capacity to understand uh and have a mental state or a model of a mental state for other people. s7 00:00:42.800 --> 00:00:43.440 What do they know? s8 00:00:43.440 --> 00:00:44.320 What do they believe in? s9 00:00:44.320 --> 00:00:46.079 How do they act? s10 00:00:46.079 --> 00:00:48.000 That's what allows us to socialize. s11 00:00:48.000 --> 00:00:50.960 That's what allows us to lie to each other and so on. s12 00:00:50.960 --> 00:00:55.600 Uh but it's a crucial developmental stage for humans. s13 00:00:55.600 --> 00:00:57.039 uh in a necessary function. s14 00:00:57.039 --> 00:01:02.800 I'll I'll sort of explain it in the best possible way with a funny story just to break the ice. s15 00:01:02.800 --> 00:01:13.280 Uh about 15 16 years ago I uh worked at a company called Yahoo um that had a product called Yahoo Answers which was sort of like a Q&A forum. s16 00:01:13.280 --> 00:01:22.400 Uh we used then AI which is the stone age of AI you know it's like uh NLP and other statistical models to answer questions. s17 00:01:22.400 --> 00:01:28.960 But the funny thing is people would come to uh Yahoo and ask questions like am I fat? s18 00:01:28.960 --> 00:01:38.000 Which uh is funny because obviously uh you know Q&A forum would wouldn't have any clue whether you're you're too fat or not. s19 00:01:38.000 --> 00:01:48.399 Uh but this is a funny way to say yeah people have a very bad theory of mind on sort of what the products that they use what information they have and what they don't have. s20 00:01:48.399 --> 00:02:01.200 And I want to argue that in some sense we are making that mistake with AI today in a more subtle and less sort of like uh silly way but in a just as fundamental of a mismatch. s21 00:02:01.200 --> 00:02:05.920 And so I will categorize roughly three kinds of knowledge. s22 00:02:05.920 --> 00:02:12.480 I think there there are many more but you know in broad stro in broad broad strokes this is not a bad way to categorize. s23 00:02:12.480 --> 00:02:15.520 First of is general knowledge. s24 00:02:15.520 --> 00:02:18.879 Think about that is what's available in the public domain. s25 00:02:18.879 --> 00:02:23.599 Things that models are trained on and we expect an LLM to just know out the gate. s26 00:02:23.599 --> 00:02:36.080 Uh things about just you know improving your Rust code uh asking you like you know explaining stuff about the legal terms or uh sort of exploring use cases for for products. s27 00:02:36.080 --> 00:02:38.720 Um we know how to do that. s28 00:02:38.720 --> 00:02:40.000 Models are really good at that. s29 00:02:40.000 --> 00:02:42.720 When we use our agents they have that information. s30 00:02:42.720 --> 00:03:00.640 there is specific knowledge [clears throat] the companies of course the models wouldn't have uh any clue about whether we've uh implemented something specific in our codebase what a contract of a specific customer has of course this is information that's private to the company s31 00:03:00.640 --> 00:03:13.920 uh but we know how to deal with that and rag and search and of course pine cone veto dbs and so on uh solved these problems a long time ago there's still a lot to go but we by and large know how to do it s32 00:03:13.920 --> 00:03:16.400 and then I would say there is tribal knowledge. s33 00:03:16.400 --> 00:03:21.440 Okay, tribal knowledge is the sort of general knowledge inside the company. s34 00:03:21.440 --> 00:03:28.800 This is the kind of information that uh an employ a seasoned employee has that a new hireer does not. s35 00:03:28.800 --> 00:03:39.920 Okay, it's roughly how we do things, what our processes are, who's in charge of what, uh what are the guiding principles, what are the you know sort of how we get stuff done in this company. s36 00:03:39.920 --> 00:03:41.200 Okay. s37 00:03:41.200 --> 00:03:50.159 um that is not a that's not general knowledge because it's it's specific to your company but it's also not a specific piece of knowledge because you can't just get it by searching. s38 00:03:50.159 --> 00:03:56.640 You can't just get it by plucking it off one place and say yes this document says uh that this is our culture. s39 00:03:56.640 --> 00:04:00.319 It's the sum total of a lot of uh things. s40 00:04:00.319 --> 00:04:08.959 Um and so if our theory of mind of most of our agents in big companies today should be something like a new hire. s41 00:04:08.959 --> 00:04:09.840 They're brilliant. s42 00:04:09.840 --> 00:04:10.720 They have tools. s43 00:04:10.720 --> 00:04:13.439 They have we have very high expectations of them. s44 00:04:13.439 --> 00:04:15.680 But they are by and large clueless. s45 00:04:15.680 --> 00:04:16.239 Okay. s46 00:04:16.239 --> 00:04:21.600 They have no idea about your company's goals, culture, priorities, processes. s47 00:04:21.600 --> 00:04:30.880 Um they have to read a bunch of documents uh to be able to get any semblance of understanding of where they are in the world. s48 00:04:30.880 --> 00:04:36.240 Um and really they start most tasks from scratch. s49 00:04:36.240 --> 00:04:36.560 Okay. s50 00:04:36.560 --> 00:04:46.720 they really start kind of wake up and every time you start your your session, uh, they're like a new hireer on their first day on the job. s51 00:04:47.040 --> 00:04:57.199 So, I would argue that what they need is not necessarily to be smarter, to have better tools, but to have something new, which I would call a knowledge layer going forward. s52 00:04:57.199 --> 00:05:02.479 Um, and I have to explain to you what I think a knowledge layer has to actually do. s53 00:05:02.479 --> 00:05:06.720 But just I'll define it generally before explain to you what we do. s54 00:05:06.720 --> 00:05:11.520 Um so at the very least it needs to be persistent. s55 00:05:11.520 --> 00:05:16.960 We talked about this is the company knowledge, processes, capabilities and goals and so on. s56 00:05:16.960 --> 00:05:18.960 It has to be persistent. s57 00:05:18.960 --> 00:05:24.400 Uh it sure it uh feels like putting this thing together is going to be a monumental effort. s58 00:05:24.400 --> 00:05:27.520 So you want to do it once or at least very rarely. s59 00:05:27.520 --> 00:05:32.080 Uh and of course all your agents in company need to have access to this thing. s60 00:05:32.080 --> 00:05:38.160 The second thing is that you would argue that this [clears throat] knowledge layer needs to be uh highly specialized. s61 00:05:38.160 --> 00:05:56.240 If your agent uh is in charge of HR practices or optimizing your uh your kernels in your codebase or uh resolving customer issues that they would care about different things they would want to you know address they would want to have different information available to them. s62 00:05:56.240 --> 00:06:10.319 Um, and I would argue irrelevant of of specifically how it's created, I would argue that it's a good practice to have the domain experts in the company own those and have being able to interact with them and make sure that they actually s63 00:06:10.319 --> 00:06:14.560 uh are aligned with what the company wants to do. s64 00:06:14.560 --> 00:06:26.080 And finally, which is the hardest thing is they need to stay up to date and they need to stay concurrent constantly because companies change and you need to make information s65 00:06:26.080 --> 00:06:28.000 uh uh deprecated. s66 00:06:28.000 --> 00:06:36.479 When it becomes deprecated, if the company replaces its CEO, the very next day, everybody in the company knows who the CEO is. s67 00:06:36.479 --> 00:06:36.880 Okay? s68 00:06:36.880 --> 00:06:45.840 Even though 9,000 documents going from, you know, a day before yesterday to uh the last 10 years say it's somebody else. s69 00:06:45.840 --> 00:06:49.440 This has become common tribal knowledge immediately. s70 00:06:49.440 --> 00:06:54.880 And everything else, even you know, every rag search will tell you otherwise. s71 00:06:54.880 --> 00:06:58.639 This is now the overriding fact. s72 00:06:58.639 --> 00:07:00.639 Okay. s73 00:07:00.720 --> 00:07:09.440 Um I will argue before we move on that I'm not suggesting that this knowledge layer should replace everything else we're doing. s74 00:07:09.440 --> 00:07:14.080 Our agents are already very capable and should keep being very capable. s75 00:07:14.080 --> 00:07:21.919 Uh they have access to LLMs of course tooling local files search of course like rag vector search text search and so on. s76 00:07:21.919 --> 00:07:32.560 And I would argue that uh the knowledge layer needs to be a separate entity that all these agents have access to as as I explained why this is sort of a fundamentally s77 00:07:32.560 --> 00:07:35.120 different kind of data source. s78 00:07:35.120 --> 00:07:38.160 So I want to tell you more about Nexus. s79 00:07:38.160 --> 00:07:49.840 Um um Nexus is a product that we're uh uh of course it's a knowledge layer but I need I want to explain to you what it does. s80 00:07:49.840 --> 00:08:08.960 uh of course so the the four parts are roughly connectors what I would call context uh uh tasks and queries I'll explain each one of them uh now so connectors of course do the obvious things there's nothing much to talk about them s81 00:08:08.960 --> 00:08:24.319 um uh they bring data into the platform uh respect access uh rights and uh and control and so on um very dull as a topic of discussion but incredibly important in enterprises, s82 00:08:24.319 --> 00:08:26.319 incredibly important to do right. s83 00:08:26.319 --> 00:08:32.959 Uh and everybody who's built that knows how painstaking it is to get this thing to work well. s84 00:08:32.959 --> 00:08:41.519 Um the interesting sort of the first properly interesting part is um the context. s85 00:08:41.519 --> 00:08:53.600 So the context think about that as the um the sum of assets that the that Nexus or this knowledge layer saves about a specific topic. s86 00:08:53.600 --> 00:09:00.240 Think about this as a context for HR, for operations, for engineering and so on. s87 00:09:00.240 --> 00:09:05.040 Okay, of course it has the sources, the connections to the data and so on. s88 00:09:05.040 --> 00:09:07.680 But then it has the manifest. s89 00:09:07.680 --> 00:09:13.839 Okay, the manifest is the first truly novel idea and and a important uh entity. s90 00:09:13.839 --> 00:09:27.760 Okay, the way that we today give context to our agents is with skills and plugins and uh commands and sort of markdown files we write ourselves. s91 00:09:27.760 --> 00:09:30.240 In a knowledge layer, you don't do that anymore. s92 00:09:30.240 --> 00:09:30.959 Okay? s93 00:09:30.959 --> 00:09:36.480 You tell the you tell the knowledge layer, you tell Nexus what you care about in a manifest. s94 00:09:36.480 --> 00:09:49.360 You say what kind of tasks you're you want to complete, what entities you want to track, what kind of information you care about, and it's it's the knowledge layer's work to keep track of those files. s95 00:09:49.360 --> 00:09:49.600 Okay? s96 00:09:49.600 --> 00:09:52.320 And and organize its own data. s97 00:09:52.320 --> 00:09:56.560 Um uh and then it has the knowledge files themselves. s98 00:09:56.560 --> 00:10:02.480 So think about the manifest is the meta knowledge file or set of files. s99 00:10:02.480 --> 00:10:05.600 Um and then there's the knowledge the knowledge itself. s100 00:10:05.600 --> 00:10:10.720 uh think about this as five types of contents. s101 00:10:10.720 --> 00:10:13.760 One of them the the first kind is a semantic map. s102 00:10:13.760 --> 00:10:27.920 Okay, this is a file that organizes the catalog of where everything else is, the key schemas, the glosseries, all the information that you need to be able to operate this context. s103 00:10:27.920 --> 00:10:36.959 Then there are of course a bunch of markdown files that contain information, decisions, memories, uh all sorts of unstructured information. s104 00:10:36.959 --> 00:10:55.040 If you've uh who here has heard about uh LLM wiki okay so LLM wiki is this uh idea that Andre Karpathy suggested to have agents basically manage some stash of markdown files to sort of keep their own notes of stuff. s105 00:10:55.040 --> 00:10:57.360 This is a very similar idea. s106 00:10:57.360 --> 00:11:03.600 Okay, sort of on steroids but similar concept but that's not enough. s107 00:11:03.600 --> 00:11:10.720 Context contains also a bunch of SQL tables for tabular data for aggregation for facts that they need to memorize. s108 00:11:10.720 --> 00:11:21.519 uh a vector database to be able to semantically search and text search and filter a bunch of information and all the information that it itself decides is necessary to keep and and remember s109 00:11:21.519 --> 00:11:30.399 and graph entities and graphics to remember uh again entities relationships uh causal chains and so on. s110 00:11:30.399 --> 00:11:31.360 Okay. s111 00:11:31.360 --> 00:11:35.680 And all of that is managed and created for you. s112 00:11:35.680 --> 00:11:36.959 You don't touch any of this. s113 00:11:36.959 --> 00:11:40.000 You don't have to uh know how any of this is created. s114 00:11:40.000 --> 00:11:44.079 It's all maintained through the manifest. s115 00:11:44.079 --> 00:11:46.480 So these are this is the context. s116 00:11:46.480 --> 00:11:47.440 Now what are the tasks? s117 00:11:47.440 --> 00:11:48.000 What are the actions? s118 00:11:48.000 --> 00:11:48.800 These are the nouns. s119 00:11:48.800 --> 00:11:50.399 What are what are the verbs? s120 00:11:50.399 --> 00:11:53.120 What can we do with those uh objects? s121 00:11:53.120 --> 00:12:02.160 Uh first of all is of course import which is bring new information to the system, new files, new events, new uh concepts. s122 00:12:02.160 --> 00:12:09.440 Um curate is probably the most and most elaborate one of the two most elaborate functions. s123 00:12:09.440 --> 00:12:20.480 it's uh job is to take um to take the uh the say call it a file it doesn't have to be a file but let's say it's a new PDF or a new meeting transcription s124 00:12:20.480 --> 00:12:30.160 or a new uh presentation for the board and say now I need to update my view of the world okay so it takes that it takes the manifest which tells okay now s125 00:12:30.160 --> 00:12:41.120 what what things do I care about what concepts do I need to track what needs to be uh uh what do I need to memorize here and then it takes the s126 00:12:41.120 --> 00:12:52.880 semantic map that I told you contains all the like organizes where information is and understands okay fine I need to go update these tables these relationships my graph uh this s127 00:12:52.880 --> 00:13:06.240 uh these text indexes and so on um and when it's done it has essentially said okay fine I have assimulated the new information it's already it's it's sort of persisted in your context s128 00:13:06.240 --> 00:13:13.120 and we can move on Okay, search is sort of a internal function very important. s129 00:13:13.120 --> 00:13:27.040 It basically abstracts with simple functions all the access to that information so that the agent that I'll tell you in a second doesn't have to know uh how to sort of access the internal parts of the context. s130 00:13:27.040 --> 00:13:38.320 Now comes uh the second I think most uh most uh interesting uh part of of the system which is noql. s131 00:13:38.320 --> 00:13:45.519 Noql is our way to to uh specify how we want information out of the context. s132 00:13:45.519 --> 00:13:47.440 It's not enough to just ask questions. s133 00:13:47.440 --> 00:13:52.480 If you're an agent, it's really important for you to give us the budget in tokens or in dollars. s134 00:13:52.480 --> 00:14:01.600 how much time you want us to invest in this sort of like do you want us to just a quick off-the- cuff kind of immediate answer or do you want us to go full tilt s135 00:14:01.600 --> 00:14:09.040 and scour everything we know to make sure we give you the most accurate answer uh and so on the structure and and so on. s136 00:14:09.040 --> 00:14:11.120 It's a you know it's an agentic tool. s137 00:14:11.120 --> 00:14:13.760 This is not like a chat interface. s138 00:14:13.760 --> 00:14:15.760 Okay. s139 00:14:15.920 --> 00:14:21.680 Um and of course what you get back is like something that's agent friendly. s140 00:14:21.680 --> 00:14:28.079 This is uh you know grounded text and and kind of structured in the right way format. s141 00:14:28.079 --> 00:14:31.600 This again is not designed to be like a chat interface. s142 00:14:31.600 --> 00:14:33.760 This is really what your agents expect. s143 00:14:33.760 --> 00:14:35.279 Okay. s144 00:14:35.279 --> 00:14:46.880 Um so this is this is a uh um this is a uh trying to figure out how much time I have. s145 00:14:46.880 --> 00:14:48.720 Okay, I need to go a little bit faster. s146 00:14:48.720 --> 00:14:52.720 So uh this is a sophisticated crowd. s147 00:14:52.720 --> 00:14:57.279 I want to sort of uh pop the hood and show you a little bit about how things work. s148 00:14:57.279 --> 00:15:09.199 So uh if anything this is the interface I want to there are many many cool ideas uh that make this work but I want to tell you a little bit about just one of them. s149 00:15:09.199 --> 00:15:10.880 Okay. s150 00:15:10.880 --> 00:15:13.440 What you're seeing is the interface for Nexus. s151 00:15:13.440 --> 00:15:14.000 Okay. s152 00:15:14.000 --> 00:15:24.160 You will see at the top a question that the uh the agent issued and you see you see on the left uh the different steps that it went through. s153 00:15:24.160 --> 00:15:25.360 Okay. s154 00:15:25.360 --> 00:15:28.720 What you see on the bottom right is probably the most interesting. s155 00:15:28.720 --> 00:15:31.920 You see generated code on the bottom right. s156 00:15:31.920 --> 00:15:35.920 What Nexus does which is very different than other systems. s157 00:15:35.920 --> 00:15:38.959 It's what we call a runtime coding agent. s158 00:15:38.959 --> 00:15:39.760 Okay. s159 00:15:39.760 --> 00:15:54.560 Unlike other coding systems in in that you're used to uh where the task is take a large code base and then help me edit it right but when the edits are done s160 00:15:54.560 --> 00:16:11.120 what I have is a piece of software and that is deployed that is the artifact okay what is running does not contain the model anymore it's just the code okay this is not what's happening here the in query time you should think about the engine essentially s161 00:16:11.120 --> 00:16:16.800 building something like a Jupyter notebook, something like uh like a Python ripple, right? s162 00:16:16.800 --> 00:16:23.279 And they literally write code and execute it and write code and execute and if they get the answer, they know what to do with it. s163 00:16:23.279 --> 00:16:36.959 And if it's, you know, if it's not what they expected, they rewrite that piece of the code and what you get in the end is a is a piece of code that completes the task that gets the information that you want to get. s164 00:16:36.959 --> 00:16:38.800 Okay, that is savable. s165 00:16:38.800 --> 00:16:40.160 that's rerunnable. s166 00:16:40.160 --> 00:16:40.800 Okay. s167 00:16:40.800 --> 00:16:50.079 But it's also incredibly flexible because now the the answer is got not by by a query to a database. s168 00:16:50.079 --> 00:16:55.120 It's it's written by code which is of course incredibly flexible. s169 00:16:55.120 --> 00:17:08.000 Okay, that code is by the way, interestingly enough, if you replace if you write software this way, the the amount the amount of prompting that you need is significantly reduced. s170 00:17:08.000 --> 00:17:19.520 We went down from having something like 150,000 tokens to give our agents all the tooling they need to less than a thousand tokens to specify all the interfaces they need to get information out of Nexus. s171 00:17:19.520 --> 00:17:20.319 Okay. s172 00:17:20.319 --> 00:17:28.079 And so the the so what it looks like when you run it is like this. s173 00:17:28.079 --> 00:17:28.480 Okay. s174 00:17:28.480 --> 00:17:42.160 So on the left of course you'll see Nexus with uh NoQL answering the question and then on the right is sort the same uh task given to an agent with all the tooling to complete the task. s175 00:17:42.160 --> 00:17:46.799 Okay, but remember this on the right it's uh severely handicapped. s176 00:17:46.799 --> 00:17:49.440 It's like the employee on the first day on the job. s177 00:17:49.440 --> 00:17:50.559 They have to read a bunch. s178 00:17:50.559 --> 00:17:52.000 They have to figure out where the data is. s179 00:17:52.000 --> 00:17:53.440 They have to figure out what the task is. s180 00:17:53.440 --> 00:17:55.280 They have to write a bunch of code. s181 00:17:55.280 --> 00:18:06.960 Uh it's just it's just severely handicapped and there's I don't know if you okay stopped but um needless to say this is by far slower. s182 00:18:06.960 --> 00:18:10.160 It's a lot more expensive in terms of token consumption. s183 00:18:10.160 --> 00:18:19.840 Uh but interestingly enough it's also a hell of a lot less accurate because again they don't have the right context, the right objectives and so on in your company. s184 00:18:19.840 --> 00:18:27.600 Um, I'll just say that [clears throat] we work uh obviously with a bunch of early access customers already. s185 00:18:27.600 --> 00:18:42.559 Uh, you see some results here, but I'll just sort of fly through them and say that across different domains, across different enterprises, across different kinds of task, you we universally see the same thing that if you s186 00:18:42.559 --> 00:18:48.400 shift to this paradigm, you get significant cost savings. s187 00:18:48.400 --> 00:18:55.520 uh 77 90 you know percent token consump token reduction is pretty common. s188 00:18:55.520 --> 00:19:06.080 Um uh it gets a hell of a lot faster anywhere from 20 30% to sometimes uh uh uh 77% faster. s189 00:19:06.080 --> 00:19:06.880 Okay. s190 00:19:06.880 --> 00:19:11.919 But and the most important thing is it actually becomes a lot more accurate as well in the same time. s191 00:19:11.919 --> 00:19:15.840 So this is really uh it's a slam dunk. s192 00:19:15.840 --> 00:19:27.200 I'll just wrap up by saying that uh um the that Nexus is coming out of early access and into public preview literally tomorrow. s193 00:19:27.200 --> 00:19:32.320 So go uh try it out.