WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=0uC6u0lJJl4 NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/0uC6u0lJJl4.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.880 --> 00:00:13.599 All right. s3 00:00:13.599 --> 00:00:15.360 Um, everybody can see. s4 00:00:15.360 --> 00:00:17.039 Hey everybody. s5 00:00:17.039 --> 00:00:18.720 Um, thank you for being here. s6 00:00:18.720 --> 00:00:27.840 I'm going to talk about um the fact that if you go ahead and build a company brain, it will um likely leak company secrets. s7 00:00:27.840 --> 00:00:32.160 um which is kind of the big fear that we have about building a company brain anyway, right? s8 00:00:32.160 --> 00:00:37.760 Which is this case of like intern joins the company and then suddenly gets like comp details and everybody kind of situation, right? s9 00:00:37.760 --> 00:00:40.079 You want to you want to guard against that. s10 00:00:40.079 --> 00:00:49.200 Um this has kind of been I guess pretty much the biggest thing that's been holding us back from just deploying openclaw and Hermes all over the place, right? s11 00:00:49.200 --> 00:00:57.760 It's also kind of the reason it's like this big opportunity that Claude Tag had with it recent launch a few days ago where like it was going to be the company brain but then everybody's like well s12 00:00:57.760 --> 00:01:10.479 it's not um it doesn't look like it's going to be the company brain right so I'm going to talk about kind of what makes it challenging so before we get into that let's let's kind of understand and dissect this company brain business a little bit right um s13 00:01:10.479 --> 00:01:24.560 I'm Tan I'm the co founderql um you can check promqql out um later but uh the our background as a team building This is we come from um the hustra graph we're creators of the hassur graphql engine which is very popular s14 00:01:24.560 --> 00:01:38.079 open source project in the graphql space uh where we solved a lot of data access problems um we deployed everywhere from like apple to meta to JP Morgan etc. uh and um and that kind of gave us a lot of these this grounding s15 00:01:38.079 --> 00:01:45.040 uh for uh and and you know kind of like a lovehate relationship with uh data and data security. s16 00:01:45.040 --> 00:01:47.520 Um, all right. s17 00:01:47.520 --> 00:01:55.119 So, I'm going to show you stuff that we've been working on over the last year and kind of what we've learned from that so that you can kind of take that s18 00:01:55.119 --> 00:01:58.079 and exercise that and try it out for yourself. s19 00:01:58.079 --> 00:02:05.119 Um, and of course, at the end of the talk, happy to kind of exchange notes and see what works or what might what might not work for you. s20 00:02:05.119 --> 00:02:12.400 Um, over the last year, we've only partnered with a small set of people who've exhibited some kind of spike on scale. s21 00:02:12.400 --> 00:02:26.720 M so about 15 to 20 folks so far and now we're just starting to open it up to other people but over that course of time we've kind of looked at three different types of set of people with very different needs right you have AI native companies that are willing to just do whatever as long as it s22 00:02:26.720 --> 00:02:40.800 works you have kind of tech forward companies right folks like Instacart who um like best of breed technology right um so they'll move fast they'll be tolerable to breaking things but it just needs to be really really good right and then you have fortune banks s23 00:02:40.800 --> 00:02:42.959 uh who have like a me level security. s24 00:02:42.959 --> 00:02:45.360 Uh, thank God that they do because they're my bank. s25 00:02:45.360 --> 00:02:51.440 I definitely don't want vibecoded AI agents running inside a bank because that's where my money is. s26 00:02:51.440 --> 00:02:54.239 Um, so they have a lot of security rules. s27 00:02:54.239 --> 00:02:55.360 Uh, thank you so much. s28 00:02:55.360 --> 00:02:58.319 Uh, but we're deployed in places like those as well. s29 00:02:58.319 --> 00:03:02.720 Um, with kind of the beginnings or like the frontal lobe of a company brain, right? s30 00:03:02.720 --> 00:03:05.120 So, we'll kind of talk about those kind of learnings. s31 00:03:05.120 --> 00:03:11.040 Our own personal usage of kind of building out our company brain. s32 00:03:11.040 --> 00:03:13.200 Um it kind of is about 5,000 pages. s33 00:03:13.200 --> 00:03:14.640 So we model it as a wiki. s34 00:03:14.640 --> 00:03:16.159 Uh you can model it however you want. s35 00:03:16.159 --> 00:03:18.080 You can model it as a set of markdown files on GitHub. s36 00:03:18.080 --> 00:03:20.879 You can put it into a remember graph frag. s37 00:03:20.879 --> 00:03:23.760 Uh you can model it in a you can model it in knowledge in knowledge graphs. s38 00:03:23.760 --> 00:03:24.879 You can do whatever you want. s39 00:03:24.879 --> 00:03:27.360 Um so you can you can place it wherever you want. s40 00:03:27.360 --> 00:03:31.840 But like it's about 5,000 interconnected pages for us. s41 00:03:32.400 --> 00:03:33.599 Question for you folks. s42 00:03:33.599 --> 00:03:36.640 So suppose you had a company brain that was working. s43 00:03:36.640 --> 00:03:37.599 It was working well. s44 00:03:37.599 --> 00:03:39.920 It was all set up right. s45 00:03:39.920 --> 00:03:44.400 um there would be a kind of daily number of updates that would happen to this com to this company brain, right? s46 00:03:44.400 --> 00:03:47.360 Because it was learning stuff from everybody in the company, right? s47 00:03:47.360 --> 00:03:52.080 From finance to HR to um your engineers to everybody. s48 00:03:52.080 --> 00:04:00.720 So if you were to plot the daily number of updates happening to the company brain, what would it sort of look like? s49 00:04:00.720 --> 00:04:04.400 Would it sort of look like a roughly downward trend? s50 00:04:04.400 --> 00:04:09.680 Like all of these are like random graphs, but would you would it sort of like start and then go down? s51 00:04:09.680 --> 00:04:15.280 Would it kind of be steady going up and down as updates spike or would it kind of steadily increase upwards? s52 00:04:15.280 --> 00:04:21.359 Um so kind of think about like what would the commit history to your shared skills repo look like, right? s53 00:04:21.359 --> 00:04:26.800 How many updates are happening to a healthy company brain, right? s54 00:04:26.800 --> 00:04:28.960 Um every single day, what does that trend look like? s55 00:04:28.960 --> 00:04:31.840 Um anybody for option A? s56 00:04:31.840 --> 00:04:33.040 Anybody thinks it's option A? s57 00:04:33.040 --> 00:04:34.240 Okay, cool. s58 00:04:34.240 --> 00:04:37.520 Uh, option B. Okay. s59 00:04:37.520 --> 00:04:40.240 Option C. Ah, that's nice. s60 00:04:40.240 --> 00:04:47.360 Um, and and and so that's so when I kind of plotted our thing, right, to see what a healthy company brain looks like. s61 00:04:47.360 --> 00:04:53.120 Um, if you look at number one, it's basically saying we had a lot of enthusiasm. s62 00:04:53.120 --> 00:04:56.560 We built the company brain on day one, day two. s63 00:04:56.560 --> 00:05:05.360 somebody we gave somebody the task and said build all the shared skills repo scrape all the slack scrape all the emails build it and we'll all use it and then nobody cares s64 00:05:05.360 --> 00:05:14.880 right or you have a system which is autolearning maybe you have a Hermes that's deployed internally something like that where it's kind of steadily adding more and more comments so it kind of goes up and down depending on who has enthusiasm s65 00:05:14.880 --> 00:05:26.400 right um and then when I plotted our history over just the last uh two months and this is a little bit outdated now this is what we got and I was kind of shocked s66 00:05:26.400 --> 00:05:30.479 I was like, why is it continuously increasing? s67 00:05:30.479 --> 00:05:32.320 Like it's a gentle curve, right? s68 00:05:32.320 --> 00:05:34.320 But why is it gently just going up? s69 00:05:34.320 --> 00:05:38.320 Why is the number of updates per day increasing? s70 00:05:38.320 --> 00:05:46.160 And that was fascinating for me to see because what I realized was that if you have a system that starts to work, what happens is people start to teach it a lot more. s71 00:05:46.160 --> 00:06:00.240 It's like saying if I taught you the skill for quering data then tomorrow I'm going to teach you the skill for interpreting that data and then day after tomorrow I'm going to teach you the skill of how to take an action based on that and then after that I'm going to figure out how to do AB testing s72 00:06:00.240 --> 00:06:15.280 based on so people like you continuously add more but because everything is an agent where no amount of learning is perfect everything has its own kind of steady rate as well right so the rates kind of even your steady rates kind of keep adding up and that's what I started to notice in our thing as well this s73 00:06:15.280 --> 00:06:15.919 is early. s74 00:06:15.919 --> 00:06:18.080 So, who knows if it'll kind of peter out eventually. s75 00:06:18.080 --> 00:06:28.160 Maybe it'll start to look more like option B. But a healthy brain, of course, the overall size keeps increasing, but even your daily updates per day kind of keep increasing as well. s76 00:06:28.160 --> 00:06:32.800 So, that's a sign of a good brain that you built, right? s77 00:06:32.800 --> 00:06:35.280 A healthy brain that you built for your company. s78 00:06:35.280 --> 00:06:36.479 Awesome. s79 00:06:36.479 --> 00:06:42.560 The use cases for company brains, how we start to analyze how we build a system that won't leak secrets, right? s80 00:06:42.560 --> 00:06:44.800 So, two use cases. s81 00:06:44.800 --> 00:06:48.880 The first use case is there is a company brain. s82 00:06:48.880 --> 00:06:54.880 I want to use it in my AI agent whatever to get work done, right? s83 00:06:54.880 --> 00:06:56.639 Um I'll show you an example of that, right? s84 00:06:56.639 --> 00:07:05.280 It's like I got an email with a security questionnaire I need to answer from a customer and I talk to my AI and I'm like look up the company brain and help me answer this security questionnaire. s85 00:07:05.280 --> 00:07:05.599 Right? s86 00:07:05.599 --> 00:07:08.560 That's a totally valid use case of a company brain. s87 00:07:08.560 --> 00:07:10.240 Second, very useful use case, right? s88 00:07:10.240 --> 00:07:14.319 because it's other people's knowledge that are that is coming to me. s89 00:07:14.319 --> 00:07:31.919 Second use case of a company brain is kind of similar to what uh cloud tag is is a this idea of multiplayer and if you've been putting agents inside slack in places where multiple people can interact with it it's kind of being using using it as a shared AI right to like get stiff stuff done. s90 00:07:31.919 --> 00:07:36.960 Um, and an example of that could be collaborative incident management, right? s91 00:07:36.960 --> 00:07:40.639 So, for example, you want to say like, hey, I want to fetch logs. s92 00:07:40.639 --> 00:07:48.479 I want to investigate the there's an incident, go fetch some logs, investigate the code codebase, raise the PR, deploy to staging, deploy to prod, set up an alert, right? s93 00:07:48.479 --> 00:07:51.919 You want like multiple people are doing things with the company brain. s94 00:07:51.919 --> 00:07:53.840 So, those are kind of two use cases of the company brain. s95 00:07:53.840 --> 00:08:00.560 One is kind of this like shared collaborative knowledge use case and one is like shared AI use case itself, right? s96 00:08:00.560 --> 00:08:06.479 Both of those have a huge um security kind of problem, right? s97 00:08:06.479 --> 00:08:15.759 Um so to start to secure it, let's kind of define that a little bit more strongly, right? s98 00:08:15.759 --> 00:08:18.319 So what exactly is a company brain? s99 00:08:18.319 --> 00:08:20.240 And this is my definition of it, right? s100 00:08:20.240 --> 00:08:26.319 It's shared context that you'd put in a markdown that you'd put in a set of markdown files, right? s101 00:08:26.319 --> 00:08:34.800 and its access control rules for the different data and tools that you want to access as given to a coding agent. s102 00:08:34.800 --> 00:08:41.120 So that's what I'm calling it for um because I'm speaking so I can define whatever I want. s103 00:08:41.120 --> 00:08:42.719 Um that's my definition. s104 00:08:42.719 --> 00:08:48.560 So I'm not saying this is knowledge that is pulled into an LLM that will do tool calls, right? s105 00:08:48.560 --> 00:08:51.760 It is not a AI that is doing general purpose stuff. s106 00:08:51.760 --> 00:08:56.720 It is an AI that is a coding agent that is solving whatever problem you throw at it. s107 00:08:56.720 --> 00:08:56.959 Right? s108 00:08:56.959 --> 00:09:04.320 And similar to the a little bit of the previous talk that you folks might have heard which is this idea of like can we just use a coding agent to solve general problems. s109 00:09:04.320 --> 00:09:05.120 It's that right? s110 00:09:05.120 --> 00:09:14.880 So in the most trivial case if you say hey write me a tweet you're writing a small script that's making an AI call to write a small tweet right you probably don't need to do that. s111 00:09:14.880 --> 00:09:20.399 The AI itself can just like return the tweet back to you but like essentially claude code being used for everything. s112 00:09:20.399 --> 00:09:22.320 Cloud co-work is the same architecture. s113 00:09:22.320 --> 00:09:28.320 The codeex app is the same architecture which is this realization that you can use coding agents to solve general purpose problems. s114 00:09:28.320 --> 00:09:30.080 So we're building the brain for that. s115 00:09:30.080 --> 00:09:37.760 We're not building gigantic knowledge graph knowledge base for the company and then trying to secure it that anyway it doesn't hasn't worked won't work. s116 00:09:37.760 --> 00:09:44.880 Um, so in terms of how we want to approach designing the company brain, right? s117 00:09:45.920 --> 00:09:50.160 Should we build a company brain? s118 00:09:50.399 --> 00:10:02.959 So, if you're an enterprise and you're paid to twiddle your thumbs, then you like this idea of building a company brain because you're like, "Yes, let me take on a two-year project and I will build the company brain for JP Morgan." s119 00:10:02.959 --> 00:10:03.760 That's not going to happen. s120 00:10:03.760 --> 00:10:07.279 You can't build a company brain for an organization that's like 100 years old, right? s121 00:10:07.279 --> 00:10:10.399 you can barely build it for your own family, right? s122 00:10:10.399 --> 00:10:13.279 Which might just be months or years old, right? s123 00:10:13.279 --> 00:10:19.680 So, so the idea and the way that we want to build a company brain is we want each person who does a little bit of the work in the company s124 00:10:19.680 --> 00:10:23.360 to own and build their part of the company brain, right? s125 00:10:23.360 --> 00:10:25.600 That's the way we should build it. s126 00:10:25.600 --> 00:10:28.000 So, that's kind of constraint number two that I'm putting. s127 00:10:28.000 --> 00:10:35.200 One was the definition of the company brain and second is how the approach that we want to take for how a company brain is built. s128 00:10:35.440 --> 00:10:39.760 I like kind of this way of phrasing it, which is that we're going to grow a company brain. s129 00:10:39.760 --> 00:10:41.600 We're not going to build one, right? s130 00:10:41.600 --> 00:10:44.079 We're going to like let it let it come together. s131 00:10:44.079 --> 00:10:47.839 The system needs to come together otherwise it'll it'll it's not it's not possible to build. s132 00:10:47.839 --> 00:10:48.640 All right. s133 00:10:48.640 --> 00:10:54.880 Um broadly, we want to let each person self-s serve their bit of the company brain. s134 00:10:54.880 --> 00:10:58.320 Um and so these are kind of like the various steps that you want to follow. s135 00:10:58.320 --> 00:11:00.720 I'll come back to this in more detail if we have time. s136 00:11:00.720 --> 00:11:03.600 But let's start with a particular use case, right? s137 00:11:03.600 --> 00:11:12.160 So in this particular use case, what I have is this kind of situation where this is kind of the tangible example I want to take for you folks. s138 00:11:12.160 --> 00:11:15.519 Um I got an email just a security questionnaire example, right? s139 00:11:15.519 --> 00:11:18.800 Hey, I got an email from Dave at StitchFix. s140 00:11:18.800 --> 00:11:21.360 Um and that has a bunch of questions I want to answer, right? s141 00:11:21.360 --> 00:11:23.040 So it pulls up my email. s142 00:11:23.040 --> 00:11:30.399 It says the email has a screenshot of their security onboarding and then it starts to kind of answer those questions, right? s143 00:11:30.399 --> 00:11:32.399 Um I have no idea how it knew. s144 00:11:32.399 --> 00:11:37.360 I was kind of very surprised to see that it answered all of the questions on like, hey, this is our trust center. s145 00:11:37.360 --> 00:11:39.279 This is how our security stuff looks. s146 00:11:39.279 --> 00:11:40.880 Um, they have a gateway, right? s147 00:11:40.880 --> 00:11:42.000 It does something. s148 00:11:42.000 --> 00:11:45.839 All of this is kind of coming from the company brain, right? s149 00:11:45.839 --> 00:11:46.880 Which is the answer to that. s150 00:11:46.880 --> 00:11:50.880 And then kind of go ahead and I'm like, hey, just go ahead and send this. s151 00:11:50.880 --> 00:11:52.000 I like this draft. s152 00:11:52.000 --> 00:11:53.760 Go ahead and send this draft to Dave, right? s153 00:11:53.760 --> 00:11:55.519 And then goes and sends that email. s154 00:11:55.519 --> 00:11:57.200 Really simple example of what I want to do. s155 00:11:57.200 --> 00:12:00.399 Now, the challenge here, right? s156 00:12:00.399 --> 00:12:16.720 And the issue is that how do we build a system right which somebody else can contribute to that a third person kind of uses how did this knowledge about what our security s157 00:12:16.720 --> 00:12:33.040 thing is come in presumably somebody else had been working on the same security questionnaire right so they had let's say a Hermes agent or whatever they were working on it you autosaved some memory maybe somebody wrote down a skill somehow that piece piece has to come to my AI agent. s158 00:12:33.040 --> 00:12:36.399 How are we going to make that possible right now? s159 00:12:36.399 --> 00:12:42.880 Um let's try obvious thing uh number one right which is that everybody writes shared skills for each other on GitHub. s160 00:12:42.880 --> 00:12:50.880 So the first time the security questionnaire was answered by your security person everybody visualize like your security and compliance person in your head right now. s161 00:12:50.880 --> 00:12:57.120 Imagine that they after answering the questionnaire, it sucks to answer questionnaires. s162 00:12:57.120 --> 00:13:06.240 After answering this gigantic Excel sheet of a of a questionnaire, they then went to GitHub and updated a shared skill, right? s163 00:13:06.240 --> 00:13:17.200 Many of you are fortunate to work with people who are modeled after our Lord and Savior Christ, who are so nice, who will go and update shared skills in a GitHub repo, s164 00:13:17.200 --> 00:13:17.680 right? s165 00:13:17.680 --> 00:13:19.360 Most people will not. s166 00:13:19.360 --> 00:13:30.079 Nobody is going to write skills for another person in GitHub like that is not that is not something that is natural to us right in the dayto-day of doing work s167 00:13:30.079 --> 00:13:47.839 we don't suddenly decide that ooh this might be really useful for somebody else I don't even know I'm not connected to in this situation in the future not happening I can barely get it to like curate my own memory and my context I do not have the time to send it to somebody else um to write it s168 00:13:47.839 --> 00:13:49.239 down for somebody s169 00:13:49.239 --> 00:13:49.839 [snorts] s170 00:13:49.839 --> 00:13:53.760 Second, instead of having a company brain, why don't you do a team brain? s171 00:13:53.760 --> 00:13:55.920 Why don't you all just use one shared? s172 00:13:55.920 --> 00:14:01.680 Why doesn't the security team kind of use one more shared silo where you can do this, right? s173 00:14:01.680 --> 00:14:05.040 So, build an agent and have it kind of save to memory itself, right? s174 00:14:05.040 --> 00:14:10.079 Um, and that's kind of the architecture that I I'm guessing a lot of you folks have with maybe something like a Hermes added to Slack. s175 00:14:10.079 --> 00:14:18.240 Does anybody have kind of a team team brain situation going where you have an AI that multiple people use that autosaves memory and auto adds context? s176 00:14:18.240 --> 00:14:23.199 Does anybody have kind of like a skill that does that already just for a small team? s177 00:14:23.199 --> 00:14:23.600 One. s178 00:14:23.600 --> 00:14:24.320 Anybody else? s179 00:14:24.320 --> 00:14:24.639 Okay. s180 00:14:24.639 --> 00:14:24.880 Okay. s181 00:14:24.880 --> 00:14:25.600 A few of you have that. s182 00:14:25.600 --> 00:14:26.240 That's cool. s183 00:14:26.240 --> 00:14:31.920 Um this is nice but the problem is it's still not a company brain because it's still isolated, right? s184 00:14:31.920 --> 00:14:33.920 So it's like one more silo, right? s185 00:14:33.920 --> 00:14:52.240 Like for example um if this gets like with claw tag um it it has a per channel memory right so in every channel it gets saved but now it's another silo in that one channel right so now it's again locked into one place that can't be used anywhere else so if somebody got added to that channel it s186 00:14:52.240 --> 00:15:06.079 would work but otherwise it wouldn't work right um and so this is the third option the third option is saying all context text goes into a single shared wiki. s187 00:15:06.079 --> 00:15:08.880 A wiki is a set of markdown files and markdown files can link to each other. s188 00:15:08.880 --> 00:15:11.040 So imagine a gigantic folder. s189 00:15:11.040 --> 00:15:13.920 The folder has lots of markdown files, right? s190 00:15:13.920 --> 00:15:28.560 And So all all context instead of saving it inside a folder, siloing it, you put it in a markdown file, the equivalent of a markdown file, and you let it link with each other. s191 00:15:28.560 --> 00:15:38.320 The second thing that you do is you allow each file to have scopes on who can have readwrite access to that file. s192 00:15:38.959 --> 00:15:47.360 The third thing that you do which is the most important, you don't let the agent auto add the memory. s193 00:15:49.519 --> 00:15:55.839 You don't let it auto add because if it auto adds, you have no idea what happened, right? s194 00:15:55.839 --> 00:16:04.959 You can't the we we're back to kind of the same world where some stuff is getting added and as long as you're in that agent's memory, you're lucky, right? s195 00:16:04.959 --> 00:16:19.360 So, the third thing that you do is instead of letting the agent auto add, do something that allows your agent to suggest what is added with what scopes and then have the human s196 00:16:19.360 --> 00:16:21.920 accept or reject. s197 00:16:21.920 --> 00:16:33.120 So it's not as heavy as GitHub where I have to go and write this update a shared skill do a PR review and then get it merged but it's also not as yolo s198 00:16:33.120 --> 00:16:48.639 as the memory just kind of being autowritten by the agent right it's kind of the sweet spot where while you are working you pop it up suggest the right scopes and let somebody add it so now what happens is with this very simple addition s199 00:16:48.639 --> 00:16:57.440 right you are able to let people add to a gigantic wiki, but you let that person take on responsibility for what they can see or not. s200 00:16:57.440 --> 00:17:04.000 So, if I'm adding something to the finance wiki, I want to make sure I'm adding something that's sensitive, I want to make sure it has a finance scope. s201 00:17:04.000 --> 00:17:06.959 If I'm adding something that's personal, I want to make sure that that's personal scope. s202 00:17:06.959 --> 00:17:09.199 Let me show you an example UX of what that might look like. s203 00:17:09.199 --> 00:17:11.679 This is what we do. s204 00:17:25.520 --> 00:17:33.280 This was a recent email that I got um from one of our sales reps adding me onto a call. s205 00:17:33.280 --> 00:17:43.919 I looked at that email, helped answer it, and then I got a little box that suggested a bunch of bullets that told me what it's going to add, right? s206 00:17:43.919 --> 00:17:47.919 And when I hit add to wiki and and so now it's much easier for me to review what is getting added. s207 00:17:47.919 --> 00:17:54.960 I don't care I don't care if it gets added into this markdown file, that markdown file, what links that the agent takes care of. s208 00:17:54.960 --> 00:17:58.640 What I care about is are these facts correct? s209 00:17:58.640 --> 00:18:03.120 If these facts are correct, I'm going to hit add to wiki and I'm going to be done, right? s210 00:18:03.120 --> 00:18:08.640 And and during the time of add to wiki, I can choose what scopes need to be added per wiki page or not. s211 00:18:08.640 --> 00:18:08.880 Right? s212 00:18:08.880 --> 00:18:17.120 So each wiki page itself can get a certain set of scopes that you want to decide who gets access to what for example right so for example my email s213 00:18:17.120 --> 00:18:25.280 this is the wiki page that I have for my emails and how my emails are prioritized and I can now decide who gets access to this who are the owners for this and what the artback for this is. s214 00:18:25.280 --> 00:18:34.160 So whatever the system looks like is up to you folks but the core idea is that you want to get the agent to suggest a change instead of doing the change. s215 00:18:34.160 --> 00:18:35.200 All right. s216 00:18:35.200 --> 00:18:36.320 So, two rules. s217 00:18:36.320 --> 00:18:39.440 One, make sure that everything goes into one companywide wiki. s218 00:18:39.440 --> 00:18:41.360 Don't back down from this rule. s219 00:18:41.360 --> 00:18:47.520 Second, make sure that like as a part of that, every change is backed by a human's name. s220 00:18:47.520 --> 00:18:55.120 Nothing should be allowed inside the wiki that is Claude added this or like your AI agent added this or Hermes added this. s221 00:18:55.120 --> 00:18:56.320 No, Tanme added this. s222 00:18:56.320 --> 00:19:11.840 that that name needs to be there so that you can tie it back to this is the person who screwed up and like allowed everybody to see like everybody's comp right and like whatever now you can take remedial action uh right whatever that is put them on a pip s223 00:19:11.840 --> 00:19:24.160 um you didn't know how to edit a wiki so so that is very very important and rule number two once you decide that you can then go to the second scope of like okay you've got to make it easy for them to do that which is where this business of scopes come in where you want to scope s224 00:19:24.160 --> 00:19:27.440 each file according to who gets access you to build kind of a system around it. s225 00:19:27.440 --> 00:19:41.760 This is what an architecture diagram of that looks like where you have users um users talk to the agent uh agent when it's reading context uses that particular user's claims right so if I am reading s226 00:19:41.760 --> 00:19:51.840 something for solving a finance problem it's using the finance claim to read as me because I had access to the finance wiki so I can read it and that is done s227 00:19:51.840 --> 00:19:58.080 every single time right so the agent is always using the user's credential to read the right part of the wiki. s228 00:19:58.080 --> 00:20:01.520 Um, all right. s229 00:20:01.520 --> 00:20:05.200 I am um I'm fairly out of time for the second use case. s230 00:20:05.200 --> 00:20:09.440 So, what I'm going to do is give you a quick flavor of the second use case, but extend this idea. s231 00:20:09.440 --> 00:20:12.880 This is the daddy use case. s232 00:20:12.880 --> 00:20:14.799 This is like this is the big daddy use case. s233 00:20:14.799 --> 00:20:20.880 This is a really complicated use case because now it's not just one person answering an email. s234 00:20:20.880 --> 00:20:31.919 It's a bunch of us using the shared context to solve a problem with various different escalation like privilege levels at the same time, right? s235 00:20:31.919 --> 00:20:40.080 And these are kind of the these are the these are the interactions AI where the most amount of company brain knowledge is created, right? s236 00:20:40.080 --> 00:20:46.880 For example, I'm going to show you a quick real life example of um what it looks like for us. s237 00:20:46.880 --> 00:20:59.200 Um so this was a case from an SR situation where um somebody was like hey our autolearning our wiki learning uh fairly meta was failing it wasn't working what's going on s238 00:20:59.200 --> 00:21:12.880 right and then it starts doing the investigation and it sucks cuz it's it didn't have a skill it failed so like bro don't do this please use this open telemetry span name used an open telemetry span name it did a slightly better job but was still really slow s239 00:21:12.880 --> 00:21:16.960 so he looked at the code and he's like oh you're using a like query. s240 00:21:16.960 --> 00:21:18.480 You're you're a you're a dumbass. s241 00:21:18.480 --> 00:21:22.480 This is Opus 4.5. Um we like like don't do this. s242 00:21:22.480 --> 00:21:22.799 Right? s243 00:21:22.799 --> 00:21:24.799 So then he's like don't use a like query. s244 00:21:24.799 --> 00:21:26.159 Use an equals to query. s245 00:21:26.159 --> 00:21:26.559 Right? s246 00:21:26.559 --> 00:21:35.200 And then it does equals to query and it surfaces some details and it and then he says oh dig deeper into this and it says whatever this is a line of code where the error is coming from. s247 00:21:35.200 --> 00:21:36.320 Simple stuff right? s248 00:21:36.320 --> 00:21:43.440 This is now where it surfaces some knowledge and says aha I learned that I should use equals to and not like. s249 00:21:43.440 --> 00:21:43.679 Right? s250 00:21:43.679 --> 00:21:49.520 I learned that if you have a custom prefix added to wiki page names, it can cause issues, right? s251 00:21:49.520 --> 00:21:53.120 Um, so it offers these learnings that you can choose to accept. s252 00:21:53.120 --> 00:21:56.960 So he kind of went dug in deeper um into what the problem was. s253 00:21:56.960 --> 00:21:59.760 Somebody else joined the conversation, right? s254 00:21:59.760 --> 00:22:03.360 And said the technical decision that we've made here is wrong. s255 00:22:03.360 --> 00:22:04.640 Why is this happening? s256 00:22:04.640 --> 00:22:08.159 And now two people start to have an argument, right? s257 00:22:08.159 --> 00:22:10.720 They have an argument saying, hey, it should not be like this. s258 00:22:10.720 --> 00:22:19.679 it should be like this but why is it like this but it should be like this right that argument creates knowledge because the actual problem was that somebody made a technical decision that was not documented s259 00:22:19.679 --> 00:22:27.120 right when they decide to fix that issue and they observe that that is indeed the root cause and they decide that this is the way it's going to be fixed s260 00:22:27.120 --> 00:22:37.760 hey we should remove this prefix that's causing a problem whatever whatever the thing is that creates the highest quality context to be added to your brain because the previous suggestion s261 00:22:37.760 --> 00:22:41.280 was to say uh pages should not pages have a prefix. s262 00:22:41.280 --> 00:22:44.240 But the fact that pages have a prefix is a problem. s263 00:22:44.240 --> 00:22:45.039 Right? s264 00:22:45.039 --> 00:22:49.200 So now the thing that you're documenting in the brain is pages should not have a prefix. s265 00:22:49.200 --> 00:22:52.640 If they have a prefix, it can cause lookup issues and prod. s266 00:22:52.640 --> 00:22:56.400 This happens when multiple people talk to each other, right? s267 00:22:56.400 --> 00:22:57.360 And solve problems together. s268 00:22:57.360 --> 00:22:58.880 This is what happens in a Slack thread. s269 00:22:58.880 --> 00:23:03.280 When two people talk to each other and solve a problem, it creates the highest quality context. s270 00:23:03.280 --> 00:23:06.080 But and so so that's kind of what you want here. s271 00:23:06.080 --> 00:23:11.840 But the challenge is that the priv privilege escalation around this becomes very very serious. s272 00:23:11.840 --> 00:23:24.559 If you if you're building an agent that can do everything surrounded by multiple people, that's scary because the engineer was allowed to do the PR work but now I can use the same agent to deploy to prod. s273 00:23:24.559 --> 00:23:25.440 That's too scary. s274 00:23:25.440 --> 00:23:32.640 I can't have a conversation where I debug and deploy securely, right? s275 00:23:32.640 --> 00:23:34.080 Especially if you're in a bank, right? s276 00:23:34.080 --> 00:23:39.360 like the people who are debugging, deploying to staging, setting up an alert and deploying are not the same. s277 00:23:39.360 --> 00:23:43.440 But but being the same has a lot of value because that's where all the knowledge is, right? s278 00:23:43.440 --> 00:23:50.880 And so that kind of brings us to the second architecture which I'm not going to get into too much detail with, but think of it as the same idea s279 00:23:50.880 --> 00:23:54.799 where user credentials and claims were used to read context. s280 00:23:54.799 --> 00:24:02.640 Instead of that, also use user credentials, right, when the code is executing tools. s281 00:24:02.640 --> 00:24:05.760 So never store credentials in the sandbox. s282 00:24:05.760 --> 00:24:16.159 Instead of that at the HTTP layer, at the SQL layer, inject the user's credentials, allowing the AI to behave as the human in a particular interaction, right? s283 00:24:16.159 --> 00:24:19.200 Um so there's interesting details here. s284 00:24:19.200 --> 00:24:23.520 Um but that is what allows a shared AI to work with shared context, right? s285 00:24:23.520 --> 00:24:26.640 And those are kind of the two um key pieces to work with. s286 00:24:26.640 --> 00:24:32.240 So I would summarize and you'll this architecture is not particularly complicated, but it's very simple to work back from these two rules. s287 00:24:32.240 --> 00:24:34.480 Do not store credentials in the cloud sandbox. s288 00:24:34.480 --> 00:24:39.840 And second, virtualize all interactions with real data. s289 00:24:39.840 --> 00:24:41.919 Proxy it, virtualize it, whatever word you want to use. s290 00:24:41.919 --> 00:24:44.080 And let users control them. s291 00:24:44.080 --> 00:24:49.919 So the user who adds a particular tool should control who gets access uh to that particular tool. s292 00:24:49.919 --> 00:24:54.880 So you can derive this entire thing if you just kind of follow these four principles and work backwards from that. s293 00:24:54.880 --> 00:25:03.600 there's only one architecture that is possible that makes sense uh in how you manage context and what constraints you set up and how you manage tools and what security rules you set up. s294 00:25:03.600 --> 00:25:13.360 Um that is my time um and and so um happy to chat more um after the talk um we have a booth as well so happy to chat more through that on s295 00:25:13.360 --> 00:25:15.440 what the nuances inside this architecture are. s296 00:25:15.440 --> 00:25:16.880 I'm Tanme on Twitter. s297 00:25:16.880 --> 00:25:18.400 Um we're called PromQL. s298 00:25:18.400 --> 00:25:24.640 Um do check us out um at the end of the day um with the AI engineering community. s299 00:25:24.640 --> 00:25:26.880 We're going to do a product launch. s300 00:25:26.880 --> 00:25:30.799 Uh and so I would love to share that folks share that with everybody. s301 00:25:30.799 --> 00:25:35.840 I'm going to take a picture with everybody on stage so that I can uh I can share that. s302 00:25:35.840 --> 00:25:39.919 And so let me let me do that while I'm here. s303 00:25:40.559 --> 00:25:41.039 All right. s304 00:25:41.039 --> 00:25:43.197 Do folks want to say cheese? s305 00:25:43.197 --> 00:25:45.197 [laughter] s306 00:25:46.000 --> 00:25:47.120 Thank you so much. s307 00:25:47.120 --> 00:25:48.640 Um so watch out for that. s308 00:25:48.640 --> 00:26:03.039 Um it's our approach to um claude tag which is prompt tag which is very similar to the ideas that we tag chatted about here except that you're not stuck to cloud you can use GLM and you can use GPT and then soul comes out and we can use that and have a lot of fun. s309 00:26:03.039 --> 00:26:08.159 Um so do check that out and otherwise I'll see you folks soon.