WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=8qWIPUia2O8 NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/8qWIPUia2O8.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.559 --> 00:00:14.080 I'll get us started. s3 00:00:14.080 --> 00:00:20.560 Um, so long day of talks and uh, how are you all feeling? s4 00:00:20.800 --> 00:00:22.000 Cool. s5 00:00:22.000 --> 00:00:24.080 Yeah, good to see we still have some energy. s6 00:00:24.080 --> 00:00:26.720 You know, I know there's a lot of like evening events. s7 00:00:26.720 --> 00:00:30.800 Um, we've heard a lot about the present and I'm going to talk about the future. s8 00:00:30.800 --> 00:00:34.480 Um, so my talk is called every harness will become a claw. s9 00:00:34.480 --> 00:00:37.040 Um, here's a little bit about me. s10 00:00:37.040 --> 00:00:40.399 Um, I am the co-founder CEO of Mastra. s11 00:00:40.399 --> 00:00:42.320 We are a Typescript agent framework. s12 00:00:42.320 --> 00:00:48.160 Um, I am also the author of a book that you may have gotten a copy of either outside or at a previous event. s13 00:00:48.160 --> 00:00:54.239 Um, we have seen a lot of agents running in production um, over the last 18 months. s14 00:00:54.239 --> 00:01:03.920 And I'm not I'm saying that as kind of context for and stage setting for the thoughts and ideas that I'm about to share right now. s15 00:01:03.920 --> 00:01:10.080 Um and uh the thing that I'm going to say is is welcome to the harness era. s16 00:01:10.080 --> 00:01:12.159 What do I mean by the harness era? s17 00:01:12.159 --> 00:01:16.560 Um well u let's just talk about the types of harnesses that we see right now. s18 00:01:16.560 --> 00:01:18.640 We see local harnesses. s19 00:01:18.640 --> 00:01:22.320 Um we use them every day daily driving our our coding, right? s20 00:01:22.320 --> 00:01:24.320 Um we see cloud harnesses. s21 00:01:24.320 --> 00:01:33.200 Um these are both products that we can purchase as well as if we work and some of these uh companies that have built their own internal coding agents that live on Slack. s22 00:01:33.200 --> 00:01:42.960 Um and then of course we have the your friendly local uh open source frameworks that have some of these primitives and give you the the tools that you need to build your own. s23 00:01:42.960 --> 00:01:45.439 Um and that's where we that's where we fit in. s24 00:01:45.439 --> 00:01:55.840 Um now let's talk about where we are sort of collectively as an industry um and how things have evolved over the last we'll say year to to 18 months. s25 00:01:55.840 --> 00:01:59.680 Um last year at at AI engineer we were talking a lot about agents. s26 00:01:59.680 --> 00:02:00.960 We're talking about the agent loop. s27 00:02:00.960 --> 00:02:03.520 We were talking about agents versus workflows. s28 00:02:03.520 --> 00:02:18.720 Um so so I want to you know there's as we're thinking about um the agentic spectrum I often compare it to uh self-driving as a spectrum right there are different levels of self-driving autonomy whether that's like lane assist whether that's Tesla S FSD s29 00:02:18.720 --> 00:02:28.720 whether that's I I'm sitting in the back of my Whimo and there's nobody behind the steering wheel right um there are various aspects of the agentic spectrum between LLMs s30 00:02:28.720 --> 00:02:34.959 agents harnesses and claws and I'm going to talk about what we've seen and where we're going. s31 00:02:35.200 --> 00:02:37.599 What makes an agent different than an LM? s32 00:02:37.599 --> 00:02:40.720 Hopefully, we mostly know this, but just as a quick refresher, right? s33 00:02:40.720 --> 00:02:47.760 It's the agent loop, its tool calls, its memory, it's the ability to retry failed tasks, it's context engineering. s34 00:02:47.760 --> 00:02:52.560 Um, Dex is a close friend and an inspiration for this uh one of the inspirations for this talk. s35 00:02:52.560 --> 00:02:54.400 Um, and it's agent state, right? s36 00:02:54.400 --> 00:03:04.159 These are some things that like hey I'm running an agent in a loop and I can't just do this with a oneshot call uh to to an LLM right I didn't s37 00:03:04.159 --> 00:03:15.280 I've tried to make these qualities I don't not sure what the quality of taking actions is active or something so I just put action but um you know qualities here are starting to emerge when we move from s38 00:03:15.280 --> 00:03:25.840 an agent to a harness durability and doggedness um a friend of mine was referring to an agent that he was using and he called it dogged which I really like and I'm taking that for this talk, right? s39 00:03:25.840 --> 00:03:32.000 So, durability just the sheer quality of like being able to run not for minutes but for hours or days. s40 00:03:32.000 --> 00:03:35.519 Um, you know, what what what encompasses this? s41 00:03:35.519 --> 00:03:45.680 Well, sometimes it's like, hey, I you know, I uh lost a connection in the middle of the turn and uh you know, but I persisted the stream and so now I can resume from the place where I started, right? s42 00:03:45.680 --> 00:03:46.959 There's planning mode. s43 00:03:46.959 --> 00:03:48.879 We all see this in cloud code. s44 00:03:48.879 --> 00:03:53.040 Um parallel sub aents being able to fan out multiple tasks at the same time. s45 00:03:53.040 --> 00:03:57.680 Uh we have more affordances with a TUI and slash commands. s46 00:03:57.680 --> 00:03:58.879 We have skills. s47 00:03:58.879 --> 00:04:03.920 Um we don't have to define all our agents up front, but we can dynamically create them on the fly. s48 00:04:03.920 --> 00:04:05.840 This is, you know, very powerful. s49 00:04:05.840 --> 00:04:11.120 Um you can sp the the harness can spin up background bash tasks, right? s50 00:04:11.120 --> 00:04:14.879 Um it will autocompact when it runs out of the context window. s51 00:04:14.879 --> 00:04:20.239 Uh you know, these are all things we'll see when we use cloud code or codecs, right? s52 00:04:20.239 --> 00:04:21.919 Um it persists. s53 00:04:21.919 --> 00:04:23.600 it will persist threads, right? s54 00:04:23.600 --> 00:04:27.040 You can resume a thread once that's you've like disconnected from later. s55 00:04:27.040 --> 00:04:29.600 Um, you can cue, you can steer, you can interrupt. s56 00:04:29.600 --> 00:04:32.320 You're not just blocked waiting on the LM. s57 00:04:32.320 --> 00:04:34.000 Hey, I take a turn and then you take a turn. s58 00:04:34.000 --> 00:04:38.400 I'm playing playing Civilization here and I can't take a turn until all the other civilizations are playing. s59 00:04:38.400 --> 00:04:39.280 No, I'm playing Starcraft. s60 00:04:39.280 --> 00:04:40.240 I'm playing Age of Empires. s61 00:04:40.240 --> 00:04:41.680 And um, right. s62 00:04:41.680 --> 00:04:45.120 Uh, you know, session long um tool approval. s63 00:04:45.120 --> 00:04:54.560 So it's not just like yeah I approved this specific tool call but yeah you can run all instances of rmrf for slash right that you see in the session s64 00:04:54.560 --> 00:04:56.720 even though the first one will probably wipe your machine. s65 00:04:56.720 --> 00:05:10.800 Um okay so there there's like you know there's a um there's a few steps here and I'm I'm I'm about halfway through these and then afterwards we're going to talk about what it means and this is kind of a s66 00:05:10.800 --> 00:05:11.759 in between step. s67 00:05:11.759 --> 00:05:16.320 I think this is something we've seen over the last really 3 months. s68 00:05:16.320 --> 00:05:26.320 Um, and I think we're all still starting to grapple with what it means, which is this movement from a local harness to a cloud harness where the harness is always on. s69 00:05:26.320 --> 00:05:29.199 What do I mean by a harness that is always on? s70 00:05:29.199 --> 00:05:33.120 Well, you might be talking to it in Slack. s71 00:05:33.120 --> 00:05:36.720 Maybe you're talking to it in Slack along with your colleagues, right? s72 00:05:36.720 --> 00:05:43.120 Um maybe you're each giving it instructions and has to figure out how to parse that and use user metadata. s73 00:05:43.120 --> 00:05:44.400 Uh maybe you have a mobile app. s74 00:05:44.400 --> 00:05:48.160 I was just uh uh you know maybe you have a mobile app. s75 00:05:48.160 --> 00:05:51.680 Maybe it tunnels to your local um to to your local machine. s76 00:05:51.680 --> 00:05:55.039 Some of these uh some harness mobile apps do this. s77 00:05:55.039 --> 00:05:58.000 Um often like cool. s78 00:05:58.000 --> 00:05:58.720 How is this running? s79 00:05:58.720 --> 00:06:03.039 Well, it's probably running in a cloud sandbox because it's maybe it's running locally in your machine. s80 00:06:03.039 --> 00:06:09.759 and you're tunneling into it, but maybe it's just running in a cloud in the cloud somewhere and it's got a bunch of sandboxes which enables more parallelism. s81 00:06:09.759 --> 00:06:14.319 You can get more um parallel sub aents beyond what you can do on your machine. s82 00:06:14.319 --> 00:06:18.080 This is always a trade-off and always something you get with distributed systems, right? s83 00:06:18.080 --> 00:06:20.319 You can do more in the cloud than you can do locally. s84 00:06:20.319 --> 00:06:21.280 You have more resources. s85 00:06:21.280 --> 00:06:22.800 It requires a different architecture. s86 00:06:22.800 --> 00:06:24.000 It's more powerful. s87 00:06:24.000 --> 00:06:30.479 Um and then lastly, you're not creating code, you know, on your just on your local machine or maybe even in a git work tree. s88 00:06:30.479 --> 00:06:37.039 um you're you're probably creating, you know, if you're writing code, you're probably creating a PR that that pushes right to to GitHub. s89 00:06:37.039 --> 00:06:44.639 Um so so you know, there's a shift, right, from from local harnesses to these always on kind of like cloud harnesses. s90 00:06:44.639 --> 00:06:46.880 We're still in the middle of this. s91 00:06:46.880 --> 00:06:50.160 You may have you may only be working with a local harness. s92 00:06:50.160 --> 00:06:54.479 You may have started to see cloud harnesses pop up in your your organization, right? s93 00:06:54.479 --> 00:06:56.400 You may be figuring out how to use them. s94 00:06:56.400 --> 00:06:58.800 Your teams may be figuring out how to use them. s95 00:06:58.800 --> 00:07:10.160 Um, and then I want to talk about what the harness to claw transition is, which is imbuing these agents, imbuing these harnesses with initiative and and learning, right? s96 00:07:10.160 --> 00:07:11.360 What is initiative? s97 00:07:11.360 --> 00:07:17.840 Well, um, if you've used, let's say, a a personal assistant uh, agent, right? s98 00:07:17.840 --> 00:07:22.080 And that agent texts you and says, "Hey, I saw an urgent email come in. s99 00:07:22.080 --> 00:07:23.360 Is that email actually urgent? s100 00:07:23.360 --> 00:07:25.120 Was someone like, you know, spamming you?" s101 00:07:25.120 --> 00:07:30.720 you know, but like like the agent is listening um to external feed services. s102 00:07:30.720 --> 00:07:38.160 It has a heartbeat which means it wakes up every you know defined amount of time and um and does something right. s103 00:07:38.160 --> 00:07:44.720 Uh again like channels some uh you might be able to text it, WhatsApp it, telegram it where whatever you want. s104 00:07:44.720 --> 00:07:54.000 Um you you may persist the memory memory in a more accessible later place than just simple sort of like file storage, right? s105 00:07:54.000 --> 00:08:02.960 um you you might it might have a a Damon, it might have a a gateway uh for for sending in and receiving incoming outcoming requests. s106 00:08:02.960 --> 00:08:06.879 Uh it often will do continual learning, right? s107 00:08:06.879 --> 00:08:18.479 So this concept that you know the agent the harness runs and then you know based on the traces that it generates it it sort of autoimproves itself and there's different ways of doing this. s108 00:08:18.479 --> 00:08:23.599 you see um skill automatic skill generation for example is a common one. s109 00:08:23.599 --> 00:08:26.479 Um it could modify the code driving this as well. s110 00:08:26.479 --> 00:08:30.240 Um we haven't figured out what the right way of doing it is yet. s111 00:08:30.240 --> 00:08:34.000 We're still exploring you know the industry is still exploring options. s112 00:08:34.000 --> 00:08:46.080 Um now the reason that and and maybe this is like our unique vantage point here but you know for the last three months as a framework we've just seen this as a f as the future. s113 00:08:46.080 --> 00:08:59.279 And so we furiously looked at the the you know the features that you know openclaw have that Hermes agent have and say and and we we've said like look you know a lot of people a lot of folks want these features but they want them with power and control. s114 00:08:59.279 --> 00:09:03.200 They don't want to just put a you know a claw on a box right they want to have more. s115 00:09:03.200 --> 00:09:17.760 And so, you know, we we've been thinking about this because we we you know, my I'm not doing my job well if I'm not giving everybody the tools that they need to build agents, to build harnesses as with the maximum power, s116 00:09:17.760 --> 00:09:18.399 right? s117 00:09:18.399 --> 00:09:29.600 Um so, so hopefully like again we hopefully I've walked a little bit through the step transition with actions, durability, doggedness, always on initiative, learning. s118 00:09:29.600 --> 00:09:36.080 Again, I I think I failed in like making them all the right tense phrase and making them all qualities, but I hope you get the idea here, right? s119 00:09:36.080 --> 00:09:39.279 Um, we're ascending on the agentic spectrum. s120 00:09:39.279 --> 00:09:44.880 Um, and what was a simple LLM 18 or 24 months ago is a lot more powerful. s121 00:09:44.880 --> 00:09:56.160 So, I've called this without sort of asking consent from Pete, but I've called this Steinberger's law, which is I I believe every harness will expand until it becomes a claw. s122 00:09:56.160 --> 00:10:03.519 and and and that's a little bit um technological, that's a little bit economic, that's a little bit psychological. s123 00:10:03.519 --> 00:10:05.920 So, let me walk you through the reasoning here. s124 00:10:05.920 --> 00:10:14.640 Um the first thing that I've observed um that we've all observed um as a is that harnesses tend to expand. s125 00:10:14.640 --> 00:10:17.440 And they expand because we want them to expand. s126 00:10:17.440 --> 00:10:20.640 Um we want to DM them in Slack. s127 00:10:20.640 --> 00:10:25.440 We want to text them and like start overnight uh tasks before bedtime. s128 00:10:25.440 --> 00:10:31.040 We want this dopamine casino that we get when we put in tokens and get out code, right? s129 00:10:31.040 --> 00:10:40.320 Um or or whatever other actions, you know, um agent agents are bigger than just coding agents, but um we want our own dopamine casino. s130 00:10:40.320 --> 00:10:43.760 And this this image is thanks to uh to Dex Horty. s131 00:10:43.760 --> 00:10:49.360 Um but I see something else in our future. s132 00:10:49.360 --> 00:11:04.959 um which is that and and it's something that like I don't think we we sort of talk about as much uh but after this after this phase where where we're sort of making everything more and more powerful s133 00:11:04.959 --> 00:11:20.240 um there will be a shakeout um and and let me walk you through sort of uh through my reasoning here which is that in the 2010s we had these platforms we had Android, we had iOS. s134 00:11:20.240 --> 00:11:26.720 And all of a sudden, there were all these things we could do on our phones that we previously weren't able to do. s135 00:11:26.720 --> 00:11:31.760 We could get directions, we could hail rides, um, we could send payments. s136 00:11:31.760 --> 00:11:33.680 Um, we could play music. s137 00:11:33.680 --> 00:11:36.399 Um, other ones emerged over the course of the decade. s138 00:11:36.399 --> 00:11:38.320 We could watch short form video. s139 00:11:38.320 --> 00:11:41.600 Um, we could [clears throat] watch long form documentaries. s140 00:11:41.600 --> 00:11:42.640 We could browse the internet. s141 00:11:42.640 --> 00:11:44.959 You know, some were kind of ported over from the desktop. s142 00:11:44.959 --> 00:11:46.399 We could browse the internet. s143 00:11:46.399 --> 00:11:49.760 Um, again, you know, some, you know, we could order food, right? s144 00:11:49.760 --> 00:12:03.920 Um, we could put book accommodations, but but if you look at most of these kinds of categories, and there are quite a few categories, there really only like one or two, you know, logos here that we use, you know, okay, how many s145 00:12:03.920 --> 00:12:17.519 maybe, you know, we use Uber and we we use Lyft, but like do anyone use another rideing app here, you know, like and and so when you talk to people that are smart about like consumer s146 00:12:17.519 --> 00:12:28.160 behavior, the the The reason they say that this is is because you only really have space in your brain for like a limited number of things. s147 00:12:28.160 --> 00:12:31.920 Like if if think about something like Thumbtac. s148 00:12:31.920 --> 00:12:36.160 So uh Thumbtac didn't really serve a very high economic value. s149 00:12:36.160 --> 00:12:40.800 Like Airbnb like we only use it Airbnb occasionally, but when we use it, we like really want it. s150 00:12:40.800 --> 00:12:41.920 We really need it. s151 00:12:41.920 --> 00:12:43.279 You know, it's really valuable to us. s152 00:12:43.279 --> 00:12:45.839 Thumbtac like a little bit less so, right? s153 00:12:45.839 --> 00:12:48.480 Um and then it's also like not frequent, right? s154 00:12:48.480 --> 00:12:54.959 like maybe maybe like you know something like Door Dash or Uber people can use multiple times a day, right? s155 00:12:54.959 --> 00:12:59.600 So there's there's sort of like it either has to be very economically valuable or has to be very frequent. s156 00:12:59.600 --> 00:13:03.920 And if it's neither one of the two, um we just forget about it, right? s157 00:13:03.920 --> 00:13:08.639 It's like that, you know, college friend that like we haven't really talked to in years. s158 00:13:08.639 --> 00:13:18.079 It's not cuz like they weren't important at one point in our life, but like there's just nothing that maybe they moved to a new city or we moved to a new city or our lives, our friend groups, our careers diverged s159 00:13:18.079 --> 00:13:23.760 and all of a sudden like, you know, maybe we're calling them once a once a year or once every other year or or whatever. s160 00:13:23.760 --> 00:13:30.480 And you know, there's just nothing that makes them pertinent and brings them up in our our brains. s161 00:13:30.480 --> 00:13:40.639 And so right now we're like really excited because there's all this energy and excitement and we're all excited to these harnesses that we're like, you know, putting in tokens and getting out like useful things that we all love. s162 00:13:40.639 --> 00:13:51.519 Um and and I think that in the notsodistant future there will be this very real shakeout and and these categories will kind of emerge and we'll realize that we only have space in our lives s163 00:13:51.519 --> 00:13:54.800 for so many of these claws. s164 00:13:54.800 --> 00:13:55.600 They're very powerful. s165 00:13:55.600 --> 00:13:57.839 We we love them very much. s166 00:13:57.839 --> 00:14:04.079 Um and so I would I would think about um what what what does that mean for you? s167 00:14:04.079 --> 00:14:16.720 So the the first thing is um the first thing is don't get this is the reason that like events are are are important that like staying up with like if the rate of change increases s168 00:14:16.720 --> 00:14:21.839 3 to 4x that means you know we need to figure out what's going on even more frequently. s169 00:14:21.839 --> 00:14:23.040 That's why we're all here. s170 00:14:23.040 --> 00:14:32.240 Um, but if you're building an agent, make sure that it has the capabilities that your users need because if it doesn't and if there's newer things that come out, like they may just, s171 00:14:32.240 --> 00:14:36.560 you know, pick pick something that's more powerful because that that's happening very quickly. s172 00:14:36.560 --> 00:14:46.000 Um, and then keep in mind that if you if you if you aren't if you're the thing you're working on, um, if even if you climb up to the top of the hill, s173 00:14:46.000 --> 00:14:52.000 keep in mind there's going to be another wave of this sort of these like this this shakeout coming. s174 00:14:52.000 --> 00:14:54.959 and you know probably sometime in the later 2020s. s175 00:14:54.959 --> 00:15:03.360 Uh so um that I'm Sam um I'm the uh I'm the co-founder of Monsterra the the TypeScript agent framework. s176 00:15:03.360 --> 00:15:05.600 I'm the author of Principles of Building AI agents. s177 00:15:05.600 --> 00:15:09.120 Hopefully you can get a copy of the book outside or I've got a few here. s178 00:15:09.120 --> 00:15:11.120 Um please stop by, say hi. s179 00:15:11.120 --> 00:15:13.519 Um it's great to see all of you. s180 00:15:13.519 --> 00:15:14.800 Thank you all for coming out. s181 00:15:14.800 --> 00:15:15.920 It's a real pleasure. s182 00:15:15.920 --> 00:15:19.360 Um enjoy the rest of the conference. s183 00:15:32.075 --> 00:15:34.075 [music]