WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=X6l4lpA0_NY NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/X6l4lpA0_NY.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.880 --> 00:00:13.920 Quick introduction. s2 00:00:13.920 --> 00:00:15.280 My name is Drew Knox. s3 00:00:15.280 --> 00:00:19.199 I'm the head of product and design at Tessle. s4 00:00:19.199 --> 00:00:22.000 Tessle is an agent enablement platform. s5 00:00:22.000 --> 00:00:27.680 We help you go from scaling skills to building your own software factory. s6 00:00:27.680 --> 00:00:30.640 So today I'm going to talk about a few things. s7 00:00:30.640 --> 00:00:42.879 The core is going to focus on harness engineering which we see as sort of the new discipline of agentic development and how you can use it to ladder up to a software factory. s8 00:00:42.879 --> 00:00:46.719 So before we get started a few prerequisites. s9 00:00:46.719 --> 00:00:55.840 This is a bit of an advanced talk in the sense that like if you aren't already using coding agents what I'm about to walk through is probably not where you want to get started. s10 00:00:55.840 --> 00:01:04.159 So, I'm sort of assuming that if you're working with coding agents, you're used to having a couple sessions at a time. s11 00:01:04.159 --> 00:01:14.640 Uh you've gotten to a place where coding agents will for like easy tasks and maybe medium complexity tasks frequently do the right thing. s12 00:01:14.640 --> 00:01:21.119 This you're sort of at the sweet spot for starting to think about harness engineering, starting to think about a software factory. s13 00:01:21.119 --> 00:01:26.960 What I'm going to say here is not exclusive to that, but that's where you'll get your best results. s14 00:01:27.360 --> 00:01:29.439 So, I'm going to walk through a few things. s15 00:01:29.439 --> 00:01:37.280 First, I want to define software factory and talk a little bit about why you might want to build towards one. s16 00:01:37.280 --> 00:01:45.920 Second, we're going to then focus a lot on harness engineering, which is kind of the core practice or skill that you'll use to reach a software factory. s17 00:01:45.920 --> 00:01:50.720 We'll talk about what are going to be the components you work on while you're doing harness engineering. s18 00:01:50.720 --> 00:01:56.720 And finally, I'll walk through how the Tesla product uh briefly can help you along that journey. s19 00:01:56.720 --> 00:02:00.960 But most of the topics I'm going to talk through here today work with any stack. s20 00:02:00.960 --> 00:02:02.399 They're mostly techniques. s21 00:02:02.399 --> 00:02:07.280 Tesla makes them easy, but they're not exclusive to us. s22 00:02:07.280 --> 00:02:07.759 Okay. s23 00:02:07.759 --> 00:02:09.920 Part one, software factories. s24 00:02:09.920 --> 00:02:12.319 What are they? s25 00:02:12.480 --> 00:02:14.319 There's no strict definition. s26 00:02:14.319 --> 00:02:22.160 I'm sure you're all probably as frustrated as I am with how quickly terminology changes and migrates in the in the industry today. s27 00:02:22.160 --> 00:02:37.040 But roughly speaking, a software factory is any agentic system where all of the end product of what you're building and shipping to users is created by agents and your software engineering teams effectively s28 00:02:37.040 --> 00:02:38.800 focus on building the factory. s29 00:02:38.800 --> 00:02:45.760 So they work on making the factory more autonomous, more automated, and raising the quality of what it produces. s30 00:02:45.760 --> 00:02:48.319 I'll go into each of those terms in just a second. s31 00:02:48.319 --> 00:02:59.120 That's kind of the core idea of a factory though is everyone on your team effectively becomes internal tool builders and then you're building a system that produces your product. s32 00:03:00.080 --> 00:03:02.480 So I mentioned three key metrics. s33 00:03:02.480 --> 00:03:09.760 These are sort of like the pillars you need to think about when building towards a software factory and they sort of go in this order. s34 00:03:09.760 --> 00:03:16.480 So autonomy is effectively how much human intervention do you need to get to the right answer. s35 00:03:16.480 --> 00:03:23.840 So how many times you need to correct the code or provide a nudge on how the agent should be building. s36 00:03:23.840 --> 00:03:26.640 Automation uh oh sorry I've gotten them flipped here. s37 00:03:26.640 --> 00:03:29.200 Uh autonomy is the first one. s38 00:03:29.200 --> 00:03:31.920 It's how much do you have to do humans have to correct? s39 00:03:31.920 --> 00:03:35.519 Automation is how much can you let them run, right? s40 00:03:35.519 --> 00:03:41.519 How much do you need to review or build trust in the solution before you accept it? s41 00:03:41.519 --> 00:03:43.599 They sound quite similar, but there is a difference. s42 00:03:43.599 --> 00:03:53.360 You can have high autonomy in the sense that agents frequently are oneshotting the problems you're giving them, but low automation because you don't trust it yet. s43 00:03:53.360 --> 00:03:55.760 And so you're reviewing all of the code directly. s44 00:03:55.760 --> 00:03:58.480 You are manually verifying everything. s45 00:03:58.480 --> 00:04:01.680 So they are distinct things that obviously have a connection, right? s46 00:04:01.680 --> 00:04:07.120 You have to build up autonomy first before you can move to automation. s47 00:04:07.120 --> 00:04:15.360 The final quality, probably the most self-explanatory, how good is the product that you're actually producing for your users? s48 00:04:15.360 --> 00:04:27.919 These metrics are like your usual user analytics, your uh test quality, your test coverage, etc. As you work towards a factory, you're going to go improving autonomy. s49 00:04:27.919 --> 00:04:34.639 then approving automation all while keeping quality constant and then the sort of payoff at the end. s50 00:04:34.639 --> 00:04:40.479 Why would you want a software factory is that ultimately we think you can raise quality once you have a factory. s51 00:04:40.479 --> 00:04:42.160 So talk about this a bit. s52 00:04:42.160 --> 00:04:48.000 I think most folks look at factories as a way to ship faster. s53 00:04:48.000 --> 00:04:56.800 They'll say like oh yeah you'll get a little bit of slop but my god you'll you'll launch so much more features like the cost return the ROI will be worth it. s54 00:04:56.800 --> 00:05:00.080 I think that's a very shortterm phenomenon, right? s55 00:05:00.080 --> 00:05:04.000 As agents are sort of coming online and getting better. s56 00:05:04.000 --> 00:05:08.800 In reality, yes, you will move to higher velocity with a factory. s57 00:05:08.800 --> 00:05:21.120 But every team has backlogs of bug fixes or improvements that they wish they could be making that they just don't have time to do or explorations that they wish they were they were doing. s58 00:05:21.120 --> 00:05:24.479 With a software factory, the idea of a backlog kind of goes away. s59 00:05:24.479 --> 00:05:32.400 And so you actually just have much more capacity to work on test quality improvements, architectural refactors. s60 00:05:32.400 --> 00:05:43.360 So at Tesla, we believe that yes, at first as you transition, you will want to keep quality constant or maybe like a very small dip, but that the ultimate payoff is you should see better quality s61 00:05:43.360 --> 00:05:46.000 uh in the code that you're producing. s62 00:05:46.000 --> 00:05:49.440 Also, it's great for your overall team collaboration styles. s63 00:05:49.440 --> 00:05:56.400 So once you've moved to a software factory, it's much easier for people who aren't in technical roles to contribute ideas. s64 00:05:56.400 --> 00:06:09.120 So it's easier to explore more and to have more perspectives coming in and contributing while your engineering team is much more focused on building the underlying system that everyone is using to push features out to your users. s65 00:06:09.120 --> 00:06:15.840 So I think ultimately it leads to a more dynamic and inclusive uh development experience. s66 00:06:16.560 --> 00:06:18.800 So, how do you get to a software factory? s67 00:06:18.800 --> 00:06:27.440 Harness engineering, uh, as a sign of how fast terms evolve, it's also started to be called loop engineering over the last couple weeks. s68 00:06:27.440 --> 00:06:34.160 Um, this is really the core discipline that you're going to take to get towards a software factory. s69 00:06:34.240 --> 00:06:40.240 At its core, harness engineering is, you know, I mentioned the software factory is building your product. s70 00:06:40.240 --> 00:06:47.440 you are building the loops that automate and improve the quality of your factory. s71 00:06:47.440 --> 00:06:49.280 So what do I mean by loops? s72 00:06:49.280 --> 00:06:52.479 What are the loops that you might be working on? s73 00:06:52.479 --> 00:06:59.840 There are three core concepts each at maps to a certain phase of development. s74 00:06:59.840 --> 00:07:07.599 So you have your inner loop which is as the coding agent is working on a PR before it has put it up. s75 00:07:07.599 --> 00:07:07.759 Right? s76 00:07:07.759 --> 00:07:09.599 So these are very fast iterative loops. s77 00:07:09.599 --> 00:07:10.960 You want them to be cheap. s78 00:07:10.960 --> 00:07:13.759 You're expecting the agent to run them all the time. s79 00:07:13.759 --> 00:07:16.319 Improvements here will drive better autonomy. s80 00:07:16.319 --> 00:07:23.280 The they help catch things and correct the agent without your uh coming in and human intervention. s81 00:07:23.759 --> 00:07:27.199 Once the PR is up, you move to the outer loop. s82 00:07:27.199 --> 00:07:37.120 So this is where you're going to put more expensive exhaustive checks that don't make sense to run over and over again as the agent is iterating on features, but s83 00:07:37.120 --> 00:07:42.960 they do encode things that otherwise a human would have to verify to build confidence in the system. s84 00:07:42.960 --> 00:07:47.440 And so this is because they're more expensive, but they are taking away human review time. s85 00:07:47.440 --> 00:07:48.400 They can be worth it, right? s86 00:07:48.400 --> 00:07:58.080 So this is where you might put something like aentic QA or where you might say like run mutation testing to check the quality of our test suite. s87 00:07:58.080 --> 00:08:05.360 This is for expensive things you want to run once when the PR goes up and then have the agent iterate on on the results. s88 00:08:06.000 --> 00:08:08.879 Finally, you have the metal loop. s89 00:08:08.879 --> 00:08:11.280 This is where you'll drive quality. s90 00:08:11.280 --> 00:08:15.280 So the metal loop sits outside of the development process. s91 00:08:15.280 --> 00:08:17.759 It observes your coding agent logs. s92 00:08:17.759 --> 00:08:33.279 It looks at PRs, your issue tracker, user feedback, and it's basically finding mistakes that are making their way through the pipeline to your users or that you had to correct as a human to stop that from happening s93 00:08:33.279 --> 00:08:39.279 and it feeds back into the inner and the outer loop to make sure that that mistake is not made again. s94 00:08:39.279 --> 00:08:50.160 So the outer loop is what really kind of once you have the infrastructure for each of these loops, the outer loop or sorry the metal loop is where you're going to be actually driving from s95 00:08:50.160 --> 00:08:53.600 a certain percentage of like how AI native am I? s96 00:08:53.600 --> 00:08:58.800 You're going to be driving it up by investing in your metal loop. s97 00:08:59.920 --> 00:09:04.640 Before we get into like what are the components, what do you do to actually accomplish this? s98 00:09:04.640 --> 00:09:16.800 I think full disclosure upfront harness engineering at least uh raw unassisted harness engineering is hard for a few specific reasons that basically come down to human psychology. s99 00:09:16.800 --> 00:09:29.440 So the first hopefully this one will go away over time but it's a new discipline and it's changing very fast and so a lot of teams that get into harness engineering s100 00:09:29.440 --> 00:09:32.800 you basically become an AI researcher just to try and keep up. s101 00:09:32.800 --> 00:09:42.720 you're reading papers, you're watching blogs, you're coming up with best practices that go out of date the next week and then two weeks after that and actually at some point they become s102 00:09:42.720 --> 00:09:45.200 antiatterns and you don't want to do them anymore. s103 00:09:45.200 --> 00:09:51.680 Uh, and so there just like a lot of time and space you'll devote to keeping up with how to do harness engineering. s104 00:09:51.680 --> 00:09:58.720 And so if you want to get into this discipline, you need to have a sort of upfront approach to how you're going to solve this problem. s105 00:09:58.720 --> 00:10:04.959 How are you going to make time and space to keep up with the knowledge that's changing every week? s106 00:10:04.959 --> 00:10:10.880 The second, and this one is more durable in my opinion, harness engineering is fundamentally unplanned work. s107 00:10:10.880 --> 00:10:13.200 So you will start shipping a feature. s108 00:10:13.200 --> 00:10:16.959 You have no way to anticipate will agents fail? s109 00:10:16.959 --> 00:10:18.079 How will they fail? s110 00:10:18.079 --> 00:10:20.480 How long is it going to take me to fix it? s111 00:10:20.480 --> 00:10:25.440 And all of that work that you're going to do to try and make the agent better competes with shipping, right? s112 00:10:25.440 --> 00:10:27.839 So it will slow you down from getting the features out. s113 00:10:27.839 --> 00:10:32.000 And we all know that is just a perpetually hard trade-off to make. s114 00:10:32.000 --> 00:10:43.279 You either don't do it and you get stuck in the local maxima of the agent never improves or you do it and you miss your deadline for the feature and you get in trouble and so no one ever does it. s115 00:10:43.279 --> 00:10:54.160 The final piece is that once you've solved these initial problems to like make time and space for harness engineering, you'll find that a lot of the information you want access to s116 00:10:54.160 --> 00:10:55.200 is not available. s117 00:10:55.200 --> 00:10:59.760 it's hidden away in local coding agent logs or it is on someone's machine somewhere. s118 00:10:59.760 --> 00:11:01.279 It's in someone's head. s119 00:11:01.279 --> 00:11:12.079 A lot of the signals that you want to see to make agents better, you need to do some work to move your workflows onto surfaces where everything is saved and publicly available s120 00:11:12.079 --> 00:11:15.600 for later optimization loops. s121 00:11:16.160 --> 00:11:22.880 Okay, with all that aside, then what are the pieces that you should be working on as you're doing harness engineering? s122 00:11:23.120 --> 00:11:25.120 I've broken it into three layers. s123 00:11:25.120 --> 00:11:32.240 These aren't by any means standard terms, though if you want to help me make them so, I would be forever grateful. s124 00:11:32.240 --> 00:11:39.760 The first layer, and these are sort of like in the order you need to do them as far as Tesla is concerned, is you have to get your control plane right. s125 00:11:39.760 --> 00:11:49.600 And so I mentioned how most of the data that you want to have access to to make agents better and move towards a software factory is not legible. s126 00:11:49.600 --> 00:11:58.240 If you move your workflows into for example at Tessle we have all issue all work starts as an issue on the issue tracker. s127 00:11:58.240 --> 00:12:02.399 It gets sent to a headless agent running in a sandbox. s128 00:12:02.399 --> 00:12:07.440 The agent puts up a PR and then engineers engage with comments on the PR. s129 00:12:07.440 --> 00:12:08.959 So it still has manual correction. s130 00:12:08.959 --> 00:12:15.440 It still has manual interface but now all of your touch points with the agent are legible. s131 00:12:15.440 --> 00:12:21.440 So we can you can then point tools at them to pull that information down and make things better. s132 00:12:21.440 --> 00:12:25.920 Typically the three things that everybody comes to when they're trying to set up a control plane. s133 00:12:25.920 --> 00:12:30.399 So you have some way of tracking issues and kicking off agent work from that. s134 00:12:30.399 --> 00:12:32.000 You're going to have some way to review. s135 00:12:32.000 --> 00:12:34.720 GitHub PR review is probably like the easiest. s136 00:12:34.720 --> 00:12:42.959 Uh and then you're going to have some way to uh standardize and distribute workflows, playbooks, things that make the agents better. s137 00:12:42.959 --> 00:12:50.800 Typically, that ends up being something like a skills registry or a shared GitHub repo where you push skills that improve agent quality. s138 00:12:51.600 --> 00:13:08.959 Next is a massive grab bag that I call agent it, which is uh if anyone's ever tried to move like a project that had a lot of local config into something that someone else can set up and you realize with horror all of the dependencies that you thought were cleanly isolated and are not, s139 00:13:08.959 --> 00:13:11.360 you basically are going to have to go through the same thing with agents. s140 00:13:11.360 --> 00:13:16.160 There's going to be all these things that you thought were easily accessible by an agent, right? s141 00:13:16.160 --> 00:13:23.040 Like CLI, API access, giving a way for the agent to click through your product to build it. s142 00:13:23.040 --> 00:13:25.040 I I promise it's worse than you think. s143 00:13:25.040 --> 00:13:32.320 Like there's no one who has come in thinking like, yeah, yeah, yeah, it's not going to be that hard for us has left saying that at the end. s144 00:13:32.320 --> 00:13:35.120 And so there's just like a lot of pieces you'll work work through. s145 00:13:35.120 --> 00:13:43.519 And this is probably the most unplannable and most like spend a week, spend two weeks and just get it done. s146 00:13:43.519 --> 00:13:59.600 But it will come through things like writing writing down in your company brain how you get work done, what are your workflows, etc. You're going to have to give access to all your internal services, which will require governance and like API access questions. s147 00:13:59.600 --> 00:14:02.480 You're going to want to give a way to access production logs. s148 00:14:02.480 --> 00:14:07.920 Uh, and that's going to have its whole own question of like what's your compliance stance, things like that. s149 00:14:07.920 --> 00:14:12.160 Uh, and then you'll need an environment where the agent can execute the code that it is running. s150 00:14:12.160 --> 00:14:19.600 There's infinite more, but they're generally pretty company specific, and you'll just work through as you try to get agents to put code up without you intervening. s151 00:14:19.600 --> 00:14:22.959 You'll quickly learn what are the problems. s152 00:14:23.440 --> 00:14:26.800 Finally, this is where you'll end up spending most of your time. s153 00:14:26.800 --> 00:14:28.320 These are the improvement loops. s154 00:14:28.320 --> 00:14:32.320 So these are the things I mentioned before the like inner outer metal loop. s155 00:14:32.320 --> 00:14:35.199 These are components that tend to go into your metal loop. s156 00:14:35.199 --> 00:14:36.800 So repo maintenance. s157 00:14:36.800 --> 00:14:44.399 You're going to want things that sweep your codebase every day, every night, every week, something like that to look for problems. s158 00:14:44.399 --> 00:14:47.680 You're going to want playbooks for common development practices. s159 00:14:47.680 --> 00:14:51.040 So how to add a feature to your CLI. s160 00:14:51.040 --> 00:15:00.079 You're going to want to find a way to identify repeated tasks and automate them and ideally as quickly and easily as possible because otherwise people won't do it. s161 00:15:00.079 --> 00:15:08.079 And then you're going to want something that looks at the output quality just on a consistent basis and brings back improvements and learnings to the rest of your codebase. s162 00:15:08.079 --> 00:15:11.680 So say things like, oh, I saw the agent made this mistake. s163 00:15:11.680 --> 00:15:16.160 Let's update a skill on add like the playbook you have for adding features to the CLI. s164 00:15:16.160 --> 00:15:20.880 Let's update that so it doesn't make this mistake again in the future. s165 00:15:22.000 --> 00:15:27.120 So those three control planes Tessle sort of exist to try and help you solve them. s166 00:15:27.120 --> 00:15:30.320 As I mentioned, everything I've said before here are just techniques. s167 00:15:30.320 --> 00:15:32.399 You could go do them all yourself if you wanted. s168 00:15:32.399 --> 00:15:34.320 There's plenty of tools that do that. s169 00:15:34.320 --> 00:15:40.160 So I'm going to talk a little bit through how Tessle will help you if you want to use us. s170 00:15:40.160 --> 00:15:42.800 To start, why would you want to use us? s171 00:15:42.800 --> 00:15:50.399 So we focus really hard on making it relatively iterative and sustainable to get to the cutting edge. s172 00:15:50.399 --> 00:15:58.399 So we try to make it like a bunch of small lifts rather than one massive snake eating the moose all at once. s173 00:15:58.399 --> 00:15:59.600 We're batteries included. s174 00:15:59.600 --> 00:16:10.720 So we solve the knowledge gap by saying we will keep up and you will have an agentic experience that basically knows the best practices of harness engineering is kept up to date on your behalf. s175 00:16:10.720 --> 00:16:13.199 We have a real commitment to being modular and open. s176 00:16:13.199 --> 00:16:18.320 So we don't think every component of a factory is going to be bestin-class with a single company. s177 00:16:18.320 --> 00:16:23.519 We want to make sure you can pick and choose what is the most important piece for your stack. s178 00:16:23.519 --> 00:16:25.600 And then like I said, we focus on making it easy. s179 00:16:25.600 --> 00:16:31.199 So we have automated loops for you that you can install and just react to the changes that are proposed. s180 00:16:31.199 --> 00:16:39.040 And we really try to make sure it's just incremental steps so that you six months later like, oh wow, we're like 40% there to a software factory. s181 00:16:39.040 --> 00:16:43.199 We never had to stop and delay shipping or anything like that. s182 00:16:43.839 --> 00:16:47.920 So the first place that Tesla helps is in setting up your control plane. s183 00:16:47.920 --> 00:17:02.079 We have a skills registry where you can publish and version the workflows and the automations that you're working on with built-in governance things like security reviews, quality reviews, controls for who can publish and update what. s184 00:17:02.079 --> 00:17:06.559 We also have easy connectors for issue trackers to GitHub. s185 00:17:06.559 --> 00:17:08.959 Right now we're focused on linear to GitHub. s186 00:17:08.959 --> 00:17:15.439 So you can basically file a linear ticket and get it to run a GitHub workflow uh with more connectors coming soon. s187 00:17:15.439 --> 00:17:18.959 Uh and then we also have a suite of tools for code review. s188 00:17:18.959 --> 00:17:25.039 So this make it easy to set up your standards for agentic code review as well as more targeted checks. s189 00:17:25.039 --> 00:17:37.360 See here an example of something that like a workflow that has been posted to the registry being scanned for security quality uh and how much it actually improves the agents output. s190 00:17:38.080 --> 00:17:40.960 After this, I'm going to play a video while we talk. s191 00:17:40.960 --> 00:17:41.760 See how this goes. s192 00:17:41.760 --> 00:17:52.160 So, uh, we have an agentic experience called Tesla agent, which will help you with creating and then maintaining your improvement loops over time. s193 00:17:52.160 --> 00:18:02.559 So, the first thing that we do is we will help you with that change management of like how do I actually make the time to find repeated tasks and s194 00:18:02.559 --> 00:18:03.840 set them up in automation. s195 00:18:03.840 --> 00:18:12.400 So the agent will mine through PRs, issues, things like that and find like every week you do a hunt for flaky tests. s196 00:18:12.400 --> 00:18:18.000 Why don't we just set that up as a skill for the workflow and then we'll put it in automation on a GitHub action. s197 00:18:18.000 --> 00:18:23.120 So we help you actually make time to do the automations just like one workflow at a time. s198 00:18:23.120 --> 00:18:27.039 The second is that we come with a lot of out ofthe-box maintenance tasks. s199 00:18:27.039 --> 00:18:38.160 So I mentioned you want to have weekly scans for things like architecture quality, code duplication, uh how good are your test suites, do you have any security vulnerabilities? s200 00:18:38.160 --> 00:18:47.039 So if you use Tesla, you can just sort of like oneclick install a bunch of those things and just get weekly improvements to your codebase without any other effort. s201 00:18:47.039 --> 00:18:51.360 And then finally, we offer a way to turn any skill into an automated workflow. s202 00:18:51.360 --> 00:18:58.720 So it's through a command called Tesla launch where you effectively take a sk a skill that codifies a workflow. s203 00:18:58.720 --> 00:19:02.799 You can then pick a particular coding agent that you want to handle it. s204 00:19:02.799 --> 00:19:04.160 Tesla agent can be one of them. s205 00:19:04.160 --> 00:19:08.400 But for most coding tasks you probably want to use something like codeex, cloud code, gemini. s206 00:19:08.400 --> 00:19:14.000 We have access to all of them and we will run them in a sandbox that has the appropriate permissions. s207 00:19:14.000 --> 00:19:15.280 Can be long running. s208 00:19:15.280 --> 00:19:20.400 Uh we'll have GitHub token so that it can put up a PR and respond to comments as you leave them. s209 00:19:20.400 --> 00:19:24.640 Uh, so it just makes it really easy to automate workflows like that. s210 00:19:24.640 --> 00:19:33.919 And so with these things together, we just make it really easy to get started and just one at a time find an improvement, take a workflow, turn it into a skill, s211 00:19:33.919 --> 00:19:38.799 ship it to everyone, and then get your improvement loops going. s212 00:19:39.200 --> 00:19:47.039 The what you're going to want to focus on as you're doing this work is driving down manual takeovers, driving down human PR comments. s213 00:19:47.039 --> 00:19:52.160 So these are like metrics that you can track to see like how far am I towards software factory. s214 00:19:52.160 --> 00:19:55.760 And then over time you want to see more PRs initiated without human input. s215 00:19:55.760 --> 00:19:58.080 That's a sign that you're automating more. s216 00:19:58.080 --> 00:20:05.360 And then of course you want to first hold quality constant and then aim to drive it up after that. s217 00:20:05.360 --> 00:20:05.760 That's it. s218 00:20:05.760 --> 00:20:08.640 If you want to learn more Tesla booth is just that way. s219 00:20:08.640 --> 00:20:10.640 Please give a scan, come over, see us. s220 00:20:10.640 --> 00:20:11.840 We'd love to chat. s221 00:20:11.840 --> 00:20:14.960 Um thank you all for your time.