WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=whue9_YquGA NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/whue9_YquGA.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:00.560 --> 00:00:08.040 So, I've spent the last couple of years transforming Block's 3,500-person engineering org into an autonomous one. s2 00:00:08.040 --> 00:00:14.000 And this is a challenge that most tech companies are actively trying to solve or will be in the very near future. s3 00:00:14.000 --> 00:00:18.280 So, today I'll share our path to getting there. s4 00:00:18.280 --> 00:00:20.920 Our agentic coding journey started very early. s5 00:00:20.920 --> 00:00:27.440 We were building Goose, our internal coding agent, even before the LLM supported tool calling. s6 00:00:27.440 --> 00:00:35.520 We worked with Anthropic as design partners for the initial release of MCP, and Goose became the reference implementation for the MCP client. s7 00:00:35.520 --> 00:00:44.840 So, internally, our most curious engineers were some of the very first in the industry to use these types of coding agents. s8 00:00:44.840 --> 00:00:53.080 After a couple of months, about 90% of our engineers were now using tools like Goose and Claude Code regularly to generate code. s9 00:00:53.080 --> 00:00:57.360 So, on paper, it looked like we were all in on AI. s10 00:00:57.360 --> 00:01:00.640 But our CEO was convinced that engineering wasn't using AI at all. s11 00:01:00.640 --> 00:01:02.240 Like, they couldn't be, right? s12 00:01:02.240 --> 00:01:07.160 As far as he was concerned, because we weren't shipping any faster. s13 00:01:07.160 --> 00:01:10.560 Well, I had the numbers, both the metrics and the token bills. s14 00:01:10.560 --> 00:01:15.360 So, I knew that engineering was, in fact, using AI, but he was right. s15 00:01:15.360 --> 00:01:18.760 Features certainly weren't making it to our customers any faster. s16 00:01:18.760 --> 00:01:22.160 So, I began to dig into this a bit. s17 00:01:22.160 --> 00:01:28.600 I think of AI enablements in three phases: experimentation, adoption, and impact. s18 00:01:28.600 --> 00:01:43.400 I'd say that we surpassed the experimentation phase as 90% of our engineering org was using AI, but they still were kind of using it inside of their IDE, you know, asking questions or writing boilerplate code. s19 00:01:43.400 --> 00:01:51.920 And I knew that if we really wanted to see impactful outcomes, then we needed a way to integrate AI into how we built and ship. s20 00:01:51.920 --> 00:01:58.080 I spent the first half of 2025 leading AI enablement for our entire company. s21 00:01:58.080 --> 00:02:04.400 So, 12,000 employees across every function, marketing, design, finance, legal. s22 00:02:04.400 --> 00:02:09.920 Now, our CTO tasked me with building an agentic engineering org. s23 00:02:09.920 --> 00:02:11.520 Okay, sure. s24 00:02:11.520 --> 00:02:13.200 But, what does that actually mean? s25 00:02:13.200 --> 00:02:15.760 There's no playbook for any of this stuff, right? s26 00:02:15.760 --> 00:02:22.000 And I know because I went looking at your blogs hoping that you all had it all figured out. s27 00:02:22.000 --> 00:02:26.800 But, I only saw a bunch of posts saying how you all were making it up as you went along. s28 00:02:26.800 --> 00:02:29.520 So, I did the same, right? s29 00:02:29.520 --> 00:02:40.720 In the simplest of terms, I defined an agentic engineering org as one where engineers leverage AI agents as their primary means of producing engineering outcomes. s30 00:02:40.720 --> 00:02:47.080 So, that meant that our engineers needed to treat agents as core members of their engineering workflow. s31 00:02:47.080 --> 00:02:53.600 Not just using AI to help them write code here and there, but actually working with agents. s32 00:02:53.600 --> 00:02:59.519 Decomposing problems, delegating work, reviewing and verifying what was done. s33 00:02:59.519 --> 00:03:03.640 And we wanted them directing the work as their default way of operating. s34 00:03:03.640 --> 00:03:07.959 But, of course, the vast majority of our engineers were not here. s35 00:03:07.959 --> 00:03:12.040 So, to see where we needed to go, I came up with the maturity model. s36 00:03:12.040 --> 00:03:16.519 This model measures the engineer's relationship with AI agents. s37 00:03:16.519 --> 00:03:20.040 So, how they think and delegate and orchestrate. s38 00:03:20.040 --> 00:03:30.360 And while I had some form of this model in Q3 of last year, Steve Yegge's Gastown article helped me reorganize it into a better model. s39 00:03:30.360 --> 00:03:35.680 So, stage zero is where the engineer doesn't use AI tools in their workflow at all. s40 00:03:35.680 --> 00:03:43.200 Stage one, they use AI to auto complete, but they've never used it in agent mode, maybe. s41 00:03:43.200 --> 00:03:49.080 Uh stage two is where engineers are chatting with agents, but not using it to produce any PRs. s42 00:03:49.080 --> 00:03:54.480 Stage three, engineers are delegating tasks to agents and reviewing the output. s43 00:03:54.480 --> 00:03:57.640 Those at stage four are running multiple agents in parallel. s44 00:03:57.640 --> 00:04:09.520 And then stage five is that final boss where engineers are delegating complete tasks to agents and the agent is able to produce shippable results without the human necessarily needing to guide it. s45 00:04:09.520 --> 00:04:15.880 So, I'd say by the end of the first half, the bulk of our engineers were between stages one and two. s46 00:04:15.880 --> 00:04:18.640 And I needed to get them to stage five. s47 00:04:18.640 --> 00:04:27.520 Now, I wasn't quite sure how to get 3,500 engineers to that level, especially when one, this is all highly experimental, right? s48 00:04:27.520 --> 00:04:30.919 There's no playbook, again, for any of this. s49 00:04:30.919 --> 00:04:40.400 Um two, things were moving so fast that what might be a best practice this week could be outdated when a new tool or a new model drops next week. s50 00:04:40.400 --> 00:04:43.919 And honestly, this was leading to AI fatigue. s51 00:04:43.919 --> 00:04:52.520 And then three, people were already feeling turned off by the top-down pressure from leadership to essentially AI or die. s52 00:04:52.520 --> 00:05:02.880 So, I thought about the 1990 rule where in digital communities about 1% create, 9% interact, and 90% passively consume. s53 00:05:02.880 --> 00:05:06.800 And this maps almost perfectly to how engineers adopt AI. s54 00:05:06.800 --> 00:05:15.240 So, you'll have a small percentage that'll deep dive and start creating agentic patterns and discovering useful techniques for working with agents. s55 00:05:15.240 --> 00:05:20.280 You'll have some that might tweak the agent's MD file here or there. s56 00:05:20.280 --> 00:05:25.560 But most people aren't going to spend the extra cycles to figure this stuff out. s57 00:05:25.560 --> 00:05:35.200 So, I realized that if my AI strategy depends on every individual leveling themselves up, I'm never going to see that broad impact. s58 00:05:35.200 --> 00:05:37.360 So, I leaned into this model. s59 00:05:37.360 --> 00:05:39.480 And I used it to my advantage. s60 00:05:39.480 --> 00:05:50.640 Instead of focusing on all 3,500 engineers, I decided to focus on forming the 1%, the power users, from our most critical teams. s61 00:05:50.640 --> 00:06:00.240 So, I started an AI champions program where I handpicked about 50 engineers across the company who could pioneer what a genetic engineering looks like for their team. s62 00:06:00.240 --> 00:06:03.520 Now, this was not a call for volunteers, all right? s63 00:06:03.520 --> 00:06:07.360 I needed to be strategic about the selection. s64 00:06:07.360 --> 00:06:14.160 I needed engineers who were willing to dedicate at least 30% of their time to investing in AI enablement. s65 00:06:14.160 --> 00:06:24.400 I needed those who wouldn't be discouraged by the non-deterministic nature of AI and give up on it all when it didn't work out of the box, which it often did not, right? s66 00:06:24.400 --> 00:06:34.160 So, I spent the week talking to tech leads and managers to find the 50 engineers who could represent our most critical repositories. s67 00:06:34.160 --> 00:06:43.840 Now, remember at this point, most of our engineers were between stages one and two where they're chatting with AI but not really using it to produce PRs. s68 00:06:43.840 --> 00:06:47.720 So, I wanted to get the AI champions to stage three. s69 00:06:47.720 --> 00:06:49.919 This was about June 2025. s70 00:06:49.919 --> 00:06:53.320 So, the models were decent enough to write a feature for you, right? s71 00:06:53.320 --> 00:06:59.120 But chances were pretty high that the code wouldn't conform to your team's conventions and standards. s72 00:06:59.120 --> 00:07:03.760 So, developers didn't trust the agents enough to delegate work to it yet. s73 00:07:03.760 --> 00:07:09.480 So, the first area that we focused on was making our repos AI ready. s74 00:07:09.480 --> 00:07:22.680 My theory here was if I can get engineers to embed AI directly into their repos, then not only would the agents perform better but the entire team would benefit, not just the 1%. s75 00:07:22.680 --> 00:07:29.000 And the repos are a central point of reference for every engineer that's contributing code. s76 00:07:29.000 --> 00:07:41.960 So, to make their repos AI friendly, they added assets like contacts and rules files so that agents could reliably understand and navigate the code base as well as contribute to it. s77 00:07:41.960 --> 00:07:55.600 Now, when building the AI champions, I made sure that we have folks from every corner of Block engineering: Square, Cash App, Afterpay, Tidal, across front end, back end, mobile, data, infra, right? s78 00:07:55.600 --> 00:08:05.360 Um we covered repos of all shapes and sizes, the nasty big legacy massive mono repos, the smaller services, mobile apps, right? s79 00:08:05.360 --> 00:08:15.000 And that mix let us pressure test patterns across very different engineering realities so that we could quickly see what actually scales. s80 00:08:15.000 --> 00:08:18.800 As expected, mono repos came with their own challenges. s81 00:08:18.800 --> 00:08:31.120 But my JVM devs were already strong in inheritance patterns, so we set shared contexts and rules at the root, then layered more specific ones at the service levels. s82 00:08:31.120 --> 00:08:36.479 We also learned fast that what worked for web didn't necessarily work for mobile. s83 00:08:36.479 --> 00:08:41.440 So Android and iOS even needed different approaches at times. s84 00:08:41.440 --> 00:08:55.320 So instead of forcing a one-size-fits-all solution, each champion figured out what worked for their repo, and then teams with similar shapes and sizes naturally converged on the same tools and patterns. s85 00:08:55.320 --> 00:08:58.400 And honestly, engineers loved this, right? s86 00:08:58.400 --> 00:09:05.280 We let them choose what made sense for their repo instead of pushing a top-down mandate on them. s87 00:09:05.280 --> 00:09:13.360 So we ended up with a standard set of components that would make a repo AI friendly, but again, customized for each team's needs. s88 00:09:13.360 --> 00:09:28.760 Some context files like agents.md or claude.md to provide the agent with repo guidance, rules files to provide the agent with some guardrails, repeatable workflows like slash commands and later agent skills, s89 00:09:28.760 --> 00:09:35.839 um an enabled AI code reviewer, preferably with some instructions on what matters and what it should review. s90 00:09:35.839 --> 00:09:39.720 And then AI attribution on PRs. s91 00:09:39.720 --> 00:09:48.240 So at this point, we've done a lot of great work and technically, we do have the agents writing PRs, but I had a couple of concerns, right? s92 00:09:48.240 --> 00:09:54.120 Not many outside of the AI champions had gotten to this level where they were delegating work to agents. s93 00:09:54.120 --> 00:09:58.920 And the idea is to have the work of the 1% lift everyone up. s94 00:09:58.920 --> 00:10:01.520 So, we weren't there yet. s95 00:10:01.520 --> 00:10:07.840 And then even with the champions, they were delegating, but they were still kind of babysitting the process. s96 00:10:07.840 --> 00:10:19.080 So, I wanted to explore if we could delegate work to the agents in a more hands-off approach and make it easier for others outside of the champions program to do so as well. s97 00:10:19.080 --> 00:10:23.400 So, there's three common places where engineers receive requirements for work. s98 00:10:23.400 --> 00:10:29.160 Issue trackers like Jira or Linear, GitHub issues, and informally in Slack. s99 00:10:29.160 --> 00:10:33.400 I wanted to be able to delegate work to the agents from any of these places. s100 00:10:33.400 --> 00:10:36.200 And the champions implemented all three. s101 00:10:36.200 --> 00:10:38.520 So, true story. s102 00:10:38.520 --> 00:10:44.200 We were in Slack one day, an engineer saw a bug with the product and wrote, you know, like, "Hey, I saw this bug. s103 00:10:44.200 --> 00:10:45.640 Anybody else seen it?" s104 00:10:45.640 --> 00:10:51.800 So, engineer two responds to that and goes, "Mhm, no, I hadn't seen that one." s105 00:10:51.800 --> 00:10:58.640 Engineer three @goose right there in Slack and says, "Hey goose, have you seen this bug before? s106 00:10:58.640 --> 00:11:01.480 Like, can you can you go and check if this is a bug?" s107 00:11:01.480 --> 00:11:06.160 So, goose goes to the repo, it like pulls the files down, and says, "Yeah, this is a bug. s108 00:11:06.160 --> 00:11:07.680 Here it is right here. s109 00:11:07.680 --> 00:11:14.000 But, here's also three options of how you might implement a solution, right?" s110 00:11:14.000 --> 00:11:15.640 Right there in Slack, code snippets. s111 00:11:15.640 --> 00:11:18.680 And so, engineer one goes, "I like option one." s112 00:11:18.680 --> 00:11:20.720 Engineer two goes, "Yeah, yeah, me too." s113 00:11:20.720 --> 00:11:24.080 So, engineer three says, "Hey goose, go implement option one." s114 00:11:24.080 --> 00:11:28.360 Goose does so, returns with a link to the PR, right? s115 00:11:28.360 --> 00:11:37.440 So, the entire cycle from discussion, diagnosis, issue creation, alignment, and the fix took like 5 minutes all right there in Slack. s116 00:11:37.440 --> 00:11:39.560 Very cool party trick, by the way. s117 00:11:39.560 --> 00:11:49.320 Um engineers were also able to assign linear and Jira tickets, as well as GitHub issues to an agent, and have the agent implement the work end to end. s118 00:11:49.320 --> 00:11:53.080 So, now the agents have become a part of the sprint. s119 00:11:53.080 --> 00:11:54.760 This blew everyone's minds. s120 00:11:54.760 --> 00:12:01.480 The first time we did this, the team ran out of work and had to pull in more tickets like twice, right? s121 00:12:01.480 --> 00:12:06.160 So, of course, engineering managers, product teams, they love this flow. s122 00:12:06.160 --> 00:12:12.280 And the beauty about these flows is that all engineers didn't necessarily have to learn a new skill. s123 00:12:12.280 --> 00:12:24.920 The champions had already laid the foundation so that agents were able to work well within these repos, and they worked well all of this worked well because it made delegation feel native to how people s124 00:12:24.920 --> 00:12:27.040 already work. s125 00:12:27.040 --> 00:12:32.560 So, now I felt comfortable claiming that we were truly delegating work to the agents. s126 00:12:32.560 --> 00:12:39.520 At this point, it had been 3 months since launching the champions program, and we were seeing great results. s127 00:12:39.520 --> 00:12:49.520 AI-authored code was up by 69% reported time savings increased 37%, and automated PRs increased 21 times. s128 00:12:49.520 --> 00:12:54.400 We were ready to pursue stage four, multi-agent parallelism. s129 00:12:54.400 --> 00:12:58.600 And with our setup, honestly, moving to stage four was almost free. s130 00:12:58.600 --> 00:13:02.400 We were pretty good at delegating work to the agents now. s131 00:13:02.400 --> 00:13:07.080 But, this stage introduced several new challenges that we needed to solve. s132 00:13:07.080 --> 00:13:15.480 Engineers are now tripling, quadrupling the number of PRs that they're producing, but the PRs are stuck waiting for code reviews, right? s133 00:13:15.480 --> 00:13:23.520 So, people struggled to stay on top of code reviews even before we added AI to the mix, so you know that it's bad now. s134 00:13:23.520 --> 00:13:30.920 And I'll be honest, this isn't a totally solved problem, but I'll share how we did stop the bleeding a bit. s135 00:13:30.920 --> 00:13:34.800 We absolutely had to get the bots to help us review PRs. s136 00:13:34.800 --> 00:13:37.120 So, earlier in the process, this was optional. s137 00:13:37.120 --> 00:13:44.520 Mostly because the AI code reviewers sucked so badly, and we were just pissing the engineers off by having them use them. s138 00:13:44.520 --> 00:13:55.880 But now, with the repo readiness work that the champions had done, coupled with better models and tools, shout out to Codex, uh we were getting much better results here. s139 00:13:55.880 --> 00:14:07.480 So, we enabled Codex on all the repos, and we also created an auto-fix loop where if Codex identified issues, another agent will automatically fix those issues and commit them to the PR. s140 00:14:07.480 --> 00:14:13.640 So, now, at least people would no longer complain about not wanting to review sloppy PRs from bots, right? s141 00:14:13.640 --> 00:14:18.120 By time they got to them, they were in pretty good shape. s142 00:14:18.120 --> 00:14:25.760 Another issue, specifically when running multiple agents in parallel, was that the agents were bumping into each other as they were trying to work. s143 00:14:25.760 --> 00:14:30.040 And more importantly, our machines couldn't handle the load anymore. s144 00:14:30.040 --> 00:14:33.120 Like, engineers' laptops were running out of memory. s145 00:14:33.120 --> 00:14:34.800 CPU was choking, you know? s146 00:14:34.800 --> 00:14:41.880 So, we invested in dedicated cloud workspaces where each agent ran in its own isolated environment. s147 00:14:41.880 --> 00:14:47.560 And this allowed us to easily run them in parallel and from anywhere. s148 00:14:47.560 --> 00:14:50.040 So, at this point, the engineers are cooking. s149 00:14:50.040 --> 00:14:55.560 They got four or five agents running at any given time, and that number is growing. s150 00:14:55.560 --> 00:15:07.120 So, a small group of engineers, including many of the AI champions, started building our own orchestrator to coordinate all of these agents, Builder Bot. s151 00:15:07.120 --> 00:15:13.120 And we needed Builder Bot to get us to an autonomous engineering org. s152 00:15:13.120 --> 00:15:23.280 We realized that if we're trying to build anything close to an autonomous engineering org, the agents need to understand where everything lives and what depends on what. s153 00:15:23.280 --> 00:15:33.800 So, we built a company world modeled based on the entirety of our 25,000 repo code base, right? s154 00:15:33.800 --> 00:15:40.200 And this was a machine-readable view of every single service and how they all connect. s155 00:15:40.200 --> 00:15:50.000 And this allowed orchestrators and the agents that they delegated to to pull context in as needed and understand the landscape as they implement. s156 00:15:50.000 --> 00:16:01.440 And this enabled multiple agents to explore different parts of the system in parallel, each building their own understanding and then come back together and have the orchestrator put all of this together, s157 00:16:01.440 --> 00:16:01.920 right? s158 00:16:01.920 --> 00:16:12.440 Put in a plan that spans across multiple code bases, which was especially useful for us as we were building offerings that spanned multiple products. s159 00:16:12.440 --> 00:16:19.120 Now, with this, we had successfully reached stage five, where engineers are delegating complete tasks to agents. s160 00:16:19.120 --> 00:16:23.600 The agents are able to produce shippable results without human hand-holding. s161 00:16:23.600 --> 00:16:27.320 In fact, this wasn't even limited to engineers anymore. s162 00:16:27.320 --> 00:16:36.120 Anyone at the company could act Build-A-Bot in Slack and have it fix a bug or implement a new feature. s163 00:16:36.120 --> 00:16:38.720 They didn't even need GitHub. s164 00:16:38.720 --> 00:16:42.080 This felt like a dream. s165 00:16:43.120 --> 00:16:45.920 Until it became a nightmare. s166 00:16:47.200 --> 00:16:52.320 You know, of course, all layoffs are tough, but this one felt different. s167 00:16:52.320 --> 00:16:54.440 I had so many questions. s168 00:16:54.440 --> 00:16:56.839 Was this my fault? s169 00:16:56.839 --> 00:17:05.400 Did enabling employees to do the most incredible work of their careers ultimately result in their dismissal? s170 00:17:06.439 --> 00:17:09.680 Just the day before I had felt so proud. s171 00:17:09.680 --> 00:17:21.360 I was in awe of the way we were working, feeling quite accomplished that I successfully built an autonomous engineering But to what end? s172 00:17:21.680 --> 00:17:25.360 I guess I'll leave you all with a few questions. s173 00:17:25.480 --> 00:17:27.560 What are we doing? s174 00:17:27.560 --> 00:17:30.280 Where are we heading? s175 00:17:30.360 --> 00:17:35.520 You know, we sure that it's where we want to end up?