Why Software Factories Fail

AI Engineer · 19 min · 270 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:25What's up everybody?
00:26How we doing?
00:29Guys, give it up for all the great speakers today so far.
00:33Um
00:34[applause]
00:36All right, this is Harness Engineering is not enough and why software factories fail.
00:42And we're going to click, maybe.
00:44Oop.
00:47Oh, that's way too many slides.
00:48Hold on, guys.
00:50Okay.
00:51Um So, we're all racing to put AI coding into production.
00:55And uh there's been lots been said about loop engineering.
00:59And uh we should probably write more loops.
01:01And uh yeah, I don't know.
01:03I guess we're doing loops now.
01:06Uh StrongDM built a lights-out software factory where nobody even reads the code.
01:10And the prevailing narrative is we should just spend more tokens.
01:15You are the bottleneck.
01:17The models are good enough.
01:18Code is free.
01:20Just ship more stuff.
01:22But at the same time, we are starting to see the cracks.
01:26Our friend Mario at AI Engineer Europe begged us to slow down because companies that should not be having outages because of coding agents are having outages due to coding agent mishaps.
01:36Um codebases are falling apart faster than they ever have before.
01:41And our friends at Faros AI actually even did a report since we all picked up all these AI coding tools in January, maybe February.
01:48Um pull request code review quality is way down.
01:51We're having more comments, longer comments, and tons of PRs being merged without any review at all.
01:57Incidents are way up, bugs per developer are way up, and uh many people will tell you that you're holding it wrong.
02:04That's the only reason.
02:05You're not.
02:07Well, maybe you are, but that's not the point.
02:11Um I've spoken a lot about how to hold it better when it comes to working with AI.
02:15Uh probably a million views on YouTube at this point across a bunch of different talks.
02:20Um and the basic thing is like as engineers, we've been told that if token maxing isn't working, then it's a skill issue.
02:27You just need to spend more tokens.
02:28Uh let go of reading the code.
02:31That with enough harness engineering, if we maybe sprinkle some magic words, adversarial review on enough of our uh PR bots, that we can get the best of both worlds.
02:4210 to 100x faster, high quality, and nobody has to do that thing we all hate called code review.
02:48Uh I'm here to convince you today that this is in fact not a skill issue.
02:53That no amount of harness engineering or loops maxing can solve what is fundamentally a model training issue.
03:00That's why we say the harness is not enough.
03:02Um and to understand this, we kind of have to grapple and dig into how coding models are trained.
03:08I'm going to talk about what I think the shortcomings are with some of the current benchmarks and what better ones might look like.
03:13And we'll talk about how to move faster safely in the meantime.
03:17Um it's going to sound like a rant, uh but there is hope here.
03:20Uh I'm going to talk about our journey and a bunch of the landmines we've hit uh building in this world.
03:23A bunch of exciting new techniques that we've been working with uh a lot of our users and customers to develop.
03:28And I think how we all as a community get to the next chapter of agentic engineering after whatever this thing that we're in.
03:36Um so we use a lot of words here.
03:37I'm going to zoom out a little bit.
03:38I want to give you kind of like a brief history of the software factory.
03:41Um and it's actually I don't I I I just learned this last week.
03:44It was the term software factory was defined at a NATO conference in 1968.
03:49Uh we're going to start around 2022, like right before AI started coming around.
03:53Um and basically in a typical 2022 software factory, you will have some people building stuff.
03:59You'll have engineers, you'll have PMs, maybe you have some sort of leadership team that is driving the vision here.
04:04And they all decide that stuff needs to get done.
04:07And so you put it in a tracker, a linear, a Jira, a beads, some sort of state machine that tracks what needs to be done.
04:13And then someone goes and grabs something off of there, and they build the thing.
04:17And there may be some automated testing in that process, maybe some manual manual testing in that process.
04:21At a certain point, we make this pull request thing.
04:24Says, "Okay, cool.
04:25We got to run a bunch of checks, automated stuff.
04:27A human's going to review the change and review the code.
04:29And perhaps we might even have uh a human pull it down and test it somehow."
04:34And if anything goes wrong here, we loop back to someone builds the thing.
04:37Uh and eventually we're ready for prod.
04:39And so we ship it to production.
04:41And once it's in prod, it makes contact with our users.
04:45And users do uh a thing that we all love.
04:48Uh users love to complain.
04:51I I love our users.
04:52Uh but yeah, they're going to ask for things, they're going to find bugs, they're going to file feature requests.
04:56Uh and that goes back to your team.
04:57You might also add monitoring.
05:00And so uh you know, what do we want more than anything else?
05:03We want to wake up engineers at 3:00 in the morning when something breaks.
05:06So they can get dragged out of bed to try to go fix it.
05:08Uh and we go on and on in this loop.
05:10Uh and we ship a bunch of code.
05:13Um and one thing that we noticed here is that uh teams figured this out decades ago is that this someone builds the thing step is usually going to take hours or days in most cases.
05:24And the review part will also take hours or days for large things.
05:28And so teams started doing these upfront planning, architectural proposals, sprint planning, and they would collaborate this on on these things as a team uh with the hopes that we might decrease the percent chance that something would need to be reworked.
05:40That we would be able to reduce the time spent in reviewing every line of code because we aligned on everything ahead of time.
05:47This brings us to the agentic software factory.
05:50Uh every company and their mother is talking about how they built a coding agent factory that ships 75% of their code now.
05:58Uh literally everybody.
06:01Uh and so if we look at the software factory from 2022, uh we just replace someone builds the thing with an agent builds the thing.
06:07And we have an orchestration and a harness and a sandbox and a model and computer use and I'm not going to get into like the details of that.
06:12You can watch 100 talks about that this week, I'm sure.
06:15Um but now the building part takes minutes or hours, but this human part still takes hours or days if you're going to review the code and you're going to test the changes.
06:23And so we bring in agentic code review.
06:24We bring in agentic regression testing, uh and it makes this part faster, but it's probably still the bottleneck.
06:31But we can do more loops here.
06:32Why not?
06:33Let's do some more loops.
06:34So we can route all incidents straight into the factory.
06:36Why does someone need to get woken up uh and try to fix it when they could just wake up to a pull request and uh maybe that fixes the issue for you.
06:43You can take all the user feedback and just stick it straight into the factory so that people ask for stuff and it gets built.
06:49And now your only job is how much things can you stuff into the queue of stuff to do and how fast can you review and test the changes?
06:57Which brings us of course to, I'm sure you know, the lights off software factory where basically Dentsu Bureau coined this is we no longer read the code.
07:05We say, "You know what?
07:06This is going great.
07:07That code review thing?
07:08No, thanks.
07:09We're just not going to do that anymore."
07:11Uh and we invest into all these other parts of the system.
07:14Your your testing, your monitoring, your rollout, everything else.
07:17We just write more code and build those systems better.
07:19And now our job really is just how much how much stuff can we ask the agent to build?
07:25I am going to posit that this does not work.
07:28Uh and this is why software factories fail.
07:31Um as as an aside, what I'm going to say has nothing to do with vibe coding.
07:34So Addy had this uh great post.
07:36I'm going to just going to literally take his quote verbatim.
07:38A developer vibe coding a side project a dozen people will ever run, and a team keeping a 10-year-old enterprise system alive for another quarter share almost no constraints worth naming.
07:49And most of what you hear on the internet is one of these groups of people telling the other group of people how to live their lives.
07:56So, if you love vibe coding, please go on.
07:59Um at Human Layer, what we care about is how do we help people solve hard problems in complex codebases.
08:05Um we use the word brownfield a lot, which historically has meant like some 10-year-old Java thing.
08:10I actually think agents really start to struggle after maybe 3 to 6 months, especially with the pace at which we can ship now.
08:16Um you can ask me how I know this, and I will tell you that it is because in July 2025, we tried this.
08:21We went full lights off, and uh if you have tried this seriously for a number of months, you probably found at least one issue that the agent couldn't solve.
08:31Even with your most advanced prompting, you do research, you do reproductions, you just you have to go and dig into that codebase that you stopped reading 3 months ago to try to figure out what's broken.
08:42And in the meantime, your site was down, your users were pissed, and you were If you were like me, you were probably miserable reading all this slop code that you let
08:50slip into your system.
08:51And what I want to get to is basically models have a shortcoming.
08:55Um they can't maintain and improve codebase quality over time, not without a a decent amount of human steering.
09:01Um and when I say maintainability, I'm basically talking about issues like it becomes really, really hard to make a change in one part of the codebase without breaking other parts of the codebase.
09:10This is Martin Fowler's shotgun surgery, textbook code smell.
09:13Um I'm not going to say much more about maintainability.
09:15There's a bunch of books that you can go read about it.
09:16In fact, John Osterhout is actually here speaking this week, so you can go ask him in person about the philosophy of software design if you want to.
09:23Um but it brings us to this question of like why can't models do software maintainability?
09:28Um and you may also be saying, but Dex, you know, surely the models have gotten much better since then.
09:33Um they've gotten better in some ways, but they're still about the same in others.
09:36Um if you want to solve one-off problems or vibe code a new marketing side, yes, they got way better since 2025 and 2024.
09:44But as far as improving code base quality, I think uh they have not gotten much better.
09:49Now, I cannot prove this because there are no good benchmarks for a model's ability to maintain code base quality, and I'll get into like where we're going with that.
09:57Um, but if you've worked with coding agents for a while, a lot of people are posting about this.
10:01It's just like you probably have this vibe that they they generally make things worse over time and make the code base harder to work in.
10:08And to figure out why this happens, I want to zoom out to the first great coding agent.
10:13Why did Claude Code go from nothing to 4 billion and I think now they're at 9 billion in revenue in under a year?
10:21Cuz they were great CLI agents before Claude Code.
10:23You had Aider, you had Code Buff.
10:25There was a bunch of tools in this category.
10:27They had all the same tools, read, write, edit, grep, bash.
10:30Um, so what was the difference?
10:31The difference was was that the this was the first time that a model lab trained a model against the harness that they were going to distribute it to users in.
10:40Um, and it got really, really good at this is just some of the tools, but it got really, really good at calling these sorts of tools in an agentic loop.
10:47In fact, the OpenAI team did a talk in November about basically if you are a uh harness builder and you don't own the model weights and you can't RL the model in your harness,
10:59you will always be at a disadvantage compared to somebody who owns both the model and the harness.
11:05Um, and I'm going to decide a couple slides from my buddy Calvin French Owen, who was a MTS on Codex during the initial launch.
11:11Um, but LLMs are just next token predictors.
11:13Uh, this is a slide from over a year ago where basically as you're doing your agentic loop, context window goes in, next step comes out.
11:20And uh we're going to try to do this.
11:21I haven't actually timed this, but we're going to see if we can do coding agent reinforcement learning in 60 seconds.
11:26So, what we're going to do if we want to train a model to get better at tool calling, better at solving software problems, we're going to generate a bunch of We're going to give it a problem and we're going to generate a bunch of traces.
11:34Try to solve the problem a bunch of different times.
11:36We're going to score them all on correctness and did the test pass and all this stuff.
11:40Uh and then we're going to reinforce.
11:41We're going to make the bad behavior less likely and we're going to update the weights to make the good behavior more likely.
11:47Um this one of the classic ones here is SweetBench multilingual.
11:50Uh they're about 15-minute tasks.
11:52They're from open-source repos like Redis, JQ, and Django and all this stuff.
11:55And they have binary one or zero rewards on did you fix the problem you were trying to fix?
12:00And did you do it without breaking anything else?
12:02Um and we look at actually a real problem from one of these benchmarks.
12:05This is Fastlane, which is a Ruby project.
12:07Um basically, there was some issue where we weren't checking for nil and we have a stack trace blow up because you have a null pointer exception.
12:13And in this um in this benchmark, you have a base commit that we're going to check out before the issue was solved by a human in the past.
12:20We're going to give it a test patch that says here's what the behavior should be afterwards.
12:25We have a golden patch.
12:26Both these are hidden from the model.
12:28Uh and so we have the agent go try to solve the problem.
12:30We store its patch.
12:32We undo all the changes it made to any test files cuz I'm sure you've seen models comment out tests just to get things working.
12:38And then um we're going to apply our golden test patch.
12:41Uh and then we're going to run the test.
12:43Old test and did the new test pass?
12:44And if they both pass, then uh then we get the reward.
12:47Otherwise, we don't.
12:49Um and so models are trying to get the test to pass.
12:51There's no way in this system that we can penalize it for poor program design or for eroding the maintainability of our systems.
12:57That's why we get things like this.
12:59Try catches around things that probably don't need a try catch.
13:02Or things like this.
13:04I think Bybop gave us this example earlier of casting things to other things just so the model can just just it just wants to get the test to pass.
13:11Um and so if you can't verify the uh maintainability of the code, it gets way harder to train on this stuff.
13:16Um so you remember this picture?
13:18Verifying code quality and maintainability is orders of magnitude harder than the code runs and the test pass.
13:24Because the cost function of bad architecture is measured in months and years.
13:28If you have a coding episode and then you only find out months later that like somebody vibed this a little bit too hard, it's really hard to propagate that reward signal back across the gap.
13:39And now the frontier is getting better, slowly.
13:41And since I know someone's going to be in the YouTube comments about this, yes, I know benchmarks and verifiers are different and they actually have to be separate data sets, but they're shaped the same and the the structure of these benchmarks is directionally correct.
13:52So we're going to look at these as like what is the future of evaluating code maintainability.
13:56Um there's a really cool one called Sweep Marathon from Abundant AI where they do like 400-hour tasks of like clone all of Microsoft Excel, every single feature.
14:04Uh and they have some sophisticated reward channel stuff.
14:07Uh Deep Sweep from Data Curve is also like large tasks on OSS repos that are not actually in the training set cuz they were never actually built in the real world.
14:16Uh and then you have Frontier Code from Cognition, um which is multi-PR tasks.
14:20They do interesting things like hey, if the model writes tests that don't fail on the pre-patch code, then it gets penalized.
14:25And then we have a judge model that says, "Okay, uh did this follow all of our code quality rules?"
14:30Um so we're getting better, but I think models judging quality can only go so far uh cuz if the new model if the model knew what good code looks like,
14:37it would probably write it in the first place.
14:40Uh and review agents and throwing more tokens at the problem, it can raise the floor, um but we're still constrained by what we can teach during RL.
14:48Um and so I will I will posit that for now we're stuck reading the code, uh but we can still move pretty fast.
14:54And of course there's a world where this is solved uh in the future, and if you want to just keep yolowing prompts until you get to GPT-7 and you don't have to think about this, by all means, please.
15:02Uh but bitter lesson be damned, we've got some problems to solve.
15:05So let's engineer our way out of this.
15:07Um so turning the lights back on, we're going to put the code review back.
15:11Uh we're going to embrace this approach of like how do we plan up front to reduce the chance that we have a long or difficult review process.
15:18We're going to find leverage.
15:19We're going to use AI to help with this.
15:21Um the first thing we're going to do is we're going to do some sort of product review, understanding what problem we're solving, what's the desired behavior, maybe looking at mock-ups.
15:28Here's a product review I was working on yesterday with a mock-up of a new feature.
15:32Once we have our product review, we're going to By the way, we don't small stuff still just go straight to the agent.
15:38But once we have the product review, we're going to also do architecture, system architecture.
15:41A lot of people have been doing this for a while, component contracts, data models, constraints.
15:46This is an example of a doc that we build to understand how these systems are going to fit together and what's like the high-level picture of it.
15:52From there, we do something that I think is really under-emphasized in agentic coding these days, which is program design.
15:59I think people assume that once you get the architecture right, the model can just cook.
16:03But we we often look into the types and the method signatures, the program layout and the call stacks.
16:09So here's some examples.
16:10I don't think you'll be able to read this one, but this is like the level of abstraction we're at.
16:13It's how we're actually going to lay this stuff out and how these systems are going to interact.
16:17Dylan Mulroy from Cloudflare talks a lot about how he's using these call graphs as part of his planning process.
16:22I think this is exactly right.
16:25And then once we've done the product program design, we can do this thing called vertical slices, which is the order of implementation, multi-repo coordination, how we're going to build this across our entire system, and how are we going to check it along the way.
16:36I've talked a little bit about how models have horizontal plans.
16:39I won't go too deep into it.
16:40If you want to learn more about this, you can go watch our talk from AI Engineer Miami.
16:45Couple shots of a doc like this going through the tests and the steps in between each phase.
16:50The main idea here is 30 minutes over here in pre-planning and alignment can save you hours in review.
16:56And so it's actually feasible to still read every line of code.
17:01We'll skip this part.
17:03Basically, the the summary here is like you don't have too many PRs.
17:07If you're drowning in PRs, you actually have too many bad PRs.
17:12Because a good PR is a joy to to review.
17:15It's it's you're just reading through it like, "Yep, this is great.
17:17This is what we discussed.
17:18This is what we talked about."
17:20Um but even if a PR needs 20% rework, which is generous for a lot of AI AI vibe coded slop, um it's an It's an emotional and intellectual burden on both the reviewer and the submitter.
17:32Um and so if you use model assistant planning and alignment, your alignment is shorter cuz you used AI to get all the information at once.
17:39Your code review is faster because you aligned up front, and your coding is faster cuz AI did it.
17:44And so you're now you're actually really moving faster, but you're still reading everything and you're still owning the code.
17:49So, closing advice, um is easy to hear all this and be a little bummed out.
17:54Uh I really like the world where we just YOLO everything, and we can just like not have to ever read code ever again.
18:00But uh we're engineers, and these are just constraints, and models are good at certain things, and they're not good at other things.
18:06And so go figure out how to solve problems given a set of constraints.
18:11Uh use loops, they're great.
18:12Go solve hard problems.
18:14Seek leverage.
18:15Um if you want to help with this, um we're building Human Layer.
18:17Human Layer is an AI IDE and collaboration platform.
18:20It's building blocks for your software factory, um and soon to be better vi- verifiers for software quality.
18:27Um we've got sort of a Figma for cloud code and Codex-style collaborative workspace.
18:31It walks you through the workflows for doing this sort of work.
18:35And uh we are talking to design partners.
18:38We are hiring founding engineers here in San Francisco.
18:41And uh these slides are live.
18:43You can go get them right now.
18:45You can try Human Layer at humanlayer.com.
18:47Uh it's free for small teams.
18:49Go solve hard problems in complex codebases.
18:52Thank you all for your energy.
18:54[applause]
19:12[music]
19:16Hey.