"The engineer of the future is the person who chooses what is worth doing." — Addy Osmani, Anthropic

AI Engineer · 18 min · 228 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:20Howdy, folks.
00:23So, good afternoon or good whatever time it is when you're watching this on YouTube.
00:29I'm really excited to be here.
00:31And um today I want to talk to you about really uh what it takes to keep the human in the loop where engineering is concerned.
00:40I really want to start with a human side before we talk about the architecture here.
00:44I think that the engineer of the future is going to be really defined by the person who is able to choose what is worth doing.
00:54They're going to own the evidence.
00:55They're going to own the understanding as well as the verdict.
00:59around increasingly automated work that's being done by agents.
01:04Now, when I use the term verdict, I don't mean that we're suddenly all going to be Judge Judy.
01:10We're not.
01:11But what I mean really is something just a little bit different.
01:15I mean, we're going to be accountable for the production decisions.
01:19Does something ship?
01:21Do we block it?
01:22Do we redirect it or accept the risk?
01:25Quality is something that we all talk about a lot, but quality produces evidence.
01:30A verdict assigns responsibility.
01:33And answerability is really what lets us stand behind a verdict.
01:38And this of course is not the only way that our industry is starting to think about our roles evolving.
01:46Boris Cherney recently put some useful language around what many teams are starting to feel.
01:51The old craft boundaries are getting blurry, and roles are re-bumbling around the work itself.
01:58And the important question here becomes a lot less about what is your title and more what part of the system can you own?
02:06Now, I like this taxonomy quite a lot.
02:10Um it's optimistic without being overly vague.
02:13So, things like prototype, build, sweep, grow, and maintain.
02:17And these are real engineering modes.
02:19Agents are going to help with all of them, but the scarce thing is not merely doing the task.
02:25It's going to be knowing which mode your product needs and what quality bar applies and who owns the result at the end of the day.
02:33Now, we've been talking about harnesses and loop engineering and software factories over the last couple of days.
02:38We can talk why this shift is happening.
02:41We moved past the model as the whole story, right?
02:44With harness engineering, the coding agent is the model plus the harness around it, right?
02:49Your context, your tools, your file system, Git.
02:52And the harness is what turns intelligence into something that you can delegate to.
02:56The next move was loop engineering, where we weren't just prompting one run anymore.
03:01We were designing systems that kept prompting, checking, and remembering, and deciding what happened next.
03:07And that's really when agents [clears throat] started to feel like infrastructure.
03:10And once you start putting all of those things together, you get that software factory.
03:14Dex covered this well in his talk, where you have agents that are running inside that inner loop and evidence that comes out.
03:22Humans still end up making the production decisions in this loop.
03:26And the wind really isn't moving us from it.
03:29The wind is moving human judgments the highest leverage checkpoint, I think.
03:34And this is why it starts to matter now.
03:37AI-generated and AI-assisted code is becoming normal code for a lot of us.
03:42One of Sonar's 2026 survey said that AI-assisted code is no longer marginal.
03:47It's increasingly having a large role in our code bases.
03:51And once that happens, answerability stops being this philosophical world.
03:55It becomes an engineering requirement.
03:57And there's a quality point here as well, right?
03:59Like we used to care about clean code, code that people could read.
04:04But cleaner code is actually not just going to help the next human and the next person on your teams.
04:09It actually helps the next agent.
04:10Another one of Sonar's research uh studies found that clean and messy repos had roughly the same pass rates, but clean code actually used fewer tokens and caused fewer revisits.
04:21So there's a lot of benefit to maintainability that can fuel efficiency for your factories.
04:26Now, making generation cheaper does not automatically make review cheaper, right?
04:31I think a lot of us are facing this moment and we know that engineers are not naive.
04:36The Sonar numbers say that almost everybody is skeptical of AI code.
04:41Now, I love working in my software factory.
04:43I love building my engineering loops.
04:45But the problem is still capacity.
04:48If 96% of people don't fully trust that code, but only about half always verify before committing, we have this danger that we've got distrust without bandwidth.
04:58And so safety comes from making verification cheaper, clearer, and harder for people to skip.
05:04And if you zoom out from the individual reviewer to the organization, review and validation start becoming a bottleneck when governance isn't able to catch up and adoption is already moving way faster than any company can go and set their policies.
05:18And this means that we have some hard questions we have to deal with, like did a model actually touch this file?
05:23And the hard questions are also like what constraints guided that work?
05:27What evidence was produced?
05:29What risk was accepted and who owned the result?
05:33Now, the agent can ship more than any of us can review, right?
05:38So what are we still good for?
05:40I think it's a question that's on a lot of our minds, right?
05:43And you know, if Homer Simpson's experience automating computers can teach us anything, maybe this is our future.
05:51I don't think it is.
05:52But it's one direction things could take.
05:55Now, let's try that again.
05:56If change is where humans enter the loop, if generation scales faster than comprehension, the scarce resource becomes judgment that's backed by evidence.
06:06So, the question is no longer how much can the agent do, but where does human judgment still create leverage?
06:13Now, I want to talk to you about two terms that I'm going to use for the career part of this talk, alpha and decay.
06:22Alpha is the gap between what you can do today and what current models can do.
06:28That gap is a very real thing, and decay is the clock on that gap.
06:34If the thing that makes you special is a capability, the frontier is eventually going to come for it.
06:41Right?
06:42And there's a whole conversation around this.
06:44This is one of the reasons why taste keeps coming up.
06:47Paul Graham had a point here that I think is very right.
06:49When anyone can make anything, choosing what to make becomes very important, and I buy that.
06:55But I also think that we have to be very careful because taste can become a magic word for whatever part of the work we don't want to explain just yet.
07:05Mitchell Hashimoto gave us a more useful version of this definition.
07:09Taste is the ability to make high-quality qualitative judgments where no objective metric exists yet.
07:16That matters because it puts taste before the benchmark and before the market has fully voted.
07:22When you try out a model and you see the kind of UX and the kind of experiences that it builds, you can often tell when you think it has taste or lacks taste or where there's a gap there that humans can fill.
07:34Now, this is also only useful if we can turn some of this concept around taste into critique, examples, and better judgment over time.
07:43So, yes, taste matters when production gets cheaper.
07:47And if anyone can generate 10 options, the scarce skill is really knowing which option deserves to exist.
07:54But taste is not some eternal moat.
07:58It's alpha as well.
08:00Now, the people with taste are still going to matter.
08:03I personally think they're still going to matter for a long time.
08:06But the best version of that skill is not mystique.
08:11It's making better calls and leaving behind examples that your team and the system can learn from.
08:17Now, let's apply the decay test.
08:19Well, we used to have speed.
08:21That decayed.
08:22We used to have recall.
08:24You know, harnesses have memory.
08:26Verification is moving into harnesses, evals, static checks, and model critique.
08:31Taste, I continue to think this is going to decay much more slowly, but it still resets as models learn from examples and preferences.
08:38Even judgment in some ways is a slope rather than a wall.
08:41So, the strategy is not to cling to any one capability.
08:44It's for us to keep moving our edges up a level.
08:48So, this is one of the reasons why what can the agent do is not the best strategic question anymore.
08:54The list of things that agents can't do just keeps shrinking.
08:57The better question for us is really what can only a human be answerable for.
09:03Not because, you know, any of us are are magical in any way, but because some decisions actually require ownership.
09:10They require context, risk acceptance, and responsibility after that work ships.
09:16This is why the word engineer has to get just a little bit stricter.
09:21More people than ever can now make computers do things, and I think that's truly awesome.
09:26The total addressable market for builders has never been larger, and that's so cool.
09:31But it's a huge expansion of leverage.
09:33An engineer is not merely somebody who can code, you know, and and get things to exist.
09:39An engineer can reason about systems.
09:41They think about constraints, you defend trade-offs, you can manage risk, and you're the person that can be reached out to when things start to break.
09:48So, what are things that engineers should avoid if we want to stay effective and accountable in this moment?
09:54Well, the first thing to avoid really is cognitive debt.
09:59Now, cognitive debt is the erosion of your understanding and memory around how to solve problems.
10:05I think a lot of us start to feel this the more that we're using agents every single day.
10:09I know that I feel this a lot.
10:11And it's because we're deferring more and more to AI to solve our problems.
10:15For code, it's the gap between how much code exists in your repo and how much any human on your team genuinely understands.
10:23And this is why things like delegation debt end up mattering.
10:26You can have a build that passes, you know, your tests, a PR that you can merge, but your team can still end up losing its ability to actually explain the system that they are shipping to production.
10:38Now, a very real pressure is all much is also how much we delegate.
10:41So, agents can now stay inside the system long enough for the human to lose the thread.
10:47So, a 30-second run, right, can feel like an interaction, but an hour or a day-scale task, so something long horizon, that's a work stream.
10:56And when tasks can end up, you know, lasting that long, especially when you begin running many of them in parallel, review can't just be a glance at the end.
11:04It has to become a whole control system.
11:07The second thing to avoid is cognitive surrender.
11:11Now, this is when you blindly accept AI's um responses.
11:15Like, delegation is important cuz delegation says, "Do the work, then show me enough evidence that I can judge it."
11:22I still make a judgment in that situation.
11:25Surrender is really saying, "Hey, your answer is now my answer before I have formed any opinions myself.
11:31Now, um Wharton did a study that kind of offers us a warning light here.
11:35When AI was wrong, 73% of people still thought that they they you know they picked the wrong answer and they felt more sure.
11:44So, the failure mode is not using AI, but it's borrowed confidence.
11:50The third thing to avoid is orchestration tax.
11:53Now, if you've been in the Bay Area, you will see people who for better or worse are still walking around with their laptops open or talking to you about cloud agents.
12:01And we're increasingly trying to run more and more and more in parallel or telling each other that we're shipping with hundreds of agents or thousands of agents.
12:09More AI agents running does not mean that there is more of you available.
12:14Your cognitive bandwidth does not parallelize.
12:18So, every loop that you create ends up causing more decisions to route, merge, verify, and integrate.
12:25And the fix is not necessarily fewer agents, but it's about designing your attention like a system.
12:31Like where you enter, what you require, what you reuse.
12:34You just want to be very intentional about it.
12:37Now, accountability can be a scary word for a lot of people.
12:42And I wouldn't be surprised if it made you want to go hide in the bushes and just tell your agent to deal with it.
12:49But, accountability is not what remains after agents get good.
12:52It's what lets the rest of the whole system scale.
12:56If agents can do more work, if they can do it faster and parallel, better than what many of us could do, the scarce thing becomes the ability to explain
13:06intent, to inspect evidence, to accept risk, and improve the system when the decision was wrong.
13:14Now, here is the career math.
13:16The half-life of an edge might be one model release.
13:20Speed, recall, verification, even taste all move as the frontier moves.
13:25But, the half-life of a signature, your credibility, your expertise is much longer.
13:32And by signature, I really mean the name on the work, the person, the team, the institution, whoever stands behind what's actually shipped.
13:41So, skills can earn leverage, accountability can turn leverage into trust.
13:45And this is one of the lines that I want to draw pretty clearly.
13:48Agents can choose, they can route, they can merge, they can escalate, they can operate inside policy.
13:54And in many systems, you know, they can, they should.
13:57But execution and responsibility are very different things.
14:00The agent can follow your runbook, but it can't inherit the consequences.
14:04When something fails, the question is, who understood the policy?
14:08Who accepted the risk and who owns the blast radius?
14:11High agency is something that a lot of us talk about these days as being like this thing that we're looking for when we're hiring.
14:18High agency is actively taking ownership of your outcomes.
14:21So, knowing when to delegate, when to inspect, when to stop, and when to put your name on the result.
14:27High agency in this world is not I personally do everything.
14:31You know, that version doesn't really scale.
14:33It's not just hustle theater, but it's ownership with judgment attached.
14:38This agency ladder tries to make that a little bit more concrete.
14:42At the bottom, you've got someone that flags a problem and leaves it for the system.
14:47Higher up, they execute, diagnose, propose, recommend, and resolve.
14:52And the rare top movement is discernment.
14:55You know, maybe you find a problem and you decide whether or not it's worth investing in.
14:59Maybe it's not, and maybe you move on.
15:01But when agents make more paths possible, agency is not chasing every single path.
15:07It's really just deciding which paths deserve your ownership and attention.
15:12So, translate that into an operating model.
15:15Agents can run much more of the inner execution loop.
15:18They can investigate, implement, test, and report.
15:20I think that there's leverage in that, but that outer loop is still engineering.
15:25So, deciding, verifying, approving, owning, that inner loop is capability.
15:30The outer loop is agency.
15:33And this is a boundary that I really care about.
15:35Your agent returns evidence.
15:37It returns diffs, tests, logs, rationale, traces, trajectories, screenshots, whatever the work itself requires.
15:46But then the engineering really begins.
15:48We decide whether the work was worth doing.
15:51We verify whether the evidence is enough, and we approve or redirect or own what reaches production.
15:57It doesn't matter if you're someone that's just working with a small number of agents, or whether you're working with thousands of agents.
16:03I still very much think that these ideas apply.
16:06So, the boundary is not human looks at AI output.
16:09The boundary is evidence and responsibility.
16:13So, here's an operational rule.
16:15Explain it or don't ship it.
16:17And it's not because humans have to type every line or read every line, but because someone has to understand the work well enough to defend it.
16:25If you've ever worked in a large code base or an enterprise code base, some code bases have this concept of an owners file or certain subdirectories where there are people who are on the hook for that part of the system.
16:36You can think about this in a very similar way.
16:38Who is accountable for that part of your architecture and your code base?
16:42Your model might write the code, and the question is really still whether you can explain those changes that the agent is shipping, whether you've got the evidence where you understand the risks.
16:52Now, this is one of the things I want you to remember near the end.
16:54Automation moves the floor for all of us.
16:58Engineering continues to move up a level, and our new work might be loop design, evidence design, and brownfield stewardship, but fewer keystrokes doesn't mean less engineering over the next few years.
17:12It means that there is more surface area that needs taste, verification, ownership, and ultimately care.
17:20I don't think I've ever been more excited about the future of this field.
17:25Every time that we have made it easier to write software, we've predicted that the world would need less of it.
17:31And in fact, the opposite happened.
17:33Higher-level languages happened, frameworks, cloud, low code.
17:37The pattern always went the other way.
17:40And when you lower the cost, latent demand ends up appearing.
17:43Those ideas that people didn't think were feasible to build and get out there are suddenly unlocked.
17:48And agents are going to do the same thing for a lot of people.
17:51It's not going to remove engineering work.
17:53It's going to move the bottleneck from can we build this to should this exist and can we answer for it?
17:59So, build the factories, keep the lights on, own the verdict.
18:03I hope this was useful.
18:05Thank you.
18:23[music]