Loop Engineering from First Principles — Kyle Mistele, HumanLayer

AI Engineer · 17 min · 227 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:27Hey everybody.
00:28My name's Kyle.
00:29I'm co-founder of a company called Human Layer.
00:32And I'm here to talk about loops.
00:34I think we've all been building loops lately.
00:36And I realized recently, I think we're all doing it wrong.
00:41Loops are really powerful, don't get me wrong.
00:43But so much of the discourse around them is hype driven and just really not helpful, right?
00:48We I think we have this idea just kind of as an industry somehow that we can like pipe a prompt in a loop to a coding agent and that we can build software this way, right?
00:58Maybe we're investing a lot of time in verifiers.
01:02Maybe you have six different code review agents.
01:04But at the end of the day, if we're doing this, we're still building 40,000 line PRs that just nobody wants to read, right?
01:12And this isn't to throw shade at Jeff Huntley, right?
01:15This is Ralph is an innovative uh it's a sharp tool that works very well for certain types of problems.
01:22It works very well if you're not building on a team.
01:25And it works very well if you're not working on critical systems.
01:28But most of us are working on teams and we don't fit in that box.
01:33So today I want to talk about how to build loops that work in large complex code bases.
01:38For systems that have real customers, real users, real regulatory obligations and service level agreements and everything else that keeps us from shipping YOLO 40,000 line PRs straight to production.
01:51In other words, I want to talk about how to build loops for the real world.
01:55If you're not aware, uh this post actually dates back to July.
01:58It went viral this past January, which is when a lot of us I think started building loops.
02:03And of course much more recently, I'm sure you all are going to see this slide a lot this week, but Peter Steinberger said that we shouldn't be prompting coding agents anymore, right?
02:11We should just be designing loops that prompt our agents.
02:14Of course, open claws notoriously built on loops.
02:18Loops build the code, loops review the code, they merge and release the code, they find and fix the bugs.
02:23There's even loops for finding and fixing bugs in the loops that are merging the things, right?
02:29It's loops all the way down.
02:31It's Boris Turney, also the creator of Claude code, recently said that this is his entire job as an engineer now is just writing loops to prompt Claude.
02:41And eventually, we might not even need loops, right?
02:44We're just going to have like swarms of agents designing loops to prompt agents building swarms for loops and like I don't know, somewhere where like writing production code, what I assume.
02:56And in fact, all of our loops that we're building are producing so much code that we can't possibly read all of it, right?
03:03So we might as well just not read any of it, right?
03:06We're we're investing in verification and in code review, but all this code is read only.
03:10This is the thesis of a conference that was here in town last month.
03:14So a lot of smart people at Frontier Labs think that this is the future of software development.
03:19And if you're doing this, you're moving 10x faster and everybody else is getting left behind.
03:26Now, it's not clear how well this works yet.
03:29Uh Took 6 months to fix the Claude code terminal flicker.
03:32The open code team wrote a renderer in a fraction of that time.
03:36And open claw of course also notoriously has stability issues.
03:41What is abundantly clear, however, is that this is really expensive if you don't work at a Frontier Lab and have an unlimited token budget.
03:50And all this code that we're writing is actually really expensive, right?
03:54Matt Pocock talked about this recently.
03:56Uh bad code is much more expensive in the age of agents than it it has ever been at any point in the past.
04:03So, today I want to talk about what I think works in the real world and what we started doing at Human Layer, which to be clear is still building loops, right?
04:12I think loops are super powerful, but we can design loops and still read the code.
04:17In fact, we can design loops to make it easier to read the code because the loops are making the code better.
04:23We can solve hard problems in complex codebases with loops, and we can build our software factory incrementally.
04:29But uh to do this is going to take some real engineering, y'all.
04:33So, let's talk about control theory.
04:35Control theory is all about how we drive a dynamic system, which would be your codebase, towards some desired, stable, or optimal end state, right?
04:43You have a sensor that measures the current state of the world.
04:47You have your set point, right?
04:48The desired state of the world.
04:50And the difference between those two things is your measured error.
04:53You have a controller that reads that measured error and turns it into a control signal about an incremental change to apply to the system.
05:01We have an actuator that applies that change to the system, which is undergoing disturbances in the meantime.
05:07And then we remeasure, recompute our measured error, and we're back where we started.
05:13Now, this sounds really complicated, and it can be.
05:16I have a twin brother, actually, who's an aerospace engineer.
05:18This is how they keep fighter jets from falling out of the sky.
05:21Uh but uh it's probably a little bit simpler than most of y'all think.
05:25Does anyone have one of these?
05:28Uh a thermostat uses a control loop, right?
05:31Uh for for any of our European friends in the audience, this part of something we have uh here in the States.
05:37It's called air conditioning.
05:41And uh most of us probably actually use control loops on a daily basis, right?
05:46Kubernetes auto scaling systems are built on control loops.
05:50Infrastructure as code uses a desired state, current state, iterative change like control loop pattern.
05:58Postgres is auto vacuum and React's virtual DOM both use approximate control loops.
06:03Control loops are ideal when we have a system that we want to change, a problem we can measure, and a way to get feedback on the result of that change.
06:11Like good software engineers have always been taught to do, control loops change a system incrementally instead of just trying to get straight to the end state immediately all at once and risk blowing everything up, right?
06:23They help us to avoid over steering and destabilizing the system and it minimizes risk.
06:29So, control loops are the opposite of what I'm going to call a blind Ralph loop.
06:34They're how we avoid PRs that look like this cuz nobody wants to review this, right?
06:39Which is not to say that all Ralph loops are blind loops.
06:42The best Ralphs are actually applying control theory.
06:45I know Jeff Huntley is out in the hall somewhere wandering around.
06:47If you go talk to him, he's going to tell you the same thing, right?
06:50That Ralph is a teaching device and I think some of us read it a little too literally, but this is how we should have always been building loops.
06:59But, the other issue with Ralph loops they're not incremental, right?
07:02It's just a bash loop.
07:04So, we have to build agentic control loops.
07:07And to do that, we start by defining a set point which is the desired end state of our code base with respect to some property of it.
07:14And we add a sensor.
07:15There's a lot of ways to build a sensor.
07:16It can be strictly deterministic, your ESLint rules, your AST grep, your packwork, or it can be non-deterministic.
07:25You can have an agent and a skill and a bunch of natural language rules, and you could also just have a pipeline like a combination of the two.
07:33So, how do we build agentic Oops.
07:36There we go.
07:37Now, this is all theory, right?
07:40Practically speaking, and because we're using agents, we can blur the lines a little bit between system components.
07:45So, Aiden by React Doctor, for example, is fantastic.
07:49It is uh it's a great way to catch all of the React slot that Claude snuck into your code base last week.
07:56But uh it's a hybrid sensor and controller.
07:58It tells you what are all the problems with your React code, and also, by the way, what are the top three things you should fix, and how do you fix them?
08:05Similarly, our controller and actuator might actually just be a single agent deciding on an incremental change to make and then applying it in the same context window.
08:14But I want to zoom in on the controller a little bit because without one, or without a well-tuned one, we might make too large of a change all at once, or we might make the wrong change entirely.
08:25And if you put that in a loop, you're in trouble pretty quickly.
08:29So, we can use control loops to root out bad patterns and to clean up our code, but we can actually use them for all sorts of things, right?
08:35We could make sure that our API is compliant with someone else's open API spec.
08:39We can make sure that our MCP server is compliant with whatever version of the uh the MCP specification that we're currently on.
08:48Haven't checked.
08:49You could mirror a project from Python into HypeScript, or vice versa.
08:54You could even maintain your uh Beat-based slot fork of Next.js against the upstream.
09:00The key questions are, can we find something we can measure?
09:03Can we apply changes incrementally?
09:05And can we get feedback on the quality of those changes?
09:08To illustrate that, I'm going to walk through a control loop that we use internally at Human Layer.
09:13Uh for our loop, we are incrementally migrating our RPC API to Effect.
09:18We adopted it for some of our race-prone code.
09:20We like it, so we're adopting it across the rest of our code base.
09:23If you've never seen Effect's code before, the code on the right is just the kind of trivial procedure on the left rewritten in Effect.
09:29Uh the syntax is really weird.
09:30We're psychos.
09:31We really like it.
09:32It's not for everybody.
09:33That's okay.
09:34Uh this isn't a talk about Effect, so we'll keep moving.
09:38Ooh, clicker's not working.
09:40Cool.
09:41So, step one, we have to build our sensor to find unmigrated procedures.
09:45We can have an agent do this or we could use grep or rip grep.
09:50But, instead we're going to use AST grep cuz it's really powerful.
09:52It's a great tool to have in your toolbox for building loops.
09:54It's language-agnostic.
09:56It's out of band from your TypeScript config or ESLint rules, which if you're a TypeScript developer, you have watched Claude disable those with inline comments.
10:04Uh but, so we can just write a simple rule that finds unmigrated procedures uh based on the pattern above.
10:10We Over time, we can even layer on more rules that describe other patterns we want to get rid of with granular include and exclude paths.
10:17If you have a multilingual monorepo like we do, uh it'll work for any language you could possibly imagine.
10:23And we can just scan our codebase and it'll produce a long list of violations.
10:27Uh way too long, in fact.
10:28It'll give you about 50 keys per violation, so we're just going to filter it down to four and we're going to sort it deterministically.
10:34Why are we doing that?
10:36At the beginning, I said this was going to be practical.
10:38And so, we're going to step outside of our control loop paradigm for a second because before we start incrementally migrating procedures one at a time, we need to enforce that all new procedures are using effect, right?
10:48So, we're going to run a full scan once on main, sort all the violations deterministically, and track it in our version control.
10:54And then on every new PR, we can see if it the branch added any unmigrated procedures, right?
11:00So, this is our control loop and our system is undergoing disturbances.
11:05In this case, uh all of our teammates shipping Claude's law.
11:08And this is how we make sure that they're not undoing our loop's work.
11:11This doesn't map directly to a part of the control loop, but we can kind of like squint at it a little bit and call it a disturbance dampener.
11:18So, now that we've stopped the bleeding, we can actually design our controller.
11:21For a simple controller, we could just deterministically pick the first violation from the list.
11:25You can use bash and jq or we could get a little cleverer and use AST grep to find the smallest unmigrated procedure and always pick the smallest one to reduce the risk.
11:37Uh we could have an agent make the decision if we really want to.
11:40I don't think you should ever send an agent to do deterministic code's job, but you certainly can.
11:45In fact, depending on the complexity, we could have the agent pick the procedure to migrate and just do it at the same time like we just talked about.
11:52But we can make this even more powerful, right?
11:54Because we're not just migrating to a factor for the sake of it.
11:56We're doing it because it's helpful for handling errors and for helping us instrument our code better.
12:01And so what we could do if we want to get really clever is we can look at our telemetry and figure out which procedures have the most errors or the least instrumentation
12:09or has a gap in our APM, right?
12:11And when we send a control signal to our actuator agent, we can include not just the procedure to migrate, but also all the data about the things that we're trying to fix with this migration
12:22so that the actuator agent can actually make the code better instead of just doing a one-to-one migration.
12:31Oh, man.
12:33There we go.
12:34So next is building our actuator.
12:36Our actuator is just an agent plus a skill.
12:39Um bring your CLI coding agent of choice.
12:41You should spend a lot of time on the skill.
12:43Not all of that should be up front.
12:44You'll want to iterate on it over time based on what works.
12:47At Human Layer, we like to build out what we call golden patterns by hand before setting the agent loose.
12:51These are just like idiomatic handwritten examples for the agent to follow because they're just pattern replicators and otherwise you're getting what's in the docs or what the agent knows from the internet.
13:00And so we pipe the skill plus our control signal into our actuator agent.
13:06And if the skill of course should include a response template, and the agent's going to work and work and work, and it'll produce a final response, and then we're going to deterministically
13:14commit and push and create a PR using the final message as our PR description.
13:20Now all we have to do is actually run the loop, right?
13:22Uh my recommendation is to use GitHub actions or your GitLab or your CircleCI or whatever else you're using because it has access to your code, it has access to your secrets,
13:32and it has great dispatch and scheduling primitives, right?
13:36We don't need a new cluster for this.
13:38So, we can write a workflow that runs a single iteration of the loop, sense, control, actuate, and creates a PR.
13:44And then we can schedule this to run once a day.
13:46And every morning we walk into the office to a small incremental PR that's low risk.
13:51And when we first did this it was actually really frustrating and we turned loop off and it cuz we had to constantly update the skill, we had to constantly check out the branch, change the skill, change the code, commit and push, and our loop was actually really high friction, right?
14:06But there's a better way to do this uh where we can put a human on the loop in a really low friction way to re-steer it when it goes wrong.
14:13And the way to do this is to just create a feedback file that's tracked in version control just as a markdown file, right?
14:18We can deterministically load it into our actuator agent's context every time that it runs after we run the controller.
14:24Then we can add a label to the PR, right?
14:26Each workflow needs to be able to identify PRs that it created since there might be a bunch of different loops running and we only want workflows to respond to feedback from
14:34uh comments on their PRs.
14:35And we're going to add a comment trigger to each loop workflow.
14:38So, the when a user leaves a {slash} iterate comment on the PR uh the loop workflow is going to pick that up, it's going to deterministically load all of the PR context,
14:47the diff, the comments, the review comments, the description into the agent's context along with the skill, and it's going to instruct the agent to fix the code, but also to update that feedback file, right?
14:58It looks kind of like this.
15:00And the benefit of doing this way is that now that feedback file with instructions is tracked in your version control, you can see how you've changed it over time, you can revert it if you need to.
15:10So, the next thing we're going to do is add flow control because the other problem that we had when we did this was that if we were at a customer site for a week
15:17or if we were traveling or spent 6 days working on slides instead of writing code.
15:23Uh the PRs from all of our loops would just stack up.
15:26They duplicate work.
15:27They'd conflict.
15:28And we wouldn't get around to doing it.
15:30And like the loops work is important, but it's not that important.
15:33And so now we just had all this like junk we had to deal with that wasn't important.
15:37So this is actually really easy problem to fix uh because each loop and its workflow has a label that gets attached to PRs.
15:44When the workflow first runs uh before we check out the code and install the dependencies and run our sense actual or sense control actuate steps, we can just check and see if the last PR that we created or any PR with the loops label on it is open.
15:58And if so, we just shut down, right?
16:00Because this means that the last time that a human uh reviewed the code from this loop was before the loop ran, right?
16:08No human reviewed the last output, so there's no reason to stack up even more work for humans to review.
16:13This way we have exactly one PR at most open per loop at a time.
16:17No stacking, no duplication, hopefully no conflicts.
16:20And of course, once you're feeling confident in the loop, we're going to want to speed it up, right?
16:24I have 150 RPC procedures to migrate.
16:27If I do one at a time, it's going to take 6 months, which is way longer than I want to wait.
16:30Fortunately, there's a lot of ways to pick up the the velocity of our loop.
16:34Uh we could have our controller pick three procedures to migrate instead of one at a time or five.
16:39Uh or five and then do each of those in a separate implementation phase, which will be both cheaper and more reliable since each migration gets its own context window.
16:50Or we could just run the workflow four times and give one PR to each of four people on the team.
16:55So let's put it all together.
16:57We built a control loop that improves our code incrementally.
17:01And we're actually reading the code.
17:03It has adaptive flow control, so we're not creating a bunch of loop or a bunch of work that nobody wants to review.
17:08And we can restir it on the fly in a super low friction way.
17:12If you want to try this yourself, uh we built a scale.
17:15Please try it out.
17:15My Twitter handle is down there on the bottom.
17:18Please share it.
17:18I would love to see what you build.
17:20And if you get excited by this, at Human Layer we're hiring here in San Francisco.
17:25And if you're working on mission critical systems and want to figure out how to get more out of AI, we'd love to chat.
17:30Thank you so much.
17:32Woo!
17:33[applause]
17:37[music]