WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=xIt_mTQp6mY

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/xIt_mTQp6mY.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:27.640 --> 00:00:28.800
Hey everybody.

s3
00:00:28.800 --> 00:00:29.800
My name's Kyle.

s4
00:00:29.800 --> 00:00:32.240
I'm co-founder of a company called Human Layer.

s5
00:00:32.240 --> 00:00:34.560
And I'm here to talk about loops.

s6
00:00:34.560 --> 00:00:36.840
I think we've all been building loops lately.

s7
00:00:36.840 --> 00:00:41.000
And I realized recently, I think we're all doing it wrong.

s8
00:00:41.000 --> 00:00:43.080
Loops are really powerful, don't get me wrong.

s9
00:00:43.080 --> 00:00:48.520
But so much of the discourse around them is hype driven and just really not helpful, right?

s10
00:00:48.520 --> 00:00:58.960
We I think we have this idea just kind of as an industry somehow that we can like pipe a prompt in a loop to a coding agent and that we can build software this way, right?

s11
00:00:58.960 --> 00:01:02.200
Maybe we're investing a lot of time in verifiers.

s12
00:01:02.200 --> 00:01:04.960
Maybe you have six different code review agents.

s13
00:01:04.960 --> 00:01:12.640
But at the end of the day, if we're doing this, we're still building 40,000 line PRs that just nobody wants to read, right?

s14
00:01:12.640 --> 00:01:15.480
And this isn't to throw shade at Jeff Huntley, right?

s15
00:01:15.480 --> 00:01:22.760
This is Ralph is an innovative uh it's a sharp tool that works very well for certain types of problems.

s16
00:01:22.760 --> 00:01:25.480
It works very well if you're not building on a team.

s17
00:01:25.480 --> 00:01:28.320
And it works very well if you're not working on critical systems.

s18
00:01:28.320 --> 00:01:33.160
But most of us are working on teams and we don't fit in that box.

s19
00:01:33.160 --> 00:01:38.320
So today I want to talk about how to build loops that work in large complex code bases.

s20
00:01:38.320 --> 00:01:51.040
For systems that have real customers, real users, real regulatory obligations and service level agreements and everything else that keeps us from shipping YOLO 40,000 line PRs straight to production.

s21
00:01:51.040 --> 00:01:55.160
In other words, I want to talk about how to build loops for the real world.

s22
00:01:55.160 --> 00:01:58.920
If you're not aware, uh this post actually dates back to July.

s23
00:01:58.920 --> 00:02:03.680
It went viral this past January, which is when a lot of us I think started building loops.

s24
00:02:03.680 --> 00:02:11.920
And of course much more recently, I'm sure you all are going to see this slide a lot this week, but Peter Steinberger said that we shouldn't be prompting coding agents anymore, right?

s25
00:02:11.920 --> 00:02:14.800
We should just be designing loops that prompt our agents.

s26
00:02:14.800 --> 00:02:18.280
Of course, open claws notoriously built on loops.

s27
00:02:18.280 --> 00:02:23.600
Loops build the code, loops review the code, they merge and release the code, they find and fix the bugs.

s28
00:02:23.600 --> 00:02:29.760
There's even loops for finding and fixing bugs in the loops that are merging the things, right?

s29
00:02:29.760 --> 00:02:31.920
It's loops all the way down.

s30
00:02:31.920 --> 00:02:41.320
It's Boris Turney, also the creator of Claude code, recently said that this is his entire job as an engineer now is just writing loops to prompt Claude.

s31
00:02:41.320 --> 00:02:44.440
And eventually, we might not even need loops, right?

s32
00:02:44.440 --> 00:02:56.560
We're just going to have like swarms of agents designing loops to prompt agents building swarms for loops and like I don't know, somewhere where like writing production code, what I assume.

s33
00:02:56.560 --> 00:03:03.239
And in fact, all of our loops that we're building are producing so much code that we can't possibly read all of it, right?

s34
00:03:03.239 --> 00:03:06.040
So we might as well just not read any of it, right?

s35
00:03:06.040 --> 00:03:10.600
We're we're investing in verification and in code review, but all this code is read only.

s36
00:03:10.600 --> 00:03:14.600
This is the thesis of a conference that was here in town last month.

s37
00:03:14.600 --> 00:03:19.080
So a lot of smart people at Frontier Labs think that this is the future of software development.

s38
00:03:19.080 --> 00:03:26.560
And if you're doing this, you're moving 10x faster and everybody else is getting left behind.

s39
00:03:26.560 --> 00:03:29.400
Now, it's not clear how well this works yet.

s40
00:03:29.400 --> 00:03:32.640
Uh Took 6 months to fix the Claude code terminal flicker.

s41
00:03:32.640 --> 00:03:36.640
The open code team wrote a renderer in a fraction of that time.

s42
00:03:36.640 --> 00:03:41.120
And open claw of course also notoriously has stability issues.

s43
00:03:41.120 --> 00:03:50.440
What is abundantly clear, however, is that this is really expensive if you don't work at a Frontier Lab and have an unlimited token budget.

s44
00:03:50.440 --> 00:03:54.680
And all this code that we're writing is actually really expensive, right?

s45
00:03:54.680 --> 00:03:56.880
Matt Pocock talked about this recently.

s46
00:03:56.880 --> 00:04:03.760
Uh bad code is much more expensive in the age of agents than it it has ever been at any point in the past.

s47
00:04:03.760 --> 00:04:12.200
So, today I want to talk about what I think works in the real world and what we started doing at Human Layer, which to be clear is still building loops, right?

s48
00:04:12.200 --> 00:04:17.120
I think loops are super powerful, but we can design loops and still read the code.

s49
00:04:17.120 --> 00:04:23.040
In fact, we can design loops to make it easier to read the code because the loops are making the code better.

s50
00:04:23.040 --> 00:04:29.000
We can solve hard problems in complex codebases with loops, and we can build our software factory incrementally.

s51
00:04:29.000 --> 00:04:33.040
But uh to do this is going to take some real engineering, y'all.

s52
00:04:33.040 --> 00:04:35.400
So, let's talk about control theory.

s53
00:04:35.400 --> 00:04:43.840
Control theory is all about how we drive a dynamic system, which would be your codebase, towards some desired, stable, or optimal end state, right?

s54
00:04:43.840 --> 00:04:47.440
You have a sensor that measures the current state of the world.

s55
00:04:47.440 --> 00:04:48.960
You have your set point, right?

s56
00:04:48.960 --> 00:04:50.680
The desired state of the world.

s57
00:04:50.680 --> 00:04:53.760
And the difference between those two things is your measured error.

s58
00:04:53.760 --> 00:05:01.960
You have a controller that reads that measured error and turns it into a control signal about an incremental change to apply to the system.

s59
00:05:01.960 --> 00:05:07.680
We have an actuator that applies that change to the system, which is undergoing disturbances in the meantime.

s60
00:05:07.680 --> 00:05:13.160
And then we remeasure, recompute our measured error, and we're back where we started.

s61
00:05:13.160 --> 00:05:16.000
Now, this sounds really complicated, and it can be.

s62
00:05:16.000 --> 00:05:18.440
I have a twin brother, actually, who's an aerospace engineer.

s63
00:05:18.440 --> 00:05:21.680
This is how they keep fighter jets from falling out of the sky.

s64
00:05:21.680 --> 00:05:25.919
Uh but uh it's probably a little bit simpler than most of y'all think.

s65
00:05:25.919 --> 00:05:28.520
Does anyone have one of these?

s66
00:05:28.600 --> 00:05:31.640
Uh a thermostat uses a control loop, right?

s67
00:05:31.640 --> 00:05:37.200
Uh for for any of our European friends in the audience, this part of something we have uh here in the States.

s68
00:05:37.200 --> 00:05:39.720
It's called air conditioning.

s69
00:05:41.080 --> 00:05:46.600
And uh most of us probably actually use control loops on a daily basis, right?

s70
00:05:46.600 --> 00:05:50.760
Kubernetes auto scaling systems are built on control loops.

s71
00:05:50.760 --> 00:05:58.040
Infrastructure as code uses a desired state, current state, iterative change like control loop pattern.

s72
00:05:58.040 --> 00:06:03.880
Postgres is auto vacuum and React's virtual DOM both use approximate control loops.

s73
00:06:03.880 --> 00:06:11.960
Control loops are ideal when we have a system that we want to change, a problem we can measure, and a way to get feedback on the result of that change.

s74
00:06:11.960 --> 00:06:23.760
Like good software engineers have always been taught to do, control loops change a system incrementally instead of just trying to get straight to the end state immediately all at once and risk blowing everything up, right?

s75
00:06:23.760 --> 00:06:29.840
They help us to avoid over steering and destabilizing the system and it minimizes risk.

s76
00:06:29.840 --> 00:06:34.160
So, control loops are the opposite of what I'm going to call a blind Ralph loop.

s77
00:06:34.160 --> 00:06:39.280
They're how we avoid PRs that look like this cuz nobody wants to review this, right?

s78
00:06:39.280 --> 00:06:42.080
Which is not to say that all Ralph loops are blind loops.

s79
00:06:42.080 --> 00:06:45.000
The best Ralphs are actually applying control theory.

s80
00:06:45.000 --> 00:06:47.520
I know Jeff Huntley is out in the hall somewhere wandering around.

s81
00:06:47.520 --> 00:06:50.560
If you go talk to him, he's going to tell you the same thing, right?

s82
00:06:50.560 --> 00:06:59.240
That Ralph is a teaching device and I think some of us read it a little too literally, but this is how we should have always been building loops.

s83
00:06:59.240 --> 00:07:02.680
But, the other issue with Ralph loops they're not incremental, right?

s84
00:07:02.680 --> 00:07:04.800
It's just a bash loop.

s85
00:07:04.800 --> 00:07:07.080
So, we have to build agentic control loops.

s86
00:07:07.080 --> 00:07:14.320
And to do that, we start by defining a set point which is the desired end state of our code base with respect to some property of it.

s87
00:07:14.320 --> 00:07:15.320
And we add a sensor.

s88
00:07:15.320 --> 00:07:16.800
There's a lot of ways to build a sensor.

s89
00:07:16.800 --> 00:07:25.640
It can be strictly deterministic, your ESLint rules, your AST grep, your packwork, or it can be non-deterministic.

s90
00:07:25.640 --> 00:07:33.680
You can have an agent and a skill and a bunch of natural language rules, and you could also just have a pipeline like a combination of the two.

s91
00:07:33.680 --> 00:07:36.560
So, how do we build agentic Oops.

s92
00:07:36.560 --> 00:07:37.600
There we go.

s93
00:07:37.600 --> 00:07:40.000
Now, this is all theory, right?

s94
00:07:40.000 --> 00:07:45.600
Practically speaking, and because we're using agents, we can blur the lines a little bit between system components.

s95
00:07:45.600 --> 00:07:49.080
So, Aiden by React Doctor, for example, is fantastic.

s96
00:07:49.080 --> 00:07:56.040
It is uh it's a great way to catch all of the React slot that Claude snuck into your code base last week.

s97
00:07:56.040 --> 00:07:58.280
But uh it's a hybrid sensor and controller.

s98
00:07:58.280 --> 00:08:05.840
It tells you what are all the problems with your React code, and also, by the way, what are the top three things you should fix, and how do you fix them?

s99
00:08:05.840 --> 00:08:14.919
Similarly, our controller and actuator might actually just be a single agent deciding on an incremental change to make and then applying it in the same context window.

s100
00:08:14.919 --> 00:08:25.320
But I want to zoom in on the controller a little bit because without one, or without a well-tuned one, we might make too large of a change all at once, or we might make the wrong change entirely.

s101
00:08:25.320 --> 00:08:29.280
And if you put that in a loop, you're in trouble pretty quickly.

s102
00:08:29.280 --> 00:08:35.159
So, we can use control loops to root out bad patterns and to clean up our code, but we can actually use them for all sorts of things, right?

s103
00:08:35.159 --> 00:08:39.880
We could make sure that our API is compliant with someone else's open API spec.

s104
00:08:39.880 --> 00:08:48.160
We can make sure that our MCP server is compliant with whatever version of the uh the MCP specification that we're currently on.

s105
00:08:48.160 --> 00:08:49.200
Haven't checked.

s106
00:08:49.200 --> 00:08:54.120
You could mirror a project from Python into HypeScript, or vice versa.

s107
00:08:54.120 --> 00:09:00.600
You could even maintain your uh Beat-based slot fork of Next.js against the upstream.

s108
00:09:00.600 --> 00:09:03.839
The key questions are, can we find something we can measure?

s109
00:09:03.839 --> 00:09:05.800
Can we apply changes incrementally?

s110
00:09:05.800 --> 00:09:08.920
And can we get feedback on the quality of those changes?

s111
00:09:08.920 --> 00:09:13.400
To illustrate that, I'm going to walk through a control loop that we use internally at Human Layer.

s112
00:09:13.400 --> 00:09:18.120
Uh for our loop, we are incrementally migrating our RPC API to Effect.

s113
00:09:18.120 --> 00:09:20.080
We adopted it for some of our race-prone code.

s114
00:09:20.080 --> 00:09:23.040
We like it, so we're adopting it across the rest of our code base.

s115
00:09:23.040 --> 00:09:29.400
If you've never seen Effect's code before, the code on the right is just the kind of trivial procedure on the left rewritten in Effect.

s116
00:09:29.400 --> 00:09:30.680
Uh the syntax is really weird.

s117
00:09:30.680 --> 00:09:31.280
We're psychos.

s118
00:09:31.280 --> 00:09:32.080
We really like it.

s119
00:09:32.080 --> 00:09:33.080
It's not for everybody.

s120
00:09:33.080 --> 00:09:34.120
That's okay.

s121
00:09:34.120 --> 00:09:37.960
Uh this isn't a talk about Effect, so we'll keep moving.

s122
00:09:38.640 --> 00:09:40.720
Ooh, clicker's not working.

s123
00:09:40.720 --> 00:09:41.720
Cool.

s124
00:09:41.720 --> 00:09:45.720
So, step one, we have to build our sensor to find unmigrated procedures.

s125
00:09:45.720 --> 00:09:50.280
We can have an agent do this or we could use grep or rip grep.

s126
00:09:50.280 --> 00:09:52.680
But, instead we're going to use AST grep cuz it's really powerful.

s127
00:09:52.680 --> 00:09:54.960
It's a great tool to have in your toolbox for building loops.

s128
00:09:54.960 --> 00:09:56.400
It's language-agnostic.

s129
00:09:56.400 --> 00:10:04.800
It's out of band from your TypeScript config or ESLint rules, which if you're a TypeScript developer, you have watched Claude disable those with inline comments.

s130
00:10:04.800 --> 00:10:10.960
Uh but, so we can just write a simple rule that finds unmigrated procedures uh based on the pattern above.

s131
00:10:10.960 --> 00:10:17.240
We Over time, we can even layer on more rules that describe other patterns we want to get rid of with granular include and exclude paths.

s132
00:10:17.240 --> 00:10:23.120
If you have a multilingual monorepo like we do, uh it'll work for any language you could possibly imagine.

s133
00:10:23.120 --> 00:10:27.200
And we can just scan our codebase and it'll produce a long list of violations.

s134
00:10:27.200 --> 00:10:28.600
Uh way too long, in fact.

s135
00:10:28.600 --> 00:10:34.760
It'll give you about 50 keys per violation, so we're just going to filter it down to four and we're going to sort it deterministically.

s136
00:10:34.760 --> 00:10:36.200
Why are we doing that?

s137
00:10:36.200 --> 00:10:38.880
At the beginning, I said this was going to be practical.

s138
00:10:38.880 --> 00:10:48.720
And so, we're going to step outside of our control loop paradigm for a second because before we start incrementally migrating procedures one at a time, we need to enforce that all new procedures are using effect, right?

s139
00:10:48.720 --> 00:10:54.920
So, we're going to run a full scan once on main, sort all the violations deterministically, and track it in our version control.

s140
00:10:54.920 --> 00:11:00.960
And then on every new PR, we can see if it the branch added any unmigrated procedures, right?

s141
00:11:00.960 --> 00:11:05.320
So, this is our control loop and our system is undergoing disturbances.

s142
00:11:05.320 --> 00:11:08.160
In this case, uh all of our teammates shipping Claude's law.

s143
00:11:08.160 --> 00:11:11.800
And this is how we make sure that they're not undoing our loop's work.

s144
00:11:11.800 --> 00:11:18.400
This doesn't map directly to a part of the control loop, but we can kind of like squint at it a little bit and call it a disturbance dampener.

s145
00:11:18.400 --> 00:11:21.839
So, now that we've stopped the bleeding, we can actually design our controller.

s146
00:11:21.839 --> 00:11:25.839
For a simple controller, we could just deterministically pick the first violation from the list.

s147
00:11:25.839 --> 00:11:37.880
You can use bash and jq or we could get a little cleverer and use AST grep to find the smallest unmigrated procedure and always pick the smallest one to reduce the risk.

s148
00:11:37.960 --> 00:11:40.640
Uh we could have an agent make the decision if we really want to.

s149
00:11:40.640 --> 00:11:45.120
I don't think you should ever send an agent to do deterministic code's job, but you certainly can.

s150
00:11:45.120 --> 00:11:52.480
In fact, depending on the complexity, we could have the agent pick the procedure to migrate and just do it at the same time like we just talked about.

s151
00:11:52.480 --> 00:11:54.400
But we can make this even more powerful, right?

s152
00:11:54.400 --> 00:11:56.480
Because we're not just migrating to a factor for the sake of it.

s153
00:11:56.480 --> 00:12:01.280
We're doing it because it's helpful for handling errors and for helping us instrument our code better.

s154
00:12:01.280 --> 00:12:09.440
And so what we could do if we want to get really clever is we can look at our telemetry and figure out which procedures have the most errors or the least instrumentation

s155
00:12:09.440 --> 00:12:11.680
or has a gap in our APM, right?

s156
00:12:11.680 --> 00:12:22.840
And when we send a control signal to our actuator agent, we can include not just the procedure to migrate, but also all the data about the things that we're trying to fix with this migration

s157
00:12:22.840 --> 00:12:28.680
so that the actuator agent can actually make the code better instead of just doing a one-to-one migration.

s158
00:12:31.920 --> 00:12:33.040
Oh, man.

s159
00:12:33.040 --> 00:12:34.240
There we go.

s160
00:12:34.240 --> 00:12:36.400
So next is building our actuator.

s161
00:12:36.400 --> 00:12:39.080
Our actuator is just an agent plus a skill.

s162
00:12:39.080 --> 00:12:41.680
Um bring your CLI coding agent of choice.

s163
00:12:41.680 --> 00:12:43.400
You should spend a lot of time on the skill.

s164
00:12:43.400 --> 00:12:44.720
Not all of that should be up front.

s165
00:12:44.720 --> 00:12:47.000
You'll want to iterate on it over time based on what works.

s166
00:12:47.000 --> 00:12:51.560
At Human Layer, we like to build out what we call golden patterns by hand before setting the agent loose.

s167
00:12:51.560 --> 00:13:00.760
These are just like idiomatic handwritten examples for the agent to follow because they're just pattern replicators and otherwise you're getting what's in the docs or what the agent knows from the internet.

s168
00:13:00.760 --> 00:13:06.800
And so we pipe the skill plus our control signal into our actuator agent.

s169
00:13:06.800 --> 00:13:14.760
And if the skill of course should include a response template, and the agent's going to work and work and work, and it'll produce a final response, and then we're going to deterministically

s170
00:13:14.760 --> 00:13:20.080
commit and push and create a PR using the final message as our PR description.

s171
00:13:20.080 --> 00:13:22.760
Now all we have to do is actually run the loop, right?

s172
00:13:22.760 --> 00:13:32.600
Uh my recommendation is to use GitHub actions or your GitLab or your CircleCI or whatever else you're using because it has access to your code, it has access to your secrets,

s173
00:13:32.600 --> 00:13:36.440
and it has great dispatch and scheduling primitives, right?

s174
00:13:36.440 --> 00:13:38.240
We don't need a new cluster for this.

s175
00:13:38.240 --> 00:13:44.520
So, we can write a workflow that runs a single iteration of the loop, sense, control, actuate, and creates a PR.

s176
00:13:44.520 --> 00:13:46.720
And then we can schedule this to run once a day.

s177
00:13:46.720 --> 00:13:51.160
And every morning we walk into the office to a small incremental PR that's low risk.

s178
00:13:51.160 --> 00:14:06.600
And when we first did this it was actually really frustrating and we turned loop off and it cuz we had to constantly update the skill, we had to constantly check out the branch, change the skill, change the code, commit and push, and our loop was actually really high friction, right?

s179
00:14:06.600 --> 00:14:13.080
But there's a better way to do this uh where we can put a human on the loop in a really low friction way to re-steer it when it goes wrong.

s180
00:14:13.080 --> 00:14:18.240
And the way to do this is to just create a feedback file that's tracked in version control just as a markdown file, right?

s181
00:14:18.240 --> 00:14:24.760
We can deterministically load it into our actuator agent's context every time that it runs after we run the controller.

s182
00:14:24.760 --> 00:14:26.600
Then we can add a label to the PR, right?

s183
00:14:26.600 --> 00:14:34.320
Each workflow needs to be able to identify PRs that it created since there might be a bunch of different loops running and we only want workflows to respond to feedback from

s184
00:14:34.320 --> 00:14:35.920
uh comments on their PRs.

s185
00:14:35.920 --> 00:14:38.839
And we're going to add a comment trigger to each loop workflow.

s186
00:14:38.839 --> 00:14:47.720
So, the when a user leaves a {slash} iterate comment on the PR uh the loop workflow is going to pick that up, it's going to deterministically load all of the PR context,

s187
00:14:47.720 --> 00:14:58.520
the diff, the comments, the review comments, the description into the agent's context along with the skill, and it's going to instruct the agent to fix the code, but also to update that feedback file, right?

s188
00:14:58.520 --> 00:15:00.080
It looks kind of like this.

s189
00:15:00.080 --> 00:15:10.200
And the benefit of doing this way is that now that feedback file with instructions is tracked in your version control, you can see how you've changed it over time, you can revert it if you need to.

s190
00:15:10.360 --> 00:15:17.600
So, the next thing we're going to do is add flow control because the other problem that we had when we did this was that if we were at a customer site for a week

s191
00:15:17.600 --> 00:15:23.360
or if we were traveling or spent 6 days working on slides instead of writing code.

s192
00:15:23.360 --> 00:15:26.120
Uh the PRs from all of our loops would just stack up.

s193
00:15:26.120 --> 00:15:27.280
They duplicate work.

s194
00:15:27.280 --> 00:15:28.360
They'd conflict.

s195
00:15:28.360 --> 00:15:30.000
And we wouldn't get around to doing it.

s196
00:15:30.000 --> 00:15:33.040
And like the loops work is important, but it's not that important.

s197
00:15:33.040 --> 00:15:37.000
And so now we just had all this like junk we had to deal with that wasn't important.

s198
00:15:37.000 --> 00:15:44.280
So this is actually really easy problem to fix uh because each loop and its workflow has a label that gets attached to PRs.

s199
00:15:44.280 --> 00:15:58.800
When the workflow first runs uh before we check out the code and install the dependencies and run our sense actual or sense control actuate steps, we can just check and see if the last PR that we created or any PR with the loops label on it is open.

s200
00:15:58.800 --> 00:16:00.840
And if so, we just shut down, right?

s201
00:16:00.840 --> 00:16:08.640
Because this means that the last time that a human uh reviewed the code from this loop was before the loop ran, right?

s202
00:16:08.640 --> 00:16:13.760
No human reviewed the last output, so there's no reason to stack up even more work for humans to review.

s203
00:16:13.760 --> 00:16:17.480
This way we have exactly one PR at most open per loop at a time.

s204
00:16:17.480 --> 00:16:20.680
No stacking, no duplication, hopefully no conflicts.

s205
00:16:20.680 --> 00:16:24.440
And of course, once you're feeling confident in the loop, we're going to want to speed it up, right?

s206
00:16:24.440 --> 00:16:27.080
I have 150 RPC procedures to migrate.

s207
00:16:27.080 --> 00:16:30.839
If I do one at a time, it's going to take 6 months, which is way longer than I want to wait.

s208
00:16:30.839 --> 00:16:34.440
Fortunately, there's a lot of ways to pick up the the velocity of our loop.

s209
00:16:34.440 --> 00:16:39.520
Uh we could have our controller pick three procedures to migrate instead of one at a time or five.

s210
00:16:39.520 --> 00:16:50.320
Uh or five and then do each of those in a separate implementation phase, which will be both cheaper and more reliable since each migration gets its own context window.

s211
00:16:50.320 --> 00:16:55.839
Or we could just run the workflow four times and give one PR to each of four people on the team.

s212
00:16:55.839 --> 00:16:57.880
So let's put it all together.

s213
00:16:57.880 --> 00:17:01.120
We built a control loop that improves our code incrementally.

s214
00:17:01.120 --> 00:17:03.120
And we're actually reading the code.

s215
00:17:03.120 --> 00:17:08.720
It has adaptive flow control, so we're not creating a bunch of loop or a bunch of work that nobody wants to review.

s216
00:17:08.720 --> 00:17:12.240
And we can restir it on the fly in a super low friction way.

s217
00:17:12.240 --> 00:17:15.120
If you want to try this yourself, uh we built a scale.

s218
00:17:15.120 --> 00:17:15.959
Please try it out.

s219
00:17:15.959 --> 00:17:18.040
My Twitter handle is down there on the bottom.

s220
00:17:18.040 --> 00:17:18.640
Please share it.

s221
00:17:18.640 --> 00:17:20.680
I would love to see what you build.

s222
00:17:20.680 --> 00:17:25.000
And if you get excited by this, at Human Layer we're hiring here in San Francisco.

s223
00:17:25.000 --> 00:17:30.880
And if you're working on mission critical systems and want to figure out how to get more out of AI, we'd love to chat.

s224
00:17:30.880 --> 00:17:32.160
Thank you so much.

s225
00:17:32.160 --> 00:17:33.252
Woo!

s226
00:17:33.252 --> 00:17:35.252
[applause]

s227
00:17:37.837 --> 00:17:39.837
[music]