WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=qdZzND79mcg

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/qdZzND79mcg.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.520 --> 00:00:03.160
Hello, I'm Rajiv.

s2
00:00:03.160 --> 00:00:10.160
I'm a practicing medical doctor here in London and been doing some AI engineering for the past few years.

s3
00:00:10.160 --> 00:00:22.600
Particularly interested in real world application and really the future which is multi-agent multi-human multi-institutional collaboration through a company called Anitcha Labs.

s4
00:00:22.600 --> 00:00:33.920
And really the premise of my talk today is to introduce a new design philosophy for the future of AI engineering which is beyond a harness or beyond a fixed

s5
00:00:33.920 --> 00:00:39.120
static harness and towards adaptive engineering.

s6
00:00:41.480 --> 00:00:56.160
This slide basically summarizes my entire talk and really it's about exploring what the current AI engineering paradigm is which is to use fixed harnesses to steer agents.

s7
00:00:56.160 --> 00:01:08.240
So we either use or build an existing harness like Pi, Claude, Code, Cursor, CodeX, whatever it is ahead of run time that remains fairly consistent throughout the engineering process

s8
00:01:08.240 --> 00:01:20.920
and is largely constrained with fixed roles and tools and sequencing which again ahead of the process beginning of the process we can tweak and it essentially produces very reliable

s9
00:01:20.920 --> 00:01:25.880
and orderly work and it's great for most engineering problems and ideas.

s10
00:01:25.880 --> 00:01:32.480
And so your role as an AI engineering right now is to use a harness and steer agents.

s11
00:01:32.480 --> 00:01:36.920
But the future's going to be different in at least two ways.

s12
00:01:36.920 --> 00:01:45.680
Number one, models are going to become so powerful that existing fixed harnesses are constantly going to become outdated.

s13
00:01:45.680 --> 00:01:50.280
And number two is there's going to be a huge exposure to the real world.

s14
00:01:50.280 --> 00:01:54.160
Right now a lot of AI engineering is software based.

s15
00:01:54.160 --> 00:01:57.840
It's within behind a screen.

s16
00:01:57.840 --> 00:02:09.880
But actually, as things get more powerful, as models get more powerful, we're going to have exposure to the real world where I'd argue that this is where the real challenges are.

s17
00:02:09.880 --> 00:02:12.600
And the real world is dynamic and messy.

s18
00:02:12.600 --> 00:02:23.320
You know, it's full of multi-agent, multi-human, cross-institutional, and really touching the physical world, which means that a fixed harness is quite brittle.

s19
00:02:23.320 --> 00:02:40.240
And so that's where we enter what I coin as adaptive engineering, where you actually allow the harness to emerge and adapt mid-engineering to find its most optimal position and structure.

s20
00:02:40.240 --> 00:02:51.800
And essentially, the engineer's role is going to be to design some of the constraints, which are kind of the rules of play, and really allowing the harness to emerge,

s21
00:02:51.800 --> 00:03:00.760
stabilize, change, and eventually dissolve as you go through the runtime of engineering.

s22
00:03:03.840 --> 00:03:10.160
So, let's look firstly at the current paradigm for AI engineering.

s23
00:03:10.160 --> 00:03:23.440
And it's essentially where a harness serves as the primary method to guide an LLM, which is typically stateless, and make it into something quite useful.

s24
00:03:23.440 --> 00:03:34.760
And there's a couple of aspects of a harness, which I won't go to in detail, but essentially things like system prompts, which is established by the harness vendor, meaning that users can't actually modify it.

s25
00:03:34.760 --> 00:03:40.800
And this essentially tells the harness what it can and can't do.

s26
00:03:40.800 --> 00:03:53.760
You've also got the agents.md files or the Claude.md if you're using Claude code, which is loaded into every context window at the start of a session, tool calling, and essentially

s27
00:03:53.760 --> 00:04:08.760
then you have the genesis of agents, which are specialized entities that have been harnessed uh, granting them unique capabilities um, and and allowing them to differentiate from other agents.

s28
00:04:08.760 --> 00:04:23.600
And these capabilities are typically manifest as specific skills when it comes to role typology, their capabilities, the rules um, and that enables them to deliver targeted outputs and specified

s29
00:04:23.600 --> 00:04:39.040
outputs, whether it be code documentation issues, handoffs and and most recently we've seen the genesis of kind of loops, loop engineering has become a thing um so that eventually

s30
00:04:39.040 --> 00:04:49.080
we reach some sort of outcome uh, whether it's a feature, uh, a problem solved which is eventually reviewed by a human.

s31
00:04:49.080 --> 00:05:00.760
Um, so what you see uh, from this is that before running the engineering process the entire harness has been predetermined or pre-engineered.

s32
00:05:00.760 --> 00:05:02.160
And that's harness engineering.

s33
00:05:02.160 --> 00:05:11.000
And uh, it's been remarkable at getting the most out of the latest models.

s34
00:05:15.880 --> 00:05:21.440
Um and just sort of stepping a bit back, um, defining exactly what a harness is.

s35
00:05:21.440 --> 00:05:29.480
Essentially the model is the engine and the harness is everything built around to make that engine useful.

s36
00:05:29.480 --> 00:05:32.080
Um, there's many different types of harnesses.

s37
00:05:32.080 --> 00:05:42.400
I mean, there's hundreds uh, actually, but I guess the most common ones are, um, you know, CLI coding, um, such as Claude Code, Codex, Pi.

s38
00:05:42.400 --> 00:05:52.640
You have IDEs like Cursor uh, multi-agent orchestration through LangChain and and things like Hermes as well, Klein and Goose.

s39
00:05:52.640 --> 00:05:58.600
And And these are all differentiated based on number one, properties and how they're configured.

s40
00:05:58.600 --> 00:06:08.040
But actually more importantly, number two, based on some sort of design and engineering philosophy that they believe in.

s41
00:06:08.040 --> 00:06:15.720
And what makes them so great and so distinct is that they are opinionated to a lesser or greater degree.

s42
00:06:15.720 --> 00:06:21.400
Um and they allow for, I guess different use cases.

s43
00:06:21.400 --> 00:06:28.440
Um and depending on your temperament as an engineer, you'd pick one over the other.

s44
00:06:30.160 --> 00:06:34.840
But what we do see in all of these is that everything's predefined.

s45
00:06:34.840 --> 00:06:39.680
You know, you can customize them, of course, like for example, the Pi agent.

s46
00:06:39.680 --> 00:06:44.400
It's minimalist and it's maximally extensible.

s47
00:06:44.400 --> 00:06:47.520
Um but everything is predefined.

s48
00:06:47.520 --> 00:06:53.400
You know, the roles, fixed outputs, sequencing protocols, how memory is created.

s49
00:06:53.400 --> 00:06:56.000
Uh and so I want to be absolutely clear here.

s50
00:06:56.000 --> 00:06:58.919
I don't mean that you can't customize harnesses.

s51
00:06:58.919 --> 00:07:00.120
Of course, you can.

s52
00:07:00.120 --> 00:07:04.160
But the customization occurs ahead of the engineering runtime.

s53
00:07:04.160 --> 00:07:05.800
Uh and not mid-engineering.

s54
00:07:05.800 --> 00:07:08.440
And that's largely directed by us as humans.

s55
00:07:08.440 --> 00:07:19.280
And for someone like me who started AI engineering about two years ago, it's probably the most powerful or predictive uh predictable method.

s56
00:07:19.280 --> 00:07:20.600
And it's got really three payoffs.

s57
00:07:20.600 --> 00:07:22.160
It's reliable.

s58
00:07:22.160 --> 00:07:25.960
Um you know, the same input leads to a similar set of outputs.

s59
00:07:25.960 --> 00:07:28.880
Obviously, there's going to be some variation.

s60
00:07:28.880 --> 00:07:30.360
It's auditable.

s61
00:07:30.360 --> 00:07:33.320
Um you can inspect exactly what changed and when.

s62
00:07:33.320 --> 00:07:41.160
And there's some sort of linear causality where uh you know, if something breaks, you can actually trace it back to its source.

s63
00:07:41.160 --> 00:07:43.800
Um this is like a factory line though.

s64
00:07:43.800 --> 00:07:45.800
Like think Taylorism for AI.

s65
00:07:45.800 --> 00:07:51.160
Just like a factory assembly line, every station is engineered in advance.

s66
00:07:51.160 --> 00:08:01.480
Um, each agent has one job, a fixed place, a sequence, and maybe a defined handoff to the next agent.

s67
00:08:04.640 --> 00:08:19.560
I mean, there's clearly some incredible uses for this and, you know, factory engineering gives you, you know, accuracy, speed, reproducibility, behavior that you can certify, and it really shines in these kind of closed deterministic

s68
00:08:19.560 --> 00:08:25.919
systems, where you have a well-defined product or problem, and you know the kind of features you want to build.

s69
00:08:25.919 --> 00:08:29.280
You know the sort of solutions that you need.

s70
00:08:29.280 --> 00:08:33.159
Um, but here's the catch.

s71
00:08:33.159 --> 00:08:38.719
Um, and and it's a real trade-off that that reliability isn't free.

s72
00:08:38.719 --> 00:08:47.320
Um, you buy it by suppressing, um, variance that I'd argue novelty requires.

s73
00:08:47.320 --> 00:08:52.520
Um, and determinism and emergence pull kind of in opposite directions.

s74
00:08:52.520 --> 00:09:04.600
Um, and this is where we have to pivot to the real world, because in real life situations, in when when AI eventually, uh, when it becomes a norm for AI to

s75
00:09:04.600 --> 00:09:24.080
capture real-world scenarios, where you've got multi-agents, multi-humans, multi-institutions interacting, and also touching the physical world, um, there's a hard ceiling on novelty, um, because the real world is messy, and it keeps changing.

s76
00:09:24.080 --> 00:09:28.040
And so, fixed harness starts to break down.

s77
00:09:28.040 --> 00:09:40.000
Um, so like I mentioned, there's a hard ceiling on novelty, um, because, uh, essentially, its reliability is predicated on suppressing variants.

s78
00:09:40.000 --> 00:09:43.240
Um the models are going to keep accelerating.

s79
00:09:43.240 --> 00:09:48.720
You know, you can build a careful harness today or you can use one off the shelf.

s80
00:09:48.720 --> 00:09:50.960
But it could be irrelevant in the next month.

s81
00:09:50.960 --> 00:09:57.560
The model just got so good that it didn't need that kind of scaffolding anymore.

s82
00:09:57.560 --> 00:10:10.320
Um and that kind of leads to this whole dilemma of brutalness where every situation you didn't anticipated you didn't anticipate needs a human to patch the harness.

s83
00:10:10.320 --> 00:10:20.040
The more real world it needs, the more rules you bolt on and eventually the harness just becomes ever more complicated than the actual problem that you need to solve.

s84
00:10:20.040 --> 00:10:34.280
And I guess if there's one takeaway here is that the factory method is the right answer to a fixed problem and the wrong answer to a moving problem.

s85
00:10:37.080 --> 00:10:46.040
And so let's really examine in what the real world is about and and I'm going to go down a bit of a philosophical route here.

s86
00:10:46.040 --> 00:10:55.680
Um as we've said, you know, agentic systems will eventually collide with the real world and that raises a question.

s87
00:10:55.680 --> 00:10:58.680
What is the real world actually made of?

s88
00:10:58.680 --> 00:11:00.839
Because there's two ways to answer this actually.

s89
00:11:00.839 --> 00:11:10.240
So the first paradigm, the first way to see the world is through what we call a reductionist or analytical view which we're all very used to actually.

s90
00:11:10.240 --> 00:11:22.120
It says to understand the whole a big problem or a system or a big thing like an organization we've got to take it apart and study the individual parts.

s91
00:11:22.120 --> 00:11:35.240
So essentially the world is made up of these individual parts which are things that are stable and change is just something that happens to those stable things now and then.

s92
00:11:35.240 --> 00:11:39.880
The relationships amongst those things are secondary or an afterthought.

s93
00:11:39.880 --> 00:11:45.720
And this is really the metaphysics of a factory and of nearly all software that we build.

s94
00:11:45.720 --> 00:11:48.400
You know, you make the components.

s95
00:11:48.400 --> 00:11:51.680
Uh you wire them together.

s96
00:11:52.040 --> 00:11:55.440
And you get a product.

s97
00:11:55.800 --> 00:12:10.960
The second paradigm, what we call the systems or relational paradigm, um takes the emphasis away from there being separate things or components and actually puts more what we call ontological

s98
00:12:10.960 --> 00:12:14.400
emphasis on the relationships amongst those things.

s99
00:12:14.400 --> 00:12:15.760
So, flips the order.

s100
00:12:15.760 --> 00:12:19.960
It says that the real world isn't made up of things at all.

s101
00:12:19.960 --> 00:12:27.160
It's made up of processes and um relationships which come first.

s102
00:12:27.400 --> 00:12:32.960
So, what we call a stable thing really is just a slow pattern in an ongoing flow.

s103
00:12:32.960 --> 00:12:34.440
Think an organism.

s104
00:12:34.440 --> 00:12:37.080
It's more like a flame than a crystal.

s105
00:12:37.080 --> 00:12:39.920
A flame looks like an object, but it isn't.

s106
00:12:39.920 --> 00:12:45.520
It's a pattern held together moment by moment by a process.

s107
00:12:45.520 --> 00:12:49.800
You stop the process and the thing doesn't exist.

s108
00:12:50.640 --> 00:12:54.720
Uh and there's a really good payoff in understanding this paradigm.

s109
00:12:54.720 --> 00:12:59.120
Um and once you see the world in this way, you start to notice something remarkable.

s110
00:12:59.120 --> 00:13:17.640
Where where you have simple local rules between, let's say, particles or between people or between creatures, that local interaction gives rise to a whole new level of order that can't be predicted

s111
00:13:17.640 --> 00:13:25.000
from its constituent parts and that no constituent part designed ahead of time.

s112
00:13:25.000 --> 00:13:26.480
For example, water.

s113
00:13:26.480 --> 00:13:30.840
Water has a property of being wet.

s114
00:13:30.840 --> 00:13:36.280
But it's constituent parts, oxygen and hydrogen, are not wet.

s115
00:13:36.280 --> 00:13:48.440
But their coming together, again putting emphasis on the relationships, their coming together is where you had the emergence of a novel new property, i.e. wetness.

s116
00:13:48.440 --> 00:13:50.520
Another example is a flock of birds.

s117
00:13:50.520 --> 00:13:56.000
Not one bird wakes up every morning intending to make a flock.

s118
00:13:56.000 --> 00:14:06.200
Each probably just follows three simple local rules, which is align with your neighbor, don't crash into them, stay close, and that's it.

s119
00:14:06.200 --> 00:14:20.360
Local rules, local interactions, and out of that you get a flock, a new pattern alive and coordinated that you could never understand by pulling it apart and studying one bird because no single bird

s120
00:14:20.360 --> 00:14:21.880
has a flock in it.

s121
00:14:21.880 --> 00:14:27.080
The flock lives in the relationships, not the parts.

s122
00:14:27.080 --> 00:14:32.560
So this factory method of engineering assumes a world of parts.

s123
00:14:32.880 --> 00:14:46.320
But the non-machine world, and I'd argue that AI is sort of on the cusp of that, the non-machine world is a world of patterns and moving patterns of emergence.

s124
00:14:48.720 --> 00:14:56.480
And that sort of reflects what sort of problems we face in this moving messy world.

s125
00:14:56.480 --> 00:14:59.880
Um and there's a line from Russell Ackoff that I absolutely love.

s126
00:14:59.880 --> 00:15:04.160
He says that managers aren't handed neat separate problems.

s127
00:15:04.160 --> 00:15:11.280
They're handed dynamic situations, tangles of problems that keep changing and keep bumping to each other.

s128
00:15:11.280 --> 00:15:12.720
And he And he gave that a name.

s129
00:15:12.720 --> 00:15:14.360
He called it a mess.

s130
00:15:14.360 --> 00:15:22.240
A mess is a isn't just a pile of separate parts, it's actually a system of relationships in motion.

s131
00:15:22.240 --> 00:15:34.680
Which is exactly why the factory based on a fixed harness way of engineering will struggle when it collides with real-world live scenarios.

s132
00:15:34.680 --> 00:15:47.080
Because you can't decompose a mess into tidy boxes and solve each of them because they're not individual parts, they're a moving pattern of relationships.

s133
00:15:49.080 --> 00:15:55.920
We can then look at problems through this this distinction between different types of problems.

s134
00:15:55.920 --> 00:16:03.400
So, um engineering is all about solving problems and I'd argue that very simply put there are two kinds of problems.

s135
00:16:03.400 --> 00:16:08.160
There are complicated problems um and there are complex problems.

s136
00:16:08.160 --> 00:16:15.160
So, a complicated system or complicated problem is like building a jumbo jet or a clock.

s137
00:16:15.160 --> 00:16:24.320
There are passive parts um but experts can take them apart, analyze them, plan, predict, document.

s138
00:16:24.320 --> 00:16:31.000
It's hard, but it's actually knowable and it works in a very predictable way.

s139
00:16:31.000 --> 00:16:42.400
In contrast, when you have a complex system or a complex problem like a flock of birds, a market, a human organization, it's slightly different.

s140
00:16:42.400 --> 00:16:47.240
It's parts are constantly interacting and adapting to one another.

s141
00:16:47.240 --> 00:16:53.880
So, the whole can't be derived from analyzing the parts.

s142
00:16:53.880 --> 00:16:56.760
So, you don't analyze and plan a complex system.

s143
00:16:56.760 --> 00:17:09.520
Instead, you probe and sense and respond and this is probably one of the most expensive mistakes we make in the modern world of design and engineering, which is that we treat a complex problem

s144
00:17:09.520 --> 00:17:22.360
like a complicated one um and and things fail not due to lack of execution, but essentially because there's a failure in categorizing the problem spaces.

s145
00:17:24.079 --> 00:17:36.640
If we go just one step deeper into what a complex system is, um essentially, you know, the ingredients of it are first, you need a diverse set of agents

s146
00:17:36.640 --> 00:17:39.520
or actors if it's a human organization.

s147
00:17:39.520 --> 00:17:43.480
Um not clones essentially, but different from one another in some sort of way.

s148
00:17:43.480 --> 00:17:47.200
So, because the diversity is the fuel there.

s149
00:17:47.200 --> 00:17:55.960
Um they need to interact locally uh cuz no agent, no individual actor, no individual component sees the whole.

s150
00:17:55.960 --> 00:17:59.960
Um instead each respond to whoever's next to it.

s151
00:17:59.960 --> 00:18:04.000
And this is like the bird aligning with the neighbor.

s152
00:18:04.520 --> 00:18:05.560
And they learn.

s153
00:18:05.560 --> 00:18:06.800
They learn recursively.

s154
00:18:06.800 --> 00:18:18.360
They adapt, then they adapt to the results of their own adapting, and it constantly loops, and um you know, with respect to one another and the environment.

s155
00:18:18.360 --> 00:18:22.800
Everything is moving in response to everything else, and nothing holds still.

s156
00:18:22.800 --> 00:18:35.200
And out of all of that local adaptive looping interaction, you get something called emergence, which is a key concept in complexity science.

s157
00:18:35.200 --> 00:18:53.560
Um basically, it's a new novel pattern uh or organization or behavior that uh no single part designed on its own and can't be explained by splitting that whole into its parts and studying them.

s158
00:18:53.560 --> 00:18:56.440
Again, think the flock of birds.

s159
00:18:56.440 --> 00:18:58.400
No bird has a flock in it.

s160
00:18:58.400 --> 00:19:00.720
Think about the wetness of water.

s161
00:19:00.720 --> 00:19:07.440
None of the individual oxygen hydrogen molecules have wetness within them.

s162
00:19:07.440 --> 00:19:12.080
And yet, it leads to the emergence of something quite novel.

s163
00:19:12.080 --> 00:19:16.200
Now, these systems don't fly apart into chaos.

s164
00:19:16.200 --> 00:19:21.440
They tend to settle into attractors, which are essentially stable states.

s165
00:19:21.440 --> 00:19:25.240
Uh they keep pulling back towards some sort of equilibrium.

s166
00:19:25.240 --> 00:19:28.880
So, for example, water is stable at room temperature.

s167
00:19:28.880 --> 00:19:30.800
So, you get this sort of beautiful tension.

s168
00:19:30.800 --> 00:19:40.520
You've got constant change at a local level, but then you've got recognizable stable patterns at a whole system level.

s169
00:19:40.520 --> 00:19:43.040
And crucially, nobody's steering here.

s170
00:19:43.040 --> 00:19:44.920
It's self-organizing.

s171
00:19:44.920 --> 00:19:47.240
And that's the whole mystery.

s172
00:19:47.240 --> 00:19:52.400
And I'd argue that's the whole opportunity for design and engineering.

s173
00:19:52.400 --> 00:20:02.760
And this is where really want to introduce this new design and engineering philosophy called adaptive engineering.

s174
00:20:03.480 --> 00:20:05.600
Before that, let's just take stock a bit.

s175
00:20:05.600 --> 00:20:12.320
So, everything we covered so far lives inside a particular framework, which is this kind of fixed or factory harness.

s176
00:20:12.320 --> 00:20:18.480
Uh the human engineer decides it all up front, the sequencing, the roles, capabilities.

s177
00:20:18.480 --> 00:20:20.600
That's really the factory model, and it works.

s178
00:20:20.600 --> 00:20:28.760
Um and we've seen that um because problem spaces that are complicated yet predictable is exactly right.

s179
00:20:28.760 --> 00:20:33.960
And I'd argue for most engineering problems, that is the perfect method.

s180
00:20:33.960 --> 00:20:51.320
But the moment AI becomes exponentially better, and it collides with the real world, which is, as we found, messy and complex, not just complicated, we need something genuinely new, a different idea of what optimal design and engineering

s181
00:20:51.320 --> 00:20:52.440
even is.

s182
00:20:52.440 --> 00:20:59.680
Um And it's obviously it rests on those two assumptions that models have become become exponentially better.

s183
00:20:59.840 --> 00:21:13.880
Uh and that AI will uh come outside of a sandbox situation, and actually um engineering will run in real time in contact with the social and physical environment.

s184
00:21:13.880 --> 00:21:20.400
Um so, essentially what we're saying is that the harness must adapt mid-flow.

s185
00:21:20.400 --> 00:21:22.840
And that's the crux of harness engineering.

s186
00:21:22.840 --> 00:21:37.640
And I guess if I were to crystallize um the definition, um adaptive engineering is the discipline of designing constraints to the extent that the harness emerges on its own,

s187
00:21:37.640 --> 00:21:50.200
stabilizes, and adapts as needed in response to the changing environment in ways that you could not specify in advance.

s188
00:21:50.400 --> 00:21:56.600
Essentially, the harness becomes the ongoing output rather than the input.

s189
00:21:56.600 --> 00:22:10.760
And you know, what this means in practice, uh just to make it a bit more concrete, is given that agents have capabilities um to interact with each other, organization

s190
00:22:10.760 --> 00:22:23.880
emerges from those interactions in response to the environment or a goal, such that new levels of order appear, ones that you could never have specified up front.

s191
00:22:23.880 --> 00:22:28.520
Um so, think bird in a flock, think agent and harness.

s192
00:22:28.520 --> 00:22:38.560
So, the agents are actually creating the harness in the same way as the birds are relating to one another so that the flock can emerge.

s193
00:22:38.560 --> 00:22:40.880
So, you don't build the harness anymore.

s194
00:22:40.880 --> 00:22:48.310
You let the agents form the harness that best fits the environment in that moment, in that context.

s195
00:22:48.310 --> 00:22:48.920
[snorts]

s196
00:22:48.920 --> 00:22:57.120
The harness then becomes a self-organizing, constantly evolving multi-agent system.

s197
00:22:57.400 --> 00:23:04.240
And if we're to just simulate what it may become, because this is when agents become a lot more capable.

s198
00:23:04.240 --> 00:23:10.040
Um essentially, you have agents that are isomorphic um but yet undifferentiated.

s199
00:23:10.040 --> 00:23:14.880
Um but fully capable of interacting and um learning and changing.

s200
00:23:14.880 --> 00:23:17.160
Um you let those agents interact.

s201
00:23:17.160 --> 00:23:18.400
At first, there's nothing.

s202
00:23:18.400 --> 00:23:21.000
It's a bit of few exchanges here and there.

s203
00:23:21.000 --> 00:23:30.680
But the coupling continues and slowly builds until you reach a point where roughly one connection on average um occurs um with each agent.

s204
00:23:30.680 --> 00:23:34.161
And suddenly, you start to see this whole emerge.

s205
00:23:34.161 --> 00:23:34.320
[snorts]

s206
00:23:34.320 --> 00:23:36.480
Um a system, you could say.

s207
00:23:36.480 --> 00:23:43.280
Uh and I guess as an engineer, your new lever uh becomes uh how the rate of coupling.

s208
00:23:43.280 --> 00:23:47.720
Like, do you dial it up or do you dampen it down?

s209
00:23:47.720 --> 00:23:52.680
Um again, the adaptive path focuses on the interactions.

s210
00:23:52.920 --> 00:23:59.120
Um until, you know, agents start to actually specialize with respect to one another.

s211
00:23:59.120 --> 00:24:02.880
So, let's say you got two agents doing the same thing in the same place.

s212
00:24:02.880 --> 00:24:05.320
That's just redundant and the environment knows that.

s213
00:24:05.320 --> 00:24:07.800
The pressure from the environment knows that.

s214
00:24:07.800 --> 00:24:11.760
So, it rewards anything that essentially breaks the tie.

s215
00:24:11.760 --> 00:24:25.600
Um so I guess a tiny difference um you know, who got there first, who is slightly better gets amplified by the feedback until those two agents are no longer interchangeable.

s216
00:24:25.600 --> 00:24:31.840
And so, the pool stops becoming something uniform and it actually falls into uh niches.

s217
00:24:31.840 --> 00:24:39.000
And really, the key insight here is that the agent's identity isn't something you gave it.

s218
00:24:39.000 --> 00:24:45.280
It's actually the position, role, or capability it took relative to the others and its environment.

s219
00:24:45.280 --> 00:24:51.040
So, here we see specialization emerges from the interaction.

s220
00:24:51.720 --> 00:24:55.640
And if you go on a bit further, uh they stop connecting at random.

s221
00:24:55.640 --> 00:25:01.680
Uh in fact, they start to work well together um, in terms of sort of clusters.

s222
00:25:01.680 --> 00:25:09.800
Um, and you see the first boundaries appear and crucially the system drew them, you didn't as an engineer ahead of time.

s223
00:25:09.800 --> 00:25:12.600
Um, so these are kind of emergent clusters.

s224
00:25:12.600 --> 00:25:23.840
Um, until a new convention crystallizes, you know, a division of norms and protocols, and the system produce some level of governance without a governor.

s225
00:25:23.840 --> 00:25:36.280
Conventions emerge spontaneously from the local coordination with no central authority because as we've seen, central authority can lead to brittleness in a changing environment.

s226
00:25:36.280 --> 00:25:43.280
Um, so there's just enough repeated interaction to tip the group into some sort of shared norm.

s227
00:25:43.280 --> 00:25:55.440
And this is crucial really because it means that it can continue to adapt in a decentralized way based on the changing environment or problem space.

s228
00:25:57.720 --> 00:26:14.440
And essentially a new um, emergent order um, arises from those tiny little interactions amongst those agents, you have a new level of organization that is fairly stable up until the next iteration, up until the environment changes

s229
00:26:14.440 --> 00:26:20.080
and um, the order needs to essentially restructure itself.

s230
00:26:23.920 --> 00:26:26.120
So the question is that what is the role of an engineer?

s231
00:26:26.120 --> 00:26:34.880
And and really in adaptive engineering, uh, you don't abolish the engineer, you're just relocating the emphasis of engineering.

s232
00:26:34.880 --> 00:26:43.760
So if you can exploit the model's capabilities, um, so agents can essentially um, interact, learn, and change.

s233
00:26:43.760 --> 00:26:51.440
What you're doing as an engineer is essentially um, exploring the constraints, you're dictating the constraints, the rules of the game.

s234
00:26:51.440 --> 00:27:01.143
You're not deciding what should happen or what you want to happen, but you're giving the agent space to explore that field.

s235
00:27:01.143 --> 00:27:01.440
[clears throat]

s236
00:27:01.440 --> 00:27:15.600
And then, once that emergent harness uh starts to take shape, you are sensing it and responding to that rather than um stopping the engineering process and starting from scratch.

s237
00:27:15.600 --> 00:27:21.520
Um And I think this slide sort of um portrays that.

s238
00:27:21.520 --> 00:27:25.200
This is not binary, this is not either or.

s239
00:27:25.200 --> 00:27:29.120
Uh in fact, it's very much a continuum.

s240
00:27:29.120 --> 00:27:37.640
Um if you you there's no purely fixed harness way, and there's no purely uh completely adaptive autonomous way.

s241
00:27:37.640 --> 00:27:48.080
Um fixed engineering is essentially in one hand um prescribing the structure, which agents run and you rarely change the structure mid-engineering.

s242
00:27:48.080 --> 00:27:57.240
Um whereas on the other end of the scale, you've got adaptive engineering where you allow the agents to freely interact and allow the harness or the structure that guides them

s243
00:27:57.240 --> 00:28:01.880
uh to be created by them and change mid-engineering.

s244
00:28:02.160 --> 00:28:10.000
Uh so it's not like a swarm of agents loose with no roles and this sort of intelligence that magically appears.

s245
00:28:10.000 --> 00:28:15.280
Um and it's not sort of better one one way is not better than the other.

s246
00:28:15.280 --> 00:28:17.520
They just have two different use cases.

s247
00:28:17.520 --> 00:28:30.320
And um if I could just, you know, bring it back to some examples of existing uh harnesses, so for example, Hermes AI um on on their website, it states that it's a self-improving

s248
00:28:30.320 --> 00:28:34.280
AI agent that creates skills from experience and learns from it.

s249
00:28:34.280 --> 00:28:38.400
So, obviously, this is a huge step in terms of adaptive direction.

s250
00:28:38.400 --> 00:28:50.200
Um but I do want to make a distinction here um because um that adaptation is in the form of vertical intelligence, which is about making individual agents smarter.

s251
00:28:50.200 --> 00:28:57.760
What I'm talking about here is horizontal intelligence, which is all about how groups of agents coordinate.

s252
00:28:57.760 --> 00:29:01.280
And so, those two directions are a bit orthogonal.

s253
00:29:01.280 --> 00:29:11.280
Um And the thesis I'm presenting is that it's horizontal intelligence uh that is the most adaptive method, the most agile method.

s254
00:29:11.280 --> 00:29:16.920
Um and and and arguably the higher leverage point.

s255
00:29:17.160 --> 00:29:20.920
Uh compared to vertical.

s256
00:29:20.960 --> 00:29:34.200
Um Another example is Pi, um you know, a great harness, uh which is uh sort of essentially minimal, um yet it's maximally extensible.

s257
00:29:34.200 --> 00:29:37.360
Um so, it's fully customizable to an extent.

s258
00:29:37.360 --> 00:29:41.640
Um so, you could class that as adaptive, but again, there's a distinction here.

s259
00:29:41.640 --> 00:29:48.560
There's adaptive at the design stage, like when you're preparing for that engineering process.

s260
00:29:48.560 --> 00:29:55.320
Um Uh where the engineer can customize and the tool's quite malleable.

s261
00:29:55.320 --> 00:30:09.480
But, when it comes to adaptive engineering, that's all about being adaptive during the run time, where the system reorganizes itself whilst running in response to the changing problem space of the environment.

s262
00:30:09.480 --> 00:30:17.560
And you know, Pi is a strong example of the first kind, um but it doesn't make any claims um to be anything other.

s263
00:30:17.560 --> 00:30:23.080
Uh it's not a self-organizing multi-agent system yet.

s264
00:30:24.760 --> 00:30:30.000
I'm just going to skip over some of the uh nitty-gritties of what that could mean.

s265
00:30:30.000 --> 00:30:40.800
Um so, for example, um uh when it comes to agent capabilities, you know, obviously, they need to interact, learn, and change.

s266
00:30:40.800 --> 00:30:52.600
That's kind of a prerequisite for adaptive engineering and I don't doubt that that will become the case as models get exponentially more powerful.

s267
00:30:52.600 --> 00:31:02.120
There's a few things that the designer can actually do in the form of constraints and I'm not going to list out all the different types of constraints cuz they're sort of largely dictated by

s268
00:31:02.120 --> 00:31:05.800
some of the properties like the roles and the sequencing and the memory.

s269
00:31:05.800 --> 00:31:09.640
These are all sort of categories or properties of harness.

s270
00:31:09.640 --> 00:31:22.000
But essentially there are three questions that you could ask yourself as an engineer which is do you want to enable the agents more or do you want to close them in and govern them more

s271
00:31:22.000 --> 00:31:24.880
and produce more give them more guardrails.

s272
00:31:24.880 --> 00:31:35.400
Are you rewarding them to cohere towards a particular goal or are you costing them if they fall outside a particular container?

s273
00:31:35.400 --> 00:31:40.000
And how fast or slow do you want this um to happen?

s274
00:31:40.000 --> 00:31:44.240
Some of the properties, you know, what's the speed for example the rate of coupling.

s275
00:31:44.240 --> 00:31:50.000
So these are some of the engineering tweaks that you can make when it comes to adaptive engineering.

s276
00:31:50.600 --> 00:31:58.920
And then finally like this emergent harness that is constantly shaping and reshaping itself based on the problem space.

s277
00:31:58.920 --> 00:32:01.880
You can only really sense and respond to that emergent harness.

s278
00:32:01.880 --> 00:32:19.280
You're not going to completely be able to I guess edit it in some sort of hard engineering fashion cuz that's not in the best interest of you as an adaptive engineer.

s279
00:32:21.560 --> 00:32:29.800
This slide is just a more detailed comparison of both approaches and again I want to emphasize that one is not better than the other.

s280
00:32:29.800 --> 00:32:34.440
It's just two different design engineering philosophies.

s281
00:32:34.560 --> 00:32:43.880
Where on one hand in the factory model, um you know, the structure is imposed up front by the engineer, and so the harness becomes the input.

s282
00:32:43.880 --> 00:32:49.720
Um whereas in the adaptive path, the harness emerges and is essentially the output.

s283
00:32:49.720 --> 00:32:59.200
Um And And one of the crucial things here is that control is decentralized in the [clears throat] adaptive path.

s284
00:32:59.200 --> 00:33:14.640
Um in the field continuously um because the problem space eventually when AI um comes or AI engineering comes off a screen into the real world, um the problem space is constantly changing.

s285
00:33:14.640 --> 00:33:22.480
Um Now, it's important to understand that, you know, there are particular use cases for this.

s286
00:33:22.480 --> 00:33:27.480
Um but there are a huge number of failure modes um in this as well.

s287
00:33:27.480 --> 00:33:39.680
Um you know, just to list a few of those failure modes, you know, emergence leans towards uh stability, what we call an attractor, uh where you find a structure

s288
00:33:39.680 --> 00:33:45.080
that feels stable and optimal, but that doesn't necessarily mean it's the best.

s289
00:33:45.080 --> 00:33:49.520
Um you need some sort of genuine selection pressure.

s290
00:33:49.520 --> 00:33:59.800
So, if you imagine in the social world or the physical or the natural world, the environment is um giving us certain uh pressures.

s291
00:33:59.800 --> 00:34:02.600
You know, that's what encourages us to adapt.

s292
00:34:02.600 --> 00:34:03.680
That is evolution.

s293
00:34:03.680 --> 00:34:06.400
You know, there is some sort of selection pressure here.

s294
00:34:06.400 --> 00:34:15.360
And you kind of need to find that um in or we as a community need to find that in adaptive engineering, otherwise you just get drift.

s295
00:34:15.360 --> 00:34:22.320
There is a risk of monoculture, where you don't get genuine diversity amongst agents because they're all trained on the same data.

s296
00:34:22.320 --> 00:34:25.080
Um legibility does collapse.

s297
00:34:25.080 --> 00:34:34.879
Um you know, you as adaptability increases, um you can't really pin down or explain things that well.

s298
00:34:34.879 --> 00:34:36.840
And again, that's a feature of complex systems.

s299
00:34:36.840 --> 00:34:40.280
You can't really pin them down.

s300
00:34:40.760 --> 00:34:49.040
And and and obviously goes without saying that there's no more predictability ahead of runtime because things are constantly moving.

s301
00:34:49.040 --> 00:35:05.280
Um And and and these are the real failure modes that we must apprehend because I do feel like adaptive engineering is something that we're going to be more inclined towards and it's going to race forward

s302
00:35:05.280 --> 00:35:08.469
um from the future.

s303
00:35:08.469 --> 00:35:10.469
[clears throat and cough]

s304
00:35:13.000 --> 00:35:15.560
I guess a bit of a takeaway then.

s305
00:35:15.560 --> 00:35:26.560
Um you know, as models improve exponentially um AI engineering will move into the real world scenarios.

s306
00:35:26.560 --> 00:35:33.120
You know, the real world where it touches um the physical space uh on an ongoing basis.

s307
00:35:33.120 --> 00:35:38.280
It's uh in contact with multiple agents across different institutions.

s308
00:35:38.280 --> 00:35:40.280
It's in contact with multiple human beings.

s309
00:35:40.280 --> 00:35:46.200
So So there's a real cost peer where AI is heading towards.

s310
00:35:46.200 --> 00:35:54.974
And so the discipline of design and engineering is going to be forced to rethink itself around continual production.

s311
00:35:54.974 --> 00:35:55.320
[snorts]

s312
00:35:55.320 --> 00:36:03.040
And the limiting factor which is the case now, but probably more so in the future, is not going to be the strength of the model.

s313
00:36:03.040 --> 00:36:07.400
It's going to be the adaptability of the harness.

s314
00:36:07.400 --> 00:36:16.520
Um and adaptability really here means multi-agent um which is essentially about coordinating amongst agents.

s315
00:36:16.520 --> 00:36:22.280
A multi-agent orchestration which is decentralized.

s316
00:36:22.320 --> 00:36:28.640
Um and it's not adaptable ahead of runtime.

s317
00:36:28.640 --> 00:36:35.800
In fact, it starts becoming adaptable mid run-time or during an entire engineering a process.

s318
00:36:35.800 --> 00:36:37.320
And I think that's real intelligence.

s319
00:36:37.320 --> 00:36:45.080
That's Well, when I say real intelligence, I mean that's the intelligence when it comes to adaptability of the harness.

s320
00:36:46.320 --> 00:36:56.400
Uh, I'd be really interested in um hearing your thoughts on this um and whether this is something that uh other people are also exploring.

s321
00:36:56.400 --> 00:36:58.920
Um please do reach out to me.

s322
00:36:58.920 --> 00:37:01.160
Thank you.