WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=NaOkR3VSfR4

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/NaOkR3VSfR4.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.560 --> 00:00:14.040
Cool.

s3
00:00:14.040 --> 00:00:16.440
Thank you all for joining this morning.

s4
00:00:16.440 --> 00:00:19.760
I know it's been an exciting week.

s5
00:00:19.760 --> 00:00:25.480
A lot of talk about agents, building agents, coding agents and everything.

s6
00:00:25.480 --> 00:00:32.680
And what I want to talk in the next 15-20 minutes is is about running agents in production.

s7
00:00:32.680 --> 00:00:42.760
Um And things that is kind of on the top of pretty much everybody's mind when it comes to building agents and running agents is you know, harnesses.

s8
00:00:43.360 --> 00:00:47.640
Before I get started, like I would like to ask in the audience, right?

s9
00:00:47.640 --> 00:00:52.960
How many of our many of you are running agents in production today?

s10
00:00:53.040 --> 00:00:53.320
Awesome.

s11
00:00:53.320 --> 00:00:54.360
So, a lot of hands up.

s12
00:00:54.360 --> 00:00:59.320
Um I was in another conference couple of weeks ago and I asked the same question.

s13
00:00:59.320 --> 00:01:02.680
I think I had like one hand, so this is pretty good.

s14
00:01:02.680 --> 00:01:05.199
It seems like, you know, things are moving forward.

s15
00:01:05.199 --> 00:01:08.040
Um All right.

s16
00:01:08.040 --> 00:01:10.560
So, when we think about agents, right?

s17
00:01:10.560 --> 00:01:22.480
Like if we go back to kind of the hello world equivalent examples of agents, uh it's always starts with, you know, an agent with a tool call and how well an LLM kind of decides to call the tool,

s18
00:01:22.480 --> 00:01:26.880
maybe use some sort of context or a memory and and plans things ahead, right?

s19
00:01:26.880 --> 00:01:35.560
Um What What we start to kind of realize is is you know, that's a very small narrow picture if you think about it.

s20
00:01:35.560 --> 00:01:39.480
When you think about agents in production, it's not just always about chatbots.

s21
00:01:39.480 --> 00:01:47.760
The early examples that we had seen were all about like, you know, hey, we are going to build a customer service chatbot or some sort of chatbot.

s22
00:01:47.760 --> 00:01:58.600
Um But when we start thinking about agents in a broader terms and picture, think about background agents, background workers, agents that are running on schedule.

s23
00:01:58.600 --> 00:02:07.720
Um I would like to have an agent that runs every hour to check uh what's happening with my schedule and if there is something new that has popped up that I should think about.

s24
00:02:07.720 --> 00:02:09.240
Uh event-driven agents, right?

s25
00:02:09.240 --> 00:02:11.560
Agents that are listening on various events.

s26
00:02:11.560 --> 00:02:17.800
Um to give you an example, I have a production system running where my alerts are firing every now and then.

s27
00:02:17.800 --> 00:02:20.560
I have a logs coming through hotel.

s28
00:02:20.560 --> 00:02:25.800
I would like my agents to listen to those events and react on it and see what's going on there, right?

s29
00:02:25.800 --> 00:02:32.840
Um I could have long-running coordinators and what I mean by that is I have an agent running for a longer period of time.

s30
00:02:32.840 --> 00:02:37.480
I would like to have an agent monitoring that agent to see, "Hey, what's going on there?

s31
00:02:37.480 --> 00:02:42.000
And should I nudge that agent to kind of move forward, take a different action?"

s32
00:02:42.000 --> 00:02:43.560
And things like that, right?

s33
00:02:43.560 --> 00:02:48.520
Um also one more thing is about multi-agent systems, right?

s34
00:02:48.520 --> 00:02:50.440
Agents talking to other agents.

s35
00:02:50.440 --> 00:02:56.920
Uh the last thing you want is to have an agent be responsible for multiple things and risk hallucinations.

s36
00:02:56.920 --> 00:03:01.200
Uh So, more we think about agents, right?

s37
00:03:01.200 --> 00:03:09.120
Uh the way I would like to think about is the agent starts to feel more like an application, not just a component, right?

s38
00:03:09.120 --> 00:03:18.440
Um If you go back to the old school thought of microservices, when we and probably we are still building microservices, right?

s39
00:03:18.440 --> 00:03:23.519
Um you are not going to ship a microservice in production that does everything, right?

s40
00:03:23.519 --> 00:03:26.519
It kind of stops being a microservice at that point in time.

s41
00:03:26.519 --> 00:03:30.640
Uh typically, we are shipping multiple microservices.

s42
00:03:30.640 --> 00:03:34.959
They are talking to each other through choreography or orchestration.

s43
00:03:34.959 --> 00:03:48.040
And you have an application that is relying on that the set of microservices, uh which essentially are single responsibility principles uh to kind of deliver the business goals.

s44
00:03:48.040 --> 00:03:53.040
There is kind of a very clear uh parallel to that when it comes to agent harnesses, right?

s45
00:03:53.040 --> 00:03:57.600
If you think about a harness that is responsible for achieving a specific goal.

s46
00:03:57.600 --> 00:04:02.400
So, let's say I have an harness for doing my SRE work.

s47
00:04:02.400 --> 00:04:06.160
Um, it's operating on multiple agents to see what's going on there, right?

s48
00:04:06.160 --> 00:04:15.800
Maybe it's listening on events coming from my logging system, talking to my Kubernetes cluster as an agent uh, to find out, you know, what's going on there.

s49
00:04:15.800 --> 00:04:23.080
Uh, my metric systems, my customer service uh, dashboards and and so on and so forth.

s50
00:04:23.080 --> 00:04:26.840
Um, and what I want to drive there is that if you think about it, right?

s51
00:04:26.840 --> 00:04:29.800
Your harness is the application and vice versa, right?

s52
00:04:29.800 --> 00:04:39.040
An agent alone doesn't kind of deliver the whole premise, but rather when you start putting things together and you build an harness that controls the execution of the agent,

s53
00:04:39.040 --> 00:04:44.880
delivers on the premise, that's where you start thinking about um, an application, right?

s54
00:04:44.880 --> 00:04:52.400
And which is you have systems uh, which are your databases, your internal systems, enterprise systems.

s55
00:04:52.400 --> 00:04:54.520
You have humans, human in the loop.

s56
00:04:54.520 --> 00:05:00.560
Um, you have tools through either APIs or MCP that is all kind of playing together.

s57
00:05:00.560 --> 00:05:02.320
Um, and as I mentioned, right?

s58
00:05:02.320 --> 00:05:05.400
Harness runs more than just an agent loop.

s59
00:05:05.400 --> 00:05:11.720
And this is an important thing that I kind of learned uh, while trying to build harnesses for real-world production use cases.

s60
00:05:11.720 --> 00:05:17.320
It's a combination of both deterministic and non-deterministic part of the application, right?

s61
00:05:17.320 --> 00:05:24.000
Non-determinism is delivered by an LLM in terms of the reasoning, in terms of thinking process.

s62
00:05:24.000 --> 00:05:28.200
Um, and then there is non-deterministic and there is deterministic part, right?

s63
00:05:28.200 --> 00:05:32.200
Which you do not want non-determinism seeping into it, right?

s64
00:05:32.200 --> 00:05:34.000
Think about payments.

s65
00:05:34.000 --> 00:05:49.960
Um, or let's say if I have an agent harness that is responsible for monitoring my production deployment, how I manage my Kubernetes clusters, I probably want to have a very well-defined workflow in terms of what are the sequence of steps that I execute

s66
00:05:49.960 --> 00:05:51.760
when I want to restart my cluster.

s67
00:05:51.760 --> 00:05:56.560
And that's a very deterministic set of processes which every time it runs, I know exactly what it does.

s68
00:05:56.560 --> 00:06:02.760
So, I want a determinism to be delivered by my harness when it matters.

s69
00:06:03.240 --> 00:06:06.680
Um and of course the harness is a long running process, right?

s70
00:06:06.680 --> 00:06:07.840
It runs across the time.

s71
00:06:07.840 --> 00:06:19.840
It can run um anywhere from few seconds if it's uh something very quick like hey, check what's going on here to all the way running for days, months, uh even longer than that, right?

s72
00:06:19.840 --> 00:06:30.360
Think about long running processes, order management systems where you are waiting on third party services to deliver your shipment or waiting on humans to take actions and approve things

s73
00:06:30.360 --> 00:06:31.520
and so on and so forth.

s74
00:06:31.520 --> 00:06:33.600
Or Harness is just waiting, right?

s75
00:06:33.600 --> 00:06:34.880
It's waiting for events to happen.

s76
00:06:34.880 --> 00:06:38.840
So, when that event happens you take an action and do something around it.

s77
00:06:38.840 --> 00:06:42.640
So, that brings in another important point, right?

s78
00:06:42.640 --> 00:06:46.200
That when you think about long running systems, you need durability, right?

s79
00:06:46.200 --> 00:06:54.240
You want to be able to recover when things fail because at the end of the day this harnesses are running somewhere in your entire infrastructure stack, right?

s80
00:06:54.240 --> 00:06:59.240
Maybe in the cloud, maybe on a sandbox but those things can go up, go down.

s81
00:06:59.240 --> 00:07:02.919
There could be network failures, partitioning, anything that could be happening.

s82
00:07:02.919 --> 00:07:04.960
So, you need the harnesses to be durable.

s83
00:07:04.960 --> 00:07:08.520
And one thing about durability here is it's a table stake thing, right?

s84
00:07:08.520 --> 00:07:11.320
At the end of the day durability is a cost of admission.

s85
00:07:11.320 --> 00:07:14.360
That's not the feature that you're looking for in a harness.

s86
00:07:14.360 --> 00:07:17.160
Um And as I mentioned, right?

s87
00:07:17.160 --> 00:07:25.360
The loop that the harness runs spans across agent um and everything else.

s88
00:07:27.680 --> 00:07:33.680
So, now let's think about uh how the harnesses kind of operate, right?

s89
00:07:33.680 --> 00:07:36.480
Um If you think about a harness, right?

s90
00:07:36.480 --> 00:07:40.480
And and a loop, essentially what it is doing it it has a state of the world.

s91
00:07:40.480 --> 00:07:43.480
It knows what work been completed.

s92
00:07:43.480 --> 00:07:47.560
Um what has been recorded in terms of the side effects, right?

s93
00:07:47.560 --> 00:07:51.880
So, like if I did a cluster restart, uh I know I have that recorded.

s94
00:07:51.880 --> 00:07:54.800
Uh if I sent an email, I know that has happened.

s95
00:07:54.800 --> 00:08:02.800
Um I know what worked or didn't work, and then based on the current state of the world and the goal, I know what needs to happen next, right?

s96
00:08:02.800 --> 00:08:07.520
And that's where the reasoning and LLM comes into the picture.

s97
00:08:08.160 --> 00:08:15.280
Um and what really happens here is that if you think about a clear distinction, and this is the most important thing.

s98
00:08:15.280 --> 00:08:26.720
If If one thing that I would like everyone to take away from here is this slide, which is that the responsibility of a non-deterministic agent is to plan, is to plan what should happen next,

s99
00:08:26.720 --> 00:08:28.520
not really to do things.

s100
00:08:28.520 --> 00:08:32.520
Um and then Harness is the one who does actually execution.

s101
00:08:32.520 --> 00:08:36.880
And And this is important for uh various reasons, right?

s102
00:08:36.880 --> 00:08:45.800
One being that Harness is a deterministic piece of code that actually understands what is involved in, you know, actually executing a piece of work.

s103
00:08:45.800 --> 00:08:47.240
So, as I mentioned, right?

s104
00:08:47.240 --> 00:09:00.920
If I am trying to uh build a Harness that runs my uh DevOps or SRE uh agents, uh when the agent says that, "Hey, this cluster is unhealthy and you should restart."

s105
00:09:00.920 --> 00:09:10.839
Harness should decide how to restart the cluster, what involves in restarting cluster, and that has to be, at least in my world, has to be very deterministic set of processes, right?

s106
00:09:10.839 --> 00:09:13.440
So, that every cluster restart is exactly same.

s107
00:09:13.440 --> 00:09:16.560
There is no other uh category there, right?

s108
00:09:16.560 --> 00:09:26.960
Um Also, if it needs to have an approval, for example, if I am trying to restart a production cluster, I probably want to have a human gate, uh probably send a Slack message

s109
00:09:26.960 --> 00:09:29.280
to somebody to say, "Hey, I'm going to restart this cluster.

s110
00:09:29.280 --> 00:09:31.360
Do you think it's okay to do that or not, right?"

s111
00:09:31.360 --> 00:09:36.720
And I do not want this to be left to a hallucination by an LLM that, you know, it doesn't need to do it.

s112
00:09:36.720 --> 00:09:39.080
It needs to be guaranteed in terms of execution.

s113
00:09:39.080 --> 00:09:43.000
So, it has to be very deterministic when it comes to those kind of things.

s114
00:09:43.000 --> 00:09:50.400
Uh ideally, I want this to be idempotent and if not, I want it to be recording that, you know, it is not idempotent and this is what has happened.

s115
00:09:50.400 --> 00:09:55.160
So, I can get take care of the side effects or I can handle the side effects separately, right?

s116
00:09:55.160 --> 00:09:58.440
Uh So, this is the most important thing, right?

s117
00:09:58.440 --> 00:10:00.840
The clear separation between as we would like to call, right?

s118
00:10:00.840 --> 00:10:01.960
The brains and the hands.

s119
00:10:01.960 --> 00:10:05.480
Harness is the hands, the brain is the LLM.

s120
00:10:05.480 --> 00:10:20.560
Um So, yeah, if the agent writes the plan, uh Harness executes the plan, uh and I'll show you in a brief uh a short demo in terms of how all of these things kind of matches together.

s121
00:10:20.560 --> 00:10:24.040
Um But, is this a new concept?

s122
00:10:24.040 --> 00:10:26.880
Like, if you think about it, this is not necessarily a new concept.

s123
00:10:26.880 --> 00:10:28.960
This has been around for a while, right?

s124
00:10:28.960 --> 00:10:34.760
If you think about workflows as sagas, which we used to write ourselves and define exactly what happens.

s125
00:10:34.760 --> 00:10:38.320
It was a very and it is a very deterministic set of processes.

s126
00:10:38.320 --> 00:10:41.880
When you think about agentic harnesses, they are essentially late bound sagas.

s127
00:10:41.880 --> 00:10:50.760
And what I mean by that is that they have a very finite and deterministic set of tools uh and the task that they can operate on.

s128
00:10:50.760 --> 00:10:56.920
Instead of putting them together upfront, the agent is kind of proposing and building them at runtime.

s129
00:10:56.920 --> 00:10:59.720
Um and therefore, they are essentially late bound sagas, right?

s130
00:10:59.720 --> 00:11:13.520
Uh but you get all the benefits of a traditional saga in a workflow out of the box in terms of visibility, what's happening, being able to control things, and iterating upon like, you know, how far ahead in the future

s131
00:11:13.520 --> 00:11:15.480
uh the agent is able to plan.

s132
00:11:15.480 --> 00:11:20.640
You are able to kind of plan one step at a time or multiple steps at a time.

s133
00:11:21.080 --> 00:11:24.480
Um And yeah, if you think about it, right?

s134
00:11:24.480 --> 00:11:26.200
It's it's more like a branching workflow.

s135
00:11:26.200 --> 00:11:37.920
If only you could build a workflow with every possible combination of a branch, um then, you know, it kind of builds that thing for you right versus with agents that kind of simplifies the work.

s136
00:11:37.920 --> 00:11:49.120
If you have n number of tools it can do n different combinations of executions which otherwise is going to be almost impossible to you know think ahead of time and and do it.

s137
00:11:49.840 --> 00:11:51.200
So let's take a look at it right.

s138
00:11:51.200 --> 00:11:57.880
So what I'm going to do is I'll quickly show a completed run.

s139
00:12:02.720 --> 00:12:05.600
Of what I talked about earlier right.

s140
00:12:05.600 --> 00:12:11.680
Here is one of my example agent that essentially is acting like an SRE agent right.

s141
00:12:11.680 --> 00:12:21.960
What its job is to do is understand what's going on in the current system and try to plan what should happen next and execute on those things.

s142
00:12:21.960 --> 00:12:25.640
So it's essentially a remediation loop that runs twice.

s143
00:12:25.640 --> 00:12:41.160
In the first iteration it tries to understand the root cause of the problem, tries to react to that, observes the output of it and then runs another loop and plans another set of steps to see what should happen next right.

s144
00:12:41.160 --> 00:12:46.960
So as an input to my agent let's see what was the input given right here.

s145
00:12:49.160 --> 00:12:53.080
So step number one is you know it makes a call to LLM.

s146
00:12:53.080 --> 00:12:56.360
So as you can see right like it's an SRE agent.

s147
00:12:56.360 --> 00:13:02.360
Of course it's a demo so you know everything is kind of pre-planned and a can demo.

s148
00:13:02.360 --> 00:13:09.240
The output of LLM as you can see right are the steps what it should do.

s149
00:13:09.240 --> 00:13:19.200
It's talking about so as you can see one more thing here right is that instead of just doing one step at a time essentially it is proposing a sequence of steps to do.

s150
00:13:19.200 --> 00:13:30.800
You should gather evidences, you should analyze logs and if required based on the evidence that you gathered you should either roll back a deployment and then verify recovery.

s151
00:13:30.800 --> 00:13:35.560
This is then given to a specialized tool called plan and compile.

s152
00:13:35.560 --> 00:13:39.080
So, this is the plan that agent gave.

s153
00:13:39.080 --> 00:13:42.880
This gets compiled into a very deterministic workflow.

s154
00:13:42.880 --> 00:13:45.560
We are relying on a conductor as a workflow execution engine here.

s155
00:13:45.560 --> 00:13:49.120
So, the output is a fully runnable workflow.

s156
00:13:49.120 --> 00:13:50.160
This gets executed.

s157
00:13:50.160 --> 00:13:54.360
So, like you know, what I'm going to show you here very quickly is how that looks like.

s158
00:13:54.360 --> 00:14:02.720
Um So, this is the first step of execution, right?

s159
00:14:02.720 --> 00:14:05.600
If we look at it here as I said, right?

s160
00:14:05.600 --> 00:14:15.440
Like we did analyze the logs, look at the query of the metrics, decided to do a rollback and verify recovery and everything.

s161
00:14:15.440 --> 00:14:17.120
So, step number one is completed.

s162
00:14:17.120 --> 00:14:19.640
It runs another loop.

s163
00:14:19.640 --> 00:14:22.840
Um same thing goes here.

s164
00:14:22.840 --> 00:14:32.200
Um In the next iteration, it decides that, okay, looks like, you know, these things have been done.

s165
00:14:32.200 --> 00:14:33.680
Let's check the downstream.

s166
00:14:33.680 --> 00:14:36.880
And if required, verify the recovery and complete it, right?

s167
00:14:36.880 --> 00:14:40.839
Um And then the next one runs and completes the task.

s168
00:14:40.839 --> 00:14:46.400
But here is an example of a loop that is self uh kind of uh planning, right?

s169
00:14:46.400 --> 00:14:50.480
So, and to show you very clearly what's going on here, here is a loop.

s170
00:14:50.480 --> 00:14:56.080
Um At every step of the way, essentially, it is looking at the current state of the world.

s171
00:14:56.080 --> 00:15:07.520
So, this is my agentic loop that is kind of understanding the world, planning and executing uh one step instead of one step at a time, it is actually planning multiple steps at a time, executing that,

s172
00:15:07.520 --> 00:15:09.600
verifying, and completing the loop.

s173
00:15:09.600 --> 00:15:15.080
Um Now, this can run in production as many times as you want.

s174
00:15:15.080 --> 00:15:16.680
This can be running on the events.

s175
00:15:16.680 --> 00:15:18.480
It can be running on a schedule.

s176
00:15:18.480 --> 00:15:19.560
Um As I said, right?

s177
00:15:19.560 --> 00:15:27.480
Like this the whole thing could be running for a much longer period of time um as opposed to just running in a short period of time.

s178
00:15:30.320 --> 00:15:30.800
All right.

s179
00:15:30.800 --> 00:15:34.200
So, that's towards the end of it.

s180
00:15:34.200 --> 00:15:38.080
This is kind of overview of what we just did here.

s181
00:15:38.080 --> 00:15:46.400
As you can see right in the second iteration, we decided not to do a rollback because it was already done in the first iteration.

s182
00:15:46.920 --> 00:15:54.560
But yeah, that's about the harnesses and agents and how they bring determinism to your production application.

s183
00:15:54.560 --> 00:15:57.480
The example that I showed you runs on conductor.

s184
00:15:57.480 --> 00:16:05.480
Conductor is an open source workflow orchestration platform that supports building agentic loops as well as agentic systems.

s185
00:16:05.480 --> 00:16:06.920
It's fully open source.

s186
00:16:06.920 --> 00:16:09.440
Orkes, we provide enterprise edition.

s187
00:16:09.440 --> 00:16:11.800
But feel free to try it out.

s188
00:16:11.800 --> 00:16:13.120
Here's a QR code.

s189
00:16:13.120 --> 00:16:14.600
Give it a try.

s190
00:16:14.600 --> 00:16:18.280
Join our Slack and you know, if you have questions, happy to help.

s191
00:16:18.280 --> 00:16:21.000
We have a booth here at Orkes.

s192
00:16:21.000 --> 00:16:23.960
Drop by if you want to see a live demo.

s193
00:16:23.960 --> 00:16:30.520
If you want to run your LangChain agents or OpenAI agents or any kind of agents, Conductor can handle all of those things.

s194
00:16:30.520 --> 00:16:32.680
Thank you.