WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=L173Z8DpaJg

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/L173Z8DpaJg.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.600 --> 00:00:14.800
Uh hi, welcome to this session.

s3
00:00:14.800 --> 00:00:18.920
Uh it is mysteriously titled The Agent Behind the Curtain.

s4
00:00:18.920 --> 00:00:24.680
Um it's about how the team at Warp built our cloud agent platform.

s5
00:00:24.680 --> 00:00:32.759
Um before I talk about the how though, I want to talk about the why um and share a little bit about my own background.

s6
00:00:32.759 --> 00:00:36.880
Um so, I've spent the past 8 years building developer tooling.

s7
00:00:36.880 --> 00:00:43.800
Um I started off in open source in the Python and data science space um on the Jupiter Notebook core team.

s8
00:00:43.800 --> 00:00:46.480
I was a maintainer on the Interact project.

s9
00:00:46.480 --> 00:00:53.560
That work continued in my time at Microsoft helping build APIs and SDKs for web developers.

s10
00:00:53.560 --> 00:00:59.400
Um and now I'm working on bringing AI AI agents to the cloud at Warp.

s11
00:00:59.400 --> 00:01:10.280
And one of the lessons that I've learned in my time building developer tooling is that really good dev tools meet devs where they are and grow with them.

s12
00:01:10.280 --> 00:01:19.240
Um a tool that you're going to be using every day should accommodate your workflows, but also be really adaptable as the like complexity and the nature of the work

s13
00:01:19.240 --> 00:01:20.880
changes.

s14
00:01:20.880 --> 00:01:27.240
And it's important for us to build really great dev tools because dev tools have a compounding effect on the world.

s15
00:01:27.240 --> 00:01:36.920
If you make a piece of software that helps a developer or a builder do great work, your ability to magnify how much great software exists in the world increases.

s16
00:01:36.920 --> 00:01:41.920
So, I think it is super important that these dev tools accommodate people's workflows.

s17
00:01:41.920 --> 00:01:45.520
And people's workflows are important because they love their preferences.

s18
00:01:45.520 --> 00:01:50.720
You might have a preference for a shell, a language, a harness, a review process.

s19
00:01:50.720 --> 00:01:58.840
And something that adapts to your workflows and preferences is going to be more enjoyable to use and more enjoyable to build software with.

s20
00:01:58.840 --> 00:02:02.360
And it also becomes like a part of how you think and build.

s21
00:02:02.360 --> 00:02:05.840
Um so, it's super important to be attuned to that.

s22
00:02:05.840 --> 00:02:17.000
And this notion of tools meeting developers where they are and growing with them comes to light really clearly in the progression of AI tooling in this space.

s23
00:02:17.000 --> 00:02:23.040
So, this is the story for Warp specifically, but it's also the story for a lot of developer tools.

s24
00:02:23.040 --> 00:02:30.040
Pre-AI era, we had the Warp terminal, which kind of met people in their command-line workflows that they were used to.

s25
00:02:30.040 --> 00:02:38.640
And then this fantastic thing happened where AI got introduced to developers, and now you had a whole new set of tools that was available to you locally on your machine.

s26
00:02:38.640 --> 00:02:46.240
And a lot of people started to interact with agentic coding patterns in their terminal, in their IDE, in their editor.

s27
00:02:46.240 --> 00:02:55.360
And then eventually, we realized that we had kind of reached the limits of what we could do on our laptops, and we wanted agents to do work that was more long-running,

s28
00:02:55.360 --> 00:03:01.200
that was adapted to different constraints, and that work needed to happen in the cloud.

s29
00:03:01.200 --> 00:03:13.000
Um and whenever you send anything to the cloud, you adopt a lot of complexity um because running things in the cloud requires us to navigate a much messier stack of infrastructure concerns.

s30
00:03:13.000 --> 00:03:18.440
And when I say we here, I mean the people building developer tools cuz that is the person that I am.

s31
00:03:18.440 --> 00:03:31.680
And it gets at one of the things that are like a core principle in how we think about unlocking capabilities here and building good developer tools is that platforms should take on complexity before it reaches the user.

s32
00:03:31.680 --> 00:03:38.800
A really good experience should not expose anything of if the leaky complexity that it handles to you.

s33
00:03:38.800 --> 00:03:51.280
Um when we think about building our cloud agent platform, we try and structure it so that every primitive models this philosophy of hiding complexity from the user so they can focus on the work that matters to them.

s34
00:03:51.280 --> 00:04:01.160
Um the first place that this shows up when you're building a cloud agent platform is where does the agent run if it's not running on a developer's machine?

s35
00:04:01.240 --> 00:04:06.640
Um, it needs a place to do its work like any developer would and that place is typically a sandbox.

s36
00:04:06.640 --> 00:04:11.160
It's an isolated environment in the cloud where agents do this task.

s37
00:04:11.160 --> 00:04:22.760
When we started out building these out for our cloud agent platform, our first intuition was to provide self-hosted sandboxes so that developers had a really easy on-ramp for getting into our cloud agent platform.

s38
00:04:22.760 --> 00:04:27.480
You didn't have to think about where your compute lived, it was just there for you.

s39
00:04:27.480 --> 00:04:32.800
But the reality is that for teams doing serious work, they're probably managing their own infrastructure.

s40
00:04:32.800 --> 00:04:40.320
They probably have dev boxes that they need to interact with and so something that is hosted or managed is usually not sufficient.

s41
00:04:40.320 --> 00:04:51.960
You really need to be able to run agent workloads on infrastructure that people bring so it adapts to their like security concerns, their deployment practices, their workflows and preferences on their team.

s42
00:04:51.960 --> 00:05:03.240
And so we add support for not only manage hosting but also self-hosting to the platform and that is complexity that you abstract away from the user and how the behavior is modeled.

s43
00:05:03.240 --> 00:05:09.960
Um, the next kind of component is a little bit more personal to people and it's the harness that they want to use.

s44
00:05:09.960 --> 00:05:15.480
As we talked about earlier, people are really passionate about the tools that shape their workflows.

s45
00:05:15.480 --> 00:05:17.720
Um, one of those tools is the harness.

s46
00:05:17.720 --> 00:05:22.720
Um, who here has a preference for cloud code as a harness locally?

s47
00:05:22.720 --> 00:05:24.280
Code X?

s48
00:05:24.280 --> 00:05:26.280
Something else entirely?

s49
00:05:26.280 --> 00:05:31.600
Right, so much diversity in the room and we want to meet people where they work.

s50
00:05:31.600 --> 00:05:40.240
So you want to integrate multi-harness support that not only accommodates preferences but also gives people the ability to do the use the right tool for the job.

s51
00:05:40.240 --> 00:05:48.960
And flexibility isn't something that you just be like crammed into a platform because the real risk you run if you cram it in is that it becomes fragmented.

s52
00:05:48.960 --> 00:05:55.520
Your experience with working with Claude is different from working with Codex versus a custom harness that you might have.

s53
00:05:55.520 --> 00:06:04.560
And so, one of the key properties is making sure that the platform provides structure and guardrails around the harness so that the experience is consistent.

s54
00:06:04.560 --> 00:06:10.760
Um for us, this means that harnesses can interact with all of the platform native experiences.

s55
00:06:10.760 --> 00:06:22.919
So, just being able to store conversation state and rehydrate it, being able to interact with the artifacts and outputs that are produced by agents, whether they're PRs, issues, new files that are generated.

s56
00:06:22.919 --> 00:06:26.680
All of that should kind of be structured the same way.

s57
00:06:26.680 --> 00:06:27.760
Okay, cool.

s58
00:06:27.760 --> 00:06:37.440
So, we gave you a place for your agent to run, and we gave you a choice for what harness you use, including Warp Zone harness and any other harnesses that you want to bring.

s59
00:06:37.440 --> 00:06:41.400
Um what if one agent isn't enough to do work?

s60
00:06:41.400 --> 00:06:44.960
That's the reality of most software engineering.

s61
00:06:44.960 --> 00:06:55.000
I wish that I could just send off one prompt and solve all of the problems that exist in my software, but the reality is that real engineering work rarely fits inside one prompt.

s62
00:06:55.000 --> 00:07:00.440
In a typical workflow, you might need one agent to go research a problem and plan a solution.

s63
00:07:00.440 --> 00:07:05.800
You might need another agent to implement it, and you might need to bring in a third to validate it.

s64
00:07:05.800 --> 00:07:15.200
And you might want each of these agents to use different harnesses and different models in order to have a real adversar- adversarial and robust approach.

s65
00:07:15.200 --> 00:07:17.600
So, we have built-in support for that.

s66
00:07:17.600 --> 00:07:21.040
You can orchestrate agents across the stack.

s67
00:07:21.040 --> 00:07:26.720
Um with a lot of agent-based experiences, this orchestration happens via a prompt.

s68
00:07:26.720 --> 00:07:35.480
So, I say {slash} orchestrate, or I queue the agent via prompting that I want it to delegate work across multiple sub agents for a task that I have here.

s69
00:07:35.480 --> 00:07:47.520
And this orchestrator agent will do all of the messy complexity of interacting with sub agents, mediating messages between them, and tracking the work that's happening for me behind the scenes with a single prompt.

s70
00:07:47.520 --> 00:07:53.320
We abstract complexity away from the user by giving them this experience.

s71
00:07:53.320 --> 00:08:01.440
This sort of like prompt-based model for interacting with agents and subagents is really powerful.

s72
00:08:01.440 --> 00:08:09.120
An even more interesting one is the notion of interacting with agents and subagents via the API.

s73
00:08:09.120 --> 00:08:19.520
So, everything in our surface area is exposed via an API, and I can fire off a request to say that I want to run a subagent that is attached to a parent agent

s74
00:08:19.520 --> 00:08:21.919
um via configuration that I provide.

s75
00:08:21.919 --> 00:08:29.960
And this API is super magical because this is the key component of a platform.

s76
00:08:29.960 --> 00:08:35.159
Um it's exposing the primitives in a way that users can build on top of.

s77
00:08:35.159 --> 00:08:46.720
Um the thing about great APIs and SDKs is people can build on top of them, which means that they're not restricted to your UI or your opinion of how a particular experience should look.

s78
00:08:46.720 --> 00:08:50.200
This is where like composability becomes really powerful.

s79
00:08:50.200 --> 00:08:56.000
And so, we're trying to be intentional about exposing an API for every key component of the stack.

s80
00:08:56.000 --> 00:09:06.200
So, this is APIs for spinning up agents and subagents, for managing the environments and compute that these agents are running in, for working with the artifacts that they produce.

s81
00:09:06.200 --> 00:09:11.160
All of that is exposed in an API that you can build on top of.

s82
00:09:11.160 --> 00:09:25.271
Um and this ability to build on top of these primitives that are exposed via an API becomes really useful because anyone can build tools that overlap on top of these agentic experiences.

s83
00:09:25.271 --> 00:09:25.520
[snorts]

s84
00:09:25.520 --> 00:09:40.240
Um like interesting phenomena that's happened for us internally is we have a bunch of non-engineering teammates at Warp who have been able to use our SDK and API to build custom Slack bots to do a bunch of things.

s85
00:09:40.240 --> 00:09:47.960
Um so we have folks in our developer relations team who have actually built out tooling to help us manage all of our social mentions.

s86
00:09:47.960 --> 00:09:57.560
Um so as tweets and Reddit posts and things are coming in, we have agents that will pick them up, do some sentiment analysis on them, try and understand what the user wants, and then propose

s87
00:09:57.560 --> 00:10:05.400
a response that um folks on our social media team should use and um respond to the original tweet or Reddit post or what have you.

s88
00:10:05.400 --> 00:10:07.960
And all of this is enabled by our SDK.

s89
00:10:07.960 --> 00:10:14.360
And you see like a plethora of these types of experiences um internally at Warp.

s90
00:10:14.360 --> 00:10:24.560
Um we have people who have used them to help answer queries about how our product is working, do competitive research, all sorts of interesting things.

s91
00:10:24.800 --> 00:10:32.360
Um and these primitives became a really big deal for us specifically when we decided to go open source.

s92
00:10:32.360 --> 00:10:39.280
Um as I mentioned earlier, Warp started off as a terminal um but it grew into an agentic development environment.

s93
00:10:39.280 --> 00:10:43.520
And about 3 months ago, we decided to go open source.

s94
00:10:43.520 --> 00:10:47.560
Um this was like much anticipated, long-awaited.

s95
00:10:47.560 --> 00:10:50.240
It was a huge success for us.

s96
00:10:50.240 --> 00:10:58.440
Um the number of like GitHub stars that we had, I think catapulted from around 20,000 to over 60,000.

s97
00:10:58.440 --> 00:11:00.440
We had thousands of PRs.

s98
00:11:00.440 --> 00:11:03.360
I'll talk a little bit more about how we've been managing that.

s99
00:11:03.360 --> 00:11:11.600
And hundreds of contributors who had been long-time users of the platform and were finally getting a chance to build on top of it.

s100
00:11:11.600 --> 00:11:19.920
And when we went open source, we wanted to be really thoughtful about how we could use agents to help us manage the repository.

s101
00:11:19.920 --> 00:11:25.360
We didn't want this to be the kind of thing where agents are just writing code and firing off PRs.

s102
00:11:25.360 --> 00:11:37.560
We want them them to participate kind of meaningfully in the structure that we use to triage issues that came into the repo, provide context around them, do implementation, do reviews,

s103
00:11:37.560 --> 00:11:42.560
but still have the space for humans to participate in this loop.

s104
00:11:42.560 --> 00:11:43.200
And we did that.

s105
00:11:43.200 --> 00:11:54.520
So, if you go to the Warp open-source repo right now, you'll notice that if you file a new issue with a bug report or a feature request, an agent will kick in and start to triage the issue automatically.

s106
00:11:54.520 --> 00:12:00.160
It'll do research across the code base and context in the repo to understand what you're trying to propose.

s107
00:12:00.160 --> 00:12:13.080
It might ask you questions if it feels like your original query was a little abstract to get more information, and it will kind of do the work that's historically been very hard for open source, which is somebody has a problem or a bug that they want fixed.

s108
00:12:13.080 --> 00:12:18.960
They don't give you enough details, and it's hard to get to the clarity that you need to get to to like drive the work forward.

s109
00:12:18.960 --> 00:12:21.760
So, we can use agents to help us meaningfully in that way.

s110
00:12:21.760 --> 00:12:29.960
They can also help draft initial specifications and work for tasks, do implementation, and provide a review gate.

s111
00:12:29.960 --> 00:12:35.680
So, all PRs that get contributed to Warp go through an agent-managed review process.

s112
00:12:35.680 --> 00:12:46.560
And it goes through multiple iterations, and we don't actually ping any of the human reviewers on our team until an agent has approved our PRs, um which helps manage the workload a lot for the team.

s113
00:12:46.560 --> 00:12:53.480
So, all of those thousands of PRs, the things that humans actually have to manage are only the high-signal, high-quality ones.

s114
00:12:53.480 --> 00:13:02.280
And one of the key principles is that we improve the agent as um we get more PRs in the repo and we see more examples of code.

s115
00:13:02.280 --> 00:13:11.440
One of the things that we believe is that self-improvement loops are a really important way for you to enhance the overall SDLC life cycle that you're seeing.

s116
00:13:11.440 --> 00:13:13.920
So, we did this.

s117
00:13:14.040 --> 00:13:17.120
And we had a lightbulb moment cuz it unlocked something huge.

s118
00:13:17.120 --> 00:13:23.960
We had this like structured process that could accommodate a big influx of issues and PRs on the repo.

s119
00:13:23.960 --> 00:13:38.440
Um and the agents were there to support anyone in bringing their idea or their bug request bug feature request or bug report to us or to work and then getting it through to the actual product.

s120
00:13:38.800 --> 00:13:50.520
That key insight of agents providing like structure and context was a really big thing for us cuz it meant that it could anyone could kind of participate in translating their intent

s121
00:13:50.520 --> 00:13:52.640
into implementation.

s122
00:13:52.640 --> 00:13:59.720
And often times the people who have really interesting intents and goals are the ones who are using software in interesting ways.

s123
00:13:59.720 --> 00:14:01.640
And it's not always the person that's building it.

s124
00:14:01.640 --> 00:14:04.880
It's the person who's kind of got domain knowledge in the space.

s125
00:14:04.880 --> 00:14:10.720
And work we're lucky because we're developers building a developer tool and that's a really unique niche to fill in.

s126
00:14:10.720 --> 00:14:19.680
But most software is developers building tools for non-developers or people in situations where they don't have domain expertise.

s127
00:14:19.680 --> 00:14:33.600
If we provide these structures and guardrails though, people who are non-developers can have the necessary tools to like ship serious software because the infrastructure to support them exists.

s128
00:14:33.600 --> 00:14:35.600
Oh, this is where like things get buzzy.

s129
00:14:35.600 --> 00:14:38.800
You might have heard this term of the software factory.

s130
00:14:38.800 --> 00:14:48.560
People talk about it a lot as far as like automating how software gets built, providing these systems for doing work, all of these fun things.

s131
00:14:48.560 --> 00:14:52.000
I kind of want to push back on this term a little bit.

s132
00:14:52.000 --> 00:15:00.640
I kind of actually hate it cuz I don't think it gets the point across and it feels a little Where's the people in this?

s133
00:15:00.640 --> 00:15:06.040
So I want to tell a story before I share what I think is actually the better word.

s134
00:15:06.040 --> 00:15:07.880
So this is a mug that I have.

s135
00:15:07.880 --> 00:15:12.640
I bought this mug about two summers ago from a farmer's market.

s136
00:15:12.640 --> 00:15:22.400
And I stopped by this booth at the farmer's market and you could just tell the person who had crafted this the potter was just someone who's like really passionate about their work and what they do.

s137
00:15:22.400 --> 00:15:31.760
And so, he was telling me about all of these interesting details in the mug, the specific like curve of the handle, and the way he had structured it to accommodate different people's hands.

s138
00:15:31.760 --> 00:15:40.440
He had this like specific dimple at the top of the handle where you could rest your thumb cuz he felt like that was like a key ergonomic detail of this mug.

s139
00:15:40.440 --> 00:15:47.040
He had this glazing at the top, so if like your cup overflowed, it wouldn't dribble down the sides, the glazing would kind of catch it.

s140
00:15:47.040 --> 00:15:52.400
So, he just spent so much time thinking about the details of the mug and crafting it.

s141
00:15:52.400 --> 00:15:55.760
And then I was kind of talking to him about his workshop, like how many potters do you have?

s142
00:15:55.760 --> 00:15:57.320
How many of these mugs are you making?

s143
00:15:57.320 --> 00:15:58.560
Yada yada.

s144
00:15:58.560 --> 00:16:07.680
And he got more even more animated and he started talking about his workshop setup and how he had set up different stations for different components of the mug.

s145
00:16:07.680 --> 00:16:12.480
He had talked about how he had a specific process for sourcing clay and preparing ahead of time.

s146
00:16:12.480 --> 00:16:17.240
He talked about how he actually incorporated verification for different components of the mug.

s147
00:16:17.240 --> 00:16:20.480
If the dimple wasn't the right size, what would you do?

s148
00:16:20.480 --> 00:16:23.120
What part of the process would you restart?

s149
00:16:23.120 --> 00:16:35.760
All of this thought that he had put not into the mug itself, but how the workshop existed to support the creation of the mug, and how he was able to scale this to

s150
00:16:35.760 --> 00:16:43.839
dozens of apprentices in his shop and like hundreds of mugs handcrafted per day, which is pretty impressive.

s151
00:16:43.839 --> 00:16:48.200
And this got me thinking, I love what he did with his workshop.

s152
00:16:48.200 --> 00:17:01.880
Um he had this really great idea and he developed a serious and repeatable system that allowed anyone to take the idea of a perfect mug and turn it into the like actual existence of a perfect mug.

s153
00:17:01.880 --> 00:17:17.760
You know, some people might think like workshops are this quaint thing where it's like a workspace for a single individual, but I think the story really shows that they're actually heavy-duty systems for doing work and that they're malleable and that they react to signals in how people are interacting

s154
00:17:17.760 --> 00:17:21.120
with the thing they're building in the space they're building it.

s155
00:17:21.120 --> 00:17:29.960
Um and it also underscores the like really close interaction loop that humans have with the spaces they work in and the outputs that are produced.

s156
00:17:29.960 --> 00:17:37.000
And I'm synthesizing all of these ideas and I think this is what really we're driving at when we talk about building software factories.

s157
00:17:37.000 --> 00:17:47.640
We want to give more builders and the definition of who a builder is is expanding to non-developers serious systems for turning their ideas into code.

s158
00:17:47.640 --> 00:17:59.960
Um and we as individuals who've been building dev tooling or have been in software engineering for a long time have an understanding of what that serious system looks like and what kind of support it needs to give into individuals.

s159
00:18:00.200 --> 00:18:09.960
Um we break this down into the same techniques that my potter friend had and the same methodologies that I talked about earlier about exposing primitives.

s160
00:18:09.960 --> 00:18:20.160
Um we expose things like the ability for these agents to implement automations that react to events in the real world the same way that a human in a workspace

s161
00:18:20.160 --> 00:18:28.840
might need need to react to a real event of you know a kiln um being astray or a handle being broken.

s162
00:18:28.840 --> 00:18:30.680
We need to make these systems observable.

s163
00:18:30.680 --> 00:18:37.560
My potter friend talked about how he actually watched the way people worked in his space and refined the process over time.

s164
00:18:37.560 --> 00:18:38.560
That doesn't come for free.

s165
00:18:38.560 --> 00:18:43.040
Your system has to actually be something that you can inspect and look into.

s166
00:18:43.040 --> 00:18:45.000
And it has to improve over time.

s167
00:18:45.000 --> 00:18:48.840
The workspace is not the static component that doesn't change ever.

s168
00:18:48.840 --> 00:18:58.760
It needs to react to what's going on and modify itself to amend to like the goals of the people that are working in it and the product that it's producing.

s169
00:18:59.120 --> 00:19:01.480
And it needs to be cost-effective.

s170
00:19:01.480 --> 00:19:04.920
Uh you want to reduce the number of broken mugs that come out the other end.

s171
00:19:04.920 --> 00:19:10.440
You want to reduce the amount of buggy software that comes out of the other end of your of your factory.

s172
00:19:10.440 --> 00:19:16.800
Um and you want to do this without compromising on cost without spending too many tokens.

s173
00:19:16.960 --> 00:19:30.720
All of these principles work to achieve a shared goal and that shared goal is building systems that remove toil and drudgery from our software process so that more people have the ability to build.

s174
00:19:30.720 --> 00:19:33.720
We've seen the way like toil and drudgery have manifested.

s175
00:19:33.720 --> 00:19:41.000
It could be all of the difficulty you might have reproducing a bug, the challenges of monitoring a production system.

s176
00:19:41.000 --> 00:19:44.120
Those are things that are really hard to do.

s177
00:19:44.120 --> 00:19:55.440
And we can finally start to think about the structure of how we do them and building systems that allow us to do them repeatedly and transferably to people who are in non-technical roles.

s178
00:19:55.440 --> 00:20:06.360
If these ideas excite you about how we can build these robust and reliable systems for anyone to ship software, you could stop by the Warp booth to come talk to me and the crew.

s179
00:20:06.360 --> 00:20:08.160
We're at UG 20.

s180
00:20:08.160 --> 00:20:10.240
You can also mention me on Twitter.

s181
00:20:10.240 --> 00:20:15.520
I'm Captain Sophia on all social media, GitHub, Twitter, all of that fun stuff.

s182
00:20:15.520 --> 00:20:16.680
Or just drop me an email.

s183
00:20:16.680 --> 00:20:19.400
You can find my email on my personal site.

s184
00:20:19.400 --> 00:20:21.280
Thanks for coming to this presentation.

s185
00:20:21.280 --> 00:20:32.069
I hope you learned something interesting about some of the engineering philosophies that are driving the next set of work we do as far as agents, developers, and AI.

s186
00:20:32.069 --> 00:20:34.069
[applause]