WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=YkNulwcc5jk

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/YkNulwcc5jk.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.680 --> 00:00:13.880
Hey everybody.

s3
00:00:13.880 --> 00:00:16.320
Um my name's Alex Hancock.

s4
00:00:16.320 --> 00:00:19.360
Today I'm going to talk about a universal remote control for AI.

s5
00:00:19.360 --> 00:00:25.880
And before I start, I just want to say the previous speaker said that MCP client maintainers haven't implemented support for tasks because they're smart.

s6
00:00:25.880 --> 00:00:27.280
I'm an MCP client maintainer.

s7
00:00:27.280 --> 00:00:29.240
I can tell you it's just because I'm lazy.

s8
00:00:29.240 --> 00:00:30.960
I haven't done it.

s9
00:00:30.960 --> 00:00:33.360
Um Okay.

s10
00:00:33.360 --> 00:00:35.080
So, a little bit about me before we start.

s11
00:00:35.080 --> 00:00:40.880
I am a software engineer at Block, which is the parent company of Cash App and Square and Title.

s12
00:00:40.880 --> 00:00:42.880
We have a few different things going on now.

s13
00:00:42.880 --> 00:00:44.680
Um and I've worked there for a long time.

s14
00:00:44.680 --> 00:00:49.920
I worked on Square product stuff and Cash App stuff, but I've been doing open source AI for the last couple years.

s15
00:00:49.920 --> 00:00:57.200
Specifically, I work on this open source harness project called Goose, which started as an internal project at Block.

s16
00:00:57.200 --> 00:00:58.880
Yeah, some Goose fans out there.

s17
00:00:58.880 --> 00:01:03.320
And then uh yeah, we open sourced it and we donated it to the Linux Foundation.

s18
00:01:03.320 --> 00:01:07.760
So, now the IP is there, but we still lots of us from Block still work on it.

s19
00:01:07.760 --> 00:01:12.080
Um I'm also a maintainer of MCP, the Model Context Protocol.

s20
00:01:12.080 --> 00:01:14.800
I work on the Rust SDK for that project.

s21
00:01:14.800 --> 00:01:23.080
And more recently, I've also started some work on ACP, the Agent Client Protocol, which is what I'm going to talk about today.

s22
00:01:23.080 --> 00:01:30.720
So, I think I think we have an issue with harnesses that I want to I want to try to put to you all today, uh propose to you all today

s23
00:01:30.720 --> 00:01:34.440
as a problem and then and then recommend a a solution.

s24
00:01:34.440 --> 00:01:39.800
So, what I've been noticing recently is that we've got lots of great harnesses out there, right?

s25
00:01:39.800 --> 00:01:45.760
There are ones from the labs, there are one from ones from different companies, there's lots of open standards-based ones.

s26
00:01:45.760 --> 00:01:51.000
Um but I noticed that the interface to them is often custom or bespoke.

s27
00:01:51.000 --> 00:01:59.160
And in in the worst case, it's like you might have some harnesses where there's literally only one client application you can use to control that harness, right?

s28
00:01:59.160 --> 00:02:07.360
And I think this has a couple issues with it, but the analogy that I'll make with the web is it would be like if you had to use one browser

s29
00:02:07.360 --> 00:02:12.400
or a one given protocol to connect to a to every website, right?

s30
00:02:12.400 --> 00:02:13.200
That just wouldn't work.

s31
00:02:13.200 --> 00:02:18.280
You wouldn't have something like the open web if if that were the reality with browsers.

s32
00:02:18.280 --> 00:02:20.720
And so I think we can do better.

s33
00:02:20.720 --> 00:02:27.800
And the thing about standards by finding a standard and the thing about standards is that they create ecosystems and markets.

s34
00:02:27.800 --> 00:02:36.280
And I would argue that in the agentic AI space, we have a good standard for the agent going out and doing things, right?

s35
00:02:36.280 --> 00:02:40.680
Calling tools, taking actions in other systems, reading resources, reading data.

s36
00:02:40.680 --> 00:02:44.720
We've all benefited as a community from having MCP.

s37
00:02:44.720 --> 00:02:44.920
Right?

s38
00:02:44.920 --> 00:02:52.520
And the most powerful thing about MCP is not anything about MCP itself, but it's that everyone uses MCP.

s39
00:02:52.520 --> 00:03:02.400
And that's why we have, you know, thousands or tens of thousands of servers around the world and all the agents can connect to them and go and do things in those other systems.

s40
00:03:02.400 --> 00:03:11.920
I would say that we don't yet have a good solution or a standard for client software to tell agents what to do.

s41
00:03:11.920 --> 00:03:15.800
Giving it tasks, telling it what to work on and getting updates.

s42
00:03:15.800 --> 00:03:24.760
And so I'm going to put forward an option today that I think is a good option that that we on our team have been working on and we think is a good

s43
00:03:24.760 --> 00:03:27.600
a good solution in the open standards space.

s44
00:03:27.600 --> 00:03:34.040
And this is ACP, so agent client protocol is the name of this project and it came from the editor companies.

s45
00:03:34.040 --> 00:03:45.640
It came from like if you've used the Zed text editor or you've used any of JetBrains products, the Zed folks and the JetBrains JetBrains folks teamed up and proposed a standard for

s46
00:03:45.640 --> 00:03:47.560
um clients to be able to control harnesses.

s47
00:03:47.560 --> 00:03:49.760
And it makes sense if you put yourself in their shoes, right?

s48
00:03:49.760 --> 00:04:00.640
What they wanted to be able to do is write a single high quality client implementation in an editor, maybe in Zed or in IntelliJ or something like that, and be able to control any harness

s49
00:04:00.640 --> 00:04:11.240
by with that single client implementation, sending tasks, getting results back, seeing what files are being edited, etc. It makes a ton of sense if you put yourself in their shoes, right?

s50
00:04:11.240 --> 00:04:17.959
But we saw this on the Goose team, and we think that there is a much broader utility than just editors, right?

s51
00:04:17.959 --> 00:04:22.480
So it's a it's it's relatively neutral and it doesn't have many editor specific features.

s52
00:04:22.480 --> 00:04:28.960
And so we think that this can can go to a be spread to a wider range of client software.

s53
00:04:28.960 --> 00:04:40.600
To go into a little bit more depth about ACP's design and and what you can do with it, it lets you establish connections between clients and agent harnesses that have a given

s54
00:04:40.600 --> 00:04:46.440
a set of capabilities associated with the the connection, and then you can make sessions.

s55
00:04:46.440 --> 00:04:53.720
Within sessions, you can send user messages, the things that a user is maybe typing into the app or that the client software wants to send.

s56
00:04:53.720 --> 00:05:02.080
The agent can then respond to those with text, uh more images or audio, text, etc., or updates about what's going on.

s57
00:05:02.080 --> 00:05:08.200
So like, if a tool is called, it can send a tool call notification and explain what tool was called and what the metadata was.

s58
00:05:08.200 --> 00:05:16.440
Um and it can also send things like permission requests, so that if the client software needs to show the user, you know, "Should I do this tool call?

s59
00:05:16.440 --> 00:05:17.200
Yes or no?"

s60
00:05:17.200 --> 00:05:20.640
It can go over this uh this protocol.

s61
00:05:20.640 --> 00:05:23.200
And it's it's pretty simple in its design.

s62
00:05:23.200 --> 00:05:31.640
Uh it uses JSON RPC messages, and the thing we like about it most is that it's extensible as well, so you're not limited to just what's in the vanilla protocol.

s63
00:05:31.640 --> 00:05:33.520
You can add custom methods.

s64
00:05:33.520 --> 00:05:39.320
So the the convention is you put an underscore, and then you start to put your custom methods.

s65
00:05:39.320 --> 00:05:47.120
And the thing I like about this is that if enough harness projects or client projects adopt this, we can start to see what we're all doing that's the same,

s66
00:05:47.120 --> 00:05:47.560
right?

s67
00:05:47.560 --> 00:05:58.040
Like if the Codex team has some custom methods, the Goose team has some custom methods, uh the client team has some custom methods, whoever, we can see what emerges in the in the ecosystem

s68
00:05:58.040 --> 00:06:06.400
and what makes sense to get on a standards track and bring into the protocol itself so that this is sort of shaped by usage and shaped by the community.

s69
00:06:06.400 --> 00:06:12.080
Um I'm going to do a demo of a standard IO version of this.

s70
00:06:12.080 --> 00:06:19.520
So, I'm going to open Zed and I just have a really simple project here where I'll say, "Tell me about this project."

s71
00:06:19.520 --> 00:06:22.800
And so, this is a single HTML file.

s72
00:06:22.800 --> 00:06:30.480
So, you can see I was able to type my query into Zed and this is the agent in play here is Goose, so it's using Goose's ACP interface.

s73
00:06:30.480 --> 00:06:41.040
And it's you can see it's like sending text back, it's sending tool call information back um about what it read and what it did and then it found, you know, that it's a single HTML file and then explained it.

s74
00:06:41.040 --> 00:06:43.720
And I'll do another I'll do another one.

s75
00:06:43.720 --> 00:06:47.480
This is one from a company called Poolside AI.

s76
00:06:47.480 --> 00:06:50.600
Uh I'll say, "Tell me about this project."

s77
00:06:50.600 --> 00:06:51.680
In the same project.

s78
00:06:51.680 --> 00:07:01.240
And so, this is a terminal-based client getting exactly the same experience from the same agent, one implementation on the harness side, and you can now use any client.

s79
00:07:01.240 --> 00:07:01.560
Right?

s80
00:07:01.560 --> 00:07:02.919
And so, you can see it did the same thing.

s81
00:07:02.919 --> 00:07:09.320
It showed me some text results back, it showed a tool call, and then it showed a It's it's streaming in a summary.

s82
00:07:09.320 --> 00:07:18.240
Um So, that's a basic demo showing two clients talking to the same agent uh over standard IO locally in this case.

s83
00:07:18.240 --> 00:07:20.240
But local obviously isn't enough, right?

s84
00:07:20.240 --> 00:07:23.320
If you want this to take off, you have to be able to do remote as well.

s85
00:07:23.320 --> 00:07:25.480
Agents are going to be running in the cloud.

s86
00:07:25.480 --> 00:07:29.280
And so, when we came to this project, we saw that it did not have remote support yet.

s87
00:07:29.280 --> 00:07:32.160
So, we specified an HTTP transport.

s88
00:07:32.160 --> 00:07:35.480
There's an HTTP version and there's a websocket upgrade.

s89
00:07:35.480 --> 00:07:43.600
And so now, the messages are the same, the protocol semantics are the same, but there's a new transport that is just landing now that enables remote.

s90
00:07:43.600 --> 00:07:50.600
And the way we think about this on the Goose team, the agentic stack, is there's sort of these four important components, right?

s91
00:07:50.600 --> 00:07:57.280
You have the client, which is like the app that the user is using or a headless app running somewhere on a machine.

s92
00:07:57.280 --> 00:08:01.440
There's the harness, which is the program that implements the tool calling loop.

s93
00:08:01.440 --> 00:08:02.800
There are the tools themselves.

s94
00:08:02.800 --> 00:08:04.480
This often MCP.

s95
00:08:04.480 --> 00:08:06.240
And then there's the model, right?

s96
00:08:06.240 --> 00:08:19.919
And if you do a remote transport for the agent client protocol, and MCP has remote transport for tool calling, and the models have kind of all had remote endpoints like responses APIs for a long time.

s97
00:08:19.919 --> 00:08:23.960
Now you have the flexibility to move all of these four components around.

s98
00:08:23.960 --> 00:08:25.960
They could all be on the same machine.

s99
00:08:25.960 --> 00:08:28.840
The harness could be on a different machine than the client.

s100
00:08:28.840 --> 00:08:31.280
Uh the model could be the only thing that's remote.

s101
00:08:31.280 --> 00:08:33.719
The tools could be the only thing that's remote.

s102
00:08:33.719 --> 00:08:43.000
Uh aligning on standards and making sure that they have good transport stories is what's going to let us move all the pieces of this agentic stack around.

s103
00:08:43.000 --> 00:08:46.440
And I can show a quick demo of this as well.

s104
00:08:46.440 --> 00:08:51.160
So, this is a a client just to show how easy it is to create clients for this.

s105
00:08:51.160 --> 00:08:54.240
I just live-coded this, you know, last night.

s106
00:08:54.240 --> 00:08:56.920
And I'll say, "Write a poem."

s107
00:08:56.920 --> 00:09:01.000
So, this is again connecting to that same process on my machine.

s108
00:09:01.000 --> 00:09:03.760
Uh in this case I'm running it over the network, but it's on my machine.

s109
00:09:03.760 --> 00:09:08.760
It's connecting and sending Goose instructions for what to do uh remotely.

s110
00:09:08.760 --> 00:09:15.120
So, this could be in a container, could be up in the cloud, but the messages are the same and the library you use is the same.

s111
00:09:15.120 --> 00:09:19.800
So, you can just switch between local and remote very, very easily.

s112
00:09:19.800 --> 00:09:32.680
Um So, if you want to get plugged into this ecosystem, start experimenting with support, either making your own clients or adding stuff to harnesses, this is a this will link you to the agent client protocol site for

s113
00:09:32.680 --> 00:09:33.480
how to get started.

s114
00:09:33.480 --> 00:09:37.640
There's a number of clients and and agent servers are out there.

s115
00:09:37.640 --> 00:09:45.520
This ranges from editors, desktop applications, mobile applications, terminal based things, like there's a proliferation.

s116
00:09:45.520 --> 00:09:50.200
And I I think the use cases are are are potentially huge, right?

s117
00:09:50.200 --> 00:09:58.160
If we if we get some interoperability going here because you can have people can make personal clients that's exactly how you want it orchestrating your agents.

s118
00:09:58.160 --> 00:10:08.320
You could have sort of clients created for certain business domains or an individual company or a set of clients from a company you could customize like a white label

s119
00:10:08.320 --> 00:10:10.720
client have it work with all the harnesses.

s120
00:10:10.720 --> 00:10:16.360
And I also think if we make a new category here, we're going to see quality of the clients go up, right?

s121
00:10:16.360 --> 00:10:30.200
Because any anytime you get an ecosystem or a marketplace going and there's many options, users can vote with their feet if clients aren't meeting their needs and so people will start to compete on the quality of the user experience and and like overall I think this should drive up

s122
00:10:30.200 --> 00:10:33.320
uh the user experience of using AI.

s123
00:10:33.320 --> 00:10:34.120
That's what I've got today.

s124
00:10:34.120 --> 00:10:35.280
Thank you very much.

s125
00:10:35.280 --> 00:10:39.680
And if you want to chat with me, find me after or send me an email.

s126
00:10:39.680 --> 00:10:42.600
Um happy to get you plugged into this work.

s127
00:10:42.600 --> 00:10:44.800
Thank you.

s128
00:10:57.627 --> 00:10:59.627
[music]