WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=XWcXwnysmpY

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/XWcXwnysmpY.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:13.759 --> 00:00:19.520
Thank you for coming here to my talk to watch me talk about uh agents on the canvas.

s3
00:00:19.520 --> 00:00:24.800
Um the first thing I'm going to do though is uh before I have to record my screen.

s4
00:00:24.800 --> 00:00:30.720
Uh the first thing I'm going to do is I'm going to ask my agent to do something on the canvas.

s5
00:00:30.960 --> 00:00:40.160
And what I'm going to do is say, "Hey, uh, my colleague Spencer just emailed me a link to a notion document, uh, for a really cool demo we could build with the TL desktop app.

s6
00:00:40.160 --> 00:00:45.520
Can you like find that document and then can you build it on the desktop app?"

s7
00:00:45.760 --> 00:00:47.920
Thank you.

s8
00:00:50.160 --> 00:00:54.480
Okay, so that's going to build uh, and then we're going to come back to it later and hopefully it'll work.

s9
00:00:54.480 --> 00:00:56.960
Um, hi everyone, my name is Max Reich.

s10
00:00:56.960 --> 00:00:57.760
Thanks so much for coming.

s11
00:00:57.760 --> 00:01:02.960
Um I work on uh agents on the canvas at uh TLA.

s12
00:01:02.960 --> 00:01:04.559
I'm a product engineer there.

s13
00:01:04.559 --> 00:01:07.840
Um so first things first, am I qualified to be giving this talk?

s14
00:01:07.840 --> 00:01:08.960
I like to think so.

s15
00:01:08.960 --> 00:01:12.960
I've been doing like agents on the canvas stuff since before chat GBT came out.

s16
00:01:12.960 --> 00:01:14.240
Um I think it's really cool.

s17
00:01:14.240 --> 00:01:19.759
I think there's like so much UX stuff you can do with when you get LM, you have them working in space.

s18
00:01:19.759 --> 00:01:21.119
Uh and I think it's really interesting.

s19
00:01:21.119 --> 00:01:23.600
I've been doing it for about as long as you can have been doing it.

s20
00:01:23.600 --> 00:01:27.119
Um more recently I've been talking about this a lot.

s21
00:01:27.119 --> 00:01:28.720
Um, here's some proof.

s22
00:01:28.720 --> 00:01:32.400
Um, and yeah, so I work at this company called Teal Draw.

s23
00:01:32.400 --> 00:01:33.759
Um, can I get a quick show of hands?

s24
00:01:33.759 --> 00:01:36.479
Has anybody ever heard of or used Teal Draw before?

s25
00:01:36.479 --> 00:01:37.119
Yeah.

s26
00:01:37.119 --> 00:01:38.000
Okay.

s27
00:01:38.000 --> 00:01:38.799
Um, awesome.

s28
00:01:38.799 --> 00:01:44.560
So, yeah, the thing that you've probably used if you use Teal Draw is, um, this app right here.

s29
00:01:44.560 --> 00:01:46.079
So, this is all Teal Draw.

s30
00:01:46.079 --> 00:01:49.520
Uh, this is a free uh, infinite canvas whiteboarding app.

s31
00:01:49.520 --> 00:01:56.880
Um, you know, we have selections and arrows and resizing and, you know, all the things that you need in a in a whiteboard.

s32
00:01:56.880 --> 00:02:00.719
Um, Traw is also the company uh that makes this app.

s33
00:02:00.719 --> 00:02:01.840
It's based in London.

s34
00:02:01.840 --> 00:02:02.880
It's where I work.

s35
00:02:02.880 --> 00:02:10.959
Um, but the last thing that TLAR is, which is, I think, in my opinion, the most important, is it's the Infinite Canvas SDKs that that powers this app.

s36
00:02:10.959 --> 00:02:15.360
Um, and so what that means is that, you know, this is kind of the teal draw.

s37
00:02:15.360 --> 00:02:18.959
The SDK is the engine that powers a lot of Infinite canvas experiences.

s38
00:02:18.959 --> 00:02:22.239
Um because it turns out it's really hard to get that kind of stuff right.

s39
00:02:22.239 --> 00:02:34.319
Um and uh so if you ever want to build a Muro competitor or a slide designer or if you're like Replet um Replet has their whole new uh agent canvas stuff built on top of TLR.

s40
00:02:34.319 --> 00:02:44.720
Um, and so the reason we built TL Draw in the first place was that we were running into this issue or people were running into this issue where they had this idea for this like really great killer canvas app.

s41
00:02:44.720 --> 00:02:53.519
Uh, and they went to go build it and everybody would run into the same problem where they would run into they would have trouble making the actual canvas part of of the app

s42
00:02:53.519 --> 00:02:58.400
and they would, you know, trying to deal with resizing and selection and, you know, all the matrix math.

s43
00:02:58.400 --> 00:03:01.680
And the issue is that they wouldn't be able to build their actual app itself.

s44
00:03:01.680 --> 00:03:02.720
They would get stuck on the canvas.

s45
00:03:02.720 --> 00:03:10.159
And so we built Teal Draw to kind of be the engine um that could power that could be the canvas so that they could focus on the actual app.

s46
00:03:10.159 --> 00:03:16.480
Uh when LLM came out, we like a lot of other people saw that this is going to be this weird new type of software.

s47
00:03:16.480 --> 00:03:20.800
I don't know if anyone, you know, I'm sure a lot of you were building in 2022 and it was really exciting.

s48
00:03:20.800 --> 00:03:23.280
Um and a lot of people it was the exact same thing.

s49
00:03:23.280 --> 00:03:31.360
People had the idea had an idea for this cool app that would, you know, involve LM on the canvas having them manipulating things in space.

s50
00:03:31.360 --> 00:03:33.280
but then they would try to build it and they'd get stuck.

s51
00:03:33.280 --> 00:03:34.959
Um there were no best practices.

s52
00:03:34.959 --> 00:03:36.159
People didn't really know how to do it.

s53
00:03:36.159 --> 00:03:44.319
And so at TL Draw we uh realized that we need to make it easy for people to build uh with agents with LLMs on the canvas.

s54
00:03:44.319 --> 00:03:49.440
Um and also so Teal Draw the SDK as well as the app has multiplayer built in with like live sync.

s55
00:03:49.440 --> 00:03:50.159
It's really nice.

s56
00:03:50.159 --> 00:03:51.360
There's cursors.

s57
00:03:51.360 --> 00:03:54.799
There's you can see your collaborators cursors and selections and viewports.

s58
00:03:54.799 --> 00:04:05.280
And I think all of the things that make just the canvas in general a really great uh place for interacting with and collaborating with your colleagues also make it a really great place for interacting

s59
00:04:05.280 --> 00:04:06.799
uh and collaborating with agents.

s60
00:04:06.799 --> 00:04:11.760
Uh and I hope I'm going to be able to show you guys some of that um in the demos that come up.

s61
00:04:11.760 --> 00:04:17.280
Um so before we talk about agents on the canvas, uh really quickly I want to talk about agents not on the canvas.

s62
00:04:17.280 --> 00:04:21.359
Um I'm sure you guys have all used an app that looks like this, you know, claude code.

s63
00:04:21.359 --> 00:04:30.560
And I'm going to really oversimplify here, but basically, uh, part of the reason why these apps are so good and why they work is because they're, you know, the medium in which they're working,

s64
00:04:30.560 --> 00:04:32.960
writing code is essentially the medium in which they were trained.

s65
00:04:32.960 --> 00:04:34.479
You know, it's it's text in, text out.

s66
00:04:34.479 --> 00:04:35.759
That's how they were trained.

s67
00:04:35.759 --> 00:04:39.440
Um, and when we work with them, we give them a prompt and they write code.

s68
00:04:39.440 --> 00:04:40.880
It's text in, text out.

s69
00:04:40.880 --> 00:04:44.880
Uh, you know, again, oversimplifying, but that's essentially how they work.

s70
00:04:44.880 --> 00:04:50.479
I don't know if you guys have ever, you know, tried to get your agents to do like UI stuff and try to get them align to align something.

s71
00:04:50.479 --> 00:04:52.720
found that they could not do that whatsoever.

s72
00:04:52.720 --> 00:05:02.080
Um because it turns out agents are really really bad at working in 2D space and understanding 2D space and actually requires like a lot of engineering work to get them to

s73
00:05:02.080 --> 00:05:02.960
uh do it.

s74
00:05:02.960 --> 00:05:06.960
And that's kind of the project that we've been embarking on at Teal Draw recently.

s75
00:05:06.960 --> 00:05:13.759
Um and so the first thing we had to do, this is an older project, but the first thing we had to do is get them to teach them teach the agents

s76
00:05:13.759 --> 00:05:19.120
or at this point not agents LLM to understand the canvas and understand kind of what they're even looking at.

s77
00:05:19.120 --> 00:05:26.479
Um, so we had this project called teach um where um we tal So I'm going to I'm going to prompt this really quick.

s78
00:05:26.479 --> 00:05:30.960
I'm going to say, "Hey, make the mouse blow out the candle."

s79
00:05:31.759 --> 00:05:32.320
Yeah.

s80
00:05:32.320 --> 00:05:33.440
So that's going to take a second.

s81
00:05:33.440 --> 00:05:45.360
This is this is an older uh older project, but basically what we had to do is we had to kind of like teach the LMS how to take the like the screenshot that we give it and the JSON and all of the other information about the canvas

s82
00:05:45.360 --> 00:05:48.240
and Oh yeah, there we Okay.

s83
00:05:48.240 --> 00:05:50.000
So yeah, that's some that's some wind.

s84
00:05:50.000 --> 00:05:53.600
Uh it's is it sometimes it gives us smoke as well.

s85
00:05:53.600 --> 00:05:55.039
Yeah, and we got a little smoke as well.

s86
00:05:55.039 --> 00:06:02.160
So So we basically had to take it how to like and I want to be very clear this is not um this is not like a special mouse shape.

s87
00:06:02.160 --> 00:06:07.120
These are just like uh you know the these are just shapes uh on the canvas.

s88
00:06:07.120 --> 00:06:16.720
This is um and so the work behind this it's a singleshot prompt but we basically we tell the agent how to interpret uh both via screenshots and via the data

s89
00:06:16.720 --> 00:06:29.759
um what is actually on the canvas like what it's looking at um which is actually you know it's not a trivial problem and then also how to we teach it how to actually act on the canvas and to understand how the actions that it produces will affect the canvas.

s90
00:06:29.759 --> 00:06:37.360
So, you know, it got it, you know, it made the it made the smoke, it made the it made the wind, it got the positions right, and it understood what it was doing.

s91
00:06:37.360 --> 00:06:44.720
Um, so we we got this we kind of figured out how the um like kind of we got we taught it what the canvas is.

s92
00:06:44.720 --> 00:06:47.520
Uh, but this was like a single shot, single prompt kind of thing.

s93
00:06:47.520 --> 00:06:51.759
Um, and so the next thing we built is the uh teal draw agent starter kit.

s94
00:06:51.759 --> 00:06:56.800
Uh, which basically turns that and wraps in a harness that lets an agent work agentically on the canvas.

s95
00:06:56.800 --> 00:06:59.120
Um, the code is also MIT licensed.

s96
00:06:59.120 --> 00:07:01.520
You can find it on um that you can find on the website.

s97
00:07:01.520 --> 00:07:03.599
So, here's a little here's a little cat.

s98
00:07:03.599 --> 00:07:04.960
I'm going to make this a little bigger.

s99
00:07:04.960 --> 00:07:12.800
Um but what I'm going to say is, "Hey, so somewhere else on the canvas uh there are uh some friends for the cat.

s100
00:07:12.800 --> 00:07:15.280
Can you please uh bring one of them over to the cat?

s101
00:07:15.280 --> 00:07:18.639
Um her favorite color is red."

s102
00:07:18.960 --> 00:07:20.160
And so I'm going to zoom out.

s103
00:07:20.160 --> 00:07:21.520
I'm going to show you guys what's actually going on.

s104
00:07:21.520 --> 00:07:23.440
So you can see the the view of the agent.

s105
00:07:23.440 --> 00:07:25.680
There's some there's some potential friends over here.

s106
00:07:25.680 --> 00:07:29.039
And um you know if you read the All right.

s107
00:07:29.039 --> 00:07:37.919
So and basically what's going on is that the agent has kind of like we've given it a prompt and uh using the information it has about the canvas.

s108
00:07:37.919 --> 00:07:40.479
It's uh going to kind of like make some goals for itself.

s109
00:07:40.479 --> 00:07:42.639
You can see there's some to-dos in the corner here.

s110
00:07:42.639 --> 00:07:49.520
It's it changed its view in order to see um what was you know the other stuff that was on the canvas.

s111
00:07:49.520 --> 00:07:58.319
same way that if you ask a coding agent um you know you ask it you know where where do we define this thing in the codebase it can go and it can search it can find it.

s112
00:07:58.319 --> 00:08:09.440
So this is kind of like turning that singleshot uh prompting experience into this kind of like agentic thing that you can uh have you know it can autonomously set goals

s113
00:08:09.440 --> 00:08:11.919
and and work towards them.

s114
00:08:11.919 --> 00:08:28.479
The uh next thing we did um we did this this uh project called fairies and so we basically we had this agent experience but we realized that you know teal draw and you know the canvas in general is so collaborative it's so multiplayer and we wanted to basically we wanted people to be able to work together with

s115
00:08:28.479 --> 00:08:43.200
their agents and we also wanted the agents to be able to work together so this is this is a ferry um there's also you guys want to scan this QR code you can actually this is multiplayer you can join if you want you it requires a Gmail signup But you don't need to pay for tokens.

s116
00:08:43.200 --> 00:08:44.800
Um, this is this is what the link is.

s117
00:08:44.800 --> 00:08:47.360
So, basically, this is a this is a fairy.

s118
00:08:47.360 --> 00:08:49.120
Um, this fair's name is is Joan.

s119
00:08:49.120 --> 00:08:51.440
They don't like being they don't like being grabbed.

s120
00:08:51.440 --> 00:08:53.440
Um, you can you can throw them around.

s121
00:08:53.440 --> 00:08:55.440
You know, we added a lot of really important stuff.

s122
00:08:55.440 --> 00:08:58.880
You can um you can you can change its hat.

s123
00:08:58.880 --> 00:09:03.680
Um, you can change the the color and and this seems silly, but it's actually really important.

s124
00:09:03.680 --> 00:09:11.519
uh and I'll talk about this a little bit more later, but actually understanding uh when you get a high level view of when you see your agents working on the canvas, it's important to know which one is which.

s125
00:09:11.519 --> 00:09:15.839
And so differentiating them is actually important, which is why, of course, we added the leg slider.

s126
00:09:15.839 --> 00:09:22.720
Um but so, you know, I can say like, you know, I can I can I can say hey to it.

s127
00:09:22.720 --> 00:09:28.320
Um and I can say, you know, something like draw a cat.

s128
00:09:28.320 --> 00:09:29.360
Uh and I can have it work.

s129
00:09:29.360 --> 00:09:32.640
But the most important thing here is that fairies have friends, right?

s130
00:09:32.640 --> 00:09:34.480
and they can here we go.

s131
00:09:34.480 --> 00:09:36.000
Uh

s132
00:09:36.000 --> 00:09:37.600
and they can fairies can work together.

s133
00:09:37.600 --> 00:09:42.160
And so we kind of designed this like multi- aent collaboration system that works on the canvas.

s134
00:09:42.160 --> 00:09:49.839
Um and I'm going to actually I'm going to go to I think one of my Yeah, I think so.

s135
00:09:49.839 --> 00:09:53.760
My my colleagues agents are here working uh making this this really great scene.

s136
00:09:53.760 --> 00:10:01.600
Um I'm going to bring mine over um summon and I'm going to give them a slightly different prompt.

s137
00:10:01.600 --> 00:10:02.480
So, I'm gonna select them all.

s138
00:10:02.480 --> 00:10:05.040
And now I have a group chat of of the agents, right?

s139
00:10:05.040 --> 00:10:10.880
And I'm going to say, "Hey, I have a board meeting coming up in like 10 minutes, and I don't have any of my my figures.

s140
00:10:10.880 --> 00:10:16.240
Can you draw up like a little memo for all of my uh financial data for fiscal year 2025?

s141
00:10:16.240 --> 00:10:18.480
Thank you."

s142
00:10:18.800 --> 00:10:19.120
Okay.

s143
00:10:19.120 --> 00:10:25.680
So, what what's going to happen there basically is this kind of like creates this uh multi- aent, you know, coordination thing.

s144
00:10:25.680 --> 00:10:30.160
We have uh one of the fairies is writing uh writing out a plan.

s145
00:10:30.160 --> 00:10:30.959
You can see it.

s146
00:10:30.959 --> 00:10:35.040
Um, and again, the animations are kind of cute and funny, but it's actually really important.

s147
00:10:35.040 --> 00:10:39.360
I don't have to read a chat or go through, you know, imagine if I have 10 agents working.

s148
00:10:39.360 --> 00:10:42.399
I don't have to read a chat in order to know what's actually going on.

s149
00:10:42.399 --> 00:10:44.240
I can look at the state of the agents.

s150
00:10:44.240 --> 00:10:47.839
Uh, and I can actually, you know, I can can see what's happening.

s151
00:10:47.839 --> 00:10:52.480
So, we we have a task here that's been defined.

s152
00:10:52.480 --> 00:11:00.480
Um, it seems like, you know, the the ferry is is uh waiting for that to to finish.

s153
00:11:00.720 --> 00:11:01.120
Yeah.

s154
00:11:01.120 --> 00:11:02.480
So, that one's one's board.

s155
00:11:02.480 --> 00:11:03.040
That one's waiting.

s156
00:11:03.040 --> 00:11:04.320
So, this is the orchestrator ferry.

s157
00:11:04.320 --> 00:11:07.360
What what it's done is it's assigned the it's assigned the task.

s158
00:11:07.360 --> 00:11:10.720
Uh, and now it's waiting for the other ones to start and finish it.

s159
00:11:10.720 --> 00:11:12.079
Um, and it's going to get notified.

s160
00:11:12.079 --> 00:11:14.720
It's going to get prompted in order in order to review.

s161
00:11:14.720 --> 00:11:20.800
Um, it seems like forever reason's not working, but thank Oh, never mind.

s162
00:11:20.800 --> 00:11:23.120
So, yeah, we have one we have this one.

s163
00:11:23.120 --> 00:11:26.880
So yeah, we have one fairy who made the made the task, one fairy who's working on it.

s164
00:11:26.880 --> 00:11:31.600
And so this is this kind of um you know, multi- aent coordination system on the canvas.

s165
00:11:31.600 --> 00:11:34.399
I have, you know, you can see my colleague has his agents over here.

s166
00:11:34.399 --> 00:11:36.320
Uh they're they're working as well.

s167
00:11:36.320 --> 00:11:41.440
Um and so you can kind of collaborate with people and with agents in this uh environment.

s168
00:11:41.440 --> 00:11:44.320
And I don't know, I think that's really cool.

s169
00:11:44.320 --> 00:11:48.320
the so the the next thing.

s170
00:11:48.320 --> 00:11:54.240
So the problem with fairies is that um they're kind of trapped in the canvas and all the stuff you've seen before.

s171
00:11:54.240 --> 00:12:02.000
This requires if you want to build something like this, this requires like the you to like opt in and have your entire harness be a like canvas harness.

s172
00:12:02.000 --> 00:12:07.279
Um and the downside of that is that it makes it really hard to have any of this work with stuff like outside in the real world.

s173
00:12:07.279 --> 00:12:10.480
The fairies are the fairies are trapped in the canvas.

s174
00:12:10.480 --> 00:12:15.200
Um, and so I I I built this experiment.

s175
00:12:15.200 --> 00:12:17.600
We had a little hackathon uh internally.

s176
00:12:17.600 --> 00:12:28.320
Um, but first as a quick motivation for that, um, at Teal Draw whenever we have an whenever we're getting closer to a launch, we like abandon all of our task tracking software and we make just one massive

s177
00:12:28.320 --> 00:12:34.160
dependency graph of how like so this is what an actual this is a real thing from when we launched fairies actually.

s178
00:12:34.160 --> 00:12:41.600
Um, and so this is what it looks like when we're like really like when is hitting the fan at at tealraw when we're launching something.

s179
00:12:41.600 --> 00:12:47.040
Um, and I really like this interface because it it kind of lets you This is not like a special app.

s180
00:12:47.040 --> 00:12:48.800
This is still just tealraw.com.

s181
00:12:48.800 --> 00:12:51.600
You can, you know, move your shapes around and things like that.

s182
00:12:51.600 --> 00:12:56.480
Um, but I really like this because it both it lets you see like what depends on what.

s183
00:12:56.480 --> 00:12:57.760
It lets you know what's coming next.

s184
00:12:57.760 --> 00:12:59.200
It lets you get a highle overview.

s185
00:12:59.200 --> 00:13:04.560
You know, these are all green because we finished them, but you know, you can imagine during the project, some of them are in process.

s186
00:13:04.560 --> 00:13:07.279
And I really want I really wanted something like this.

s187
00:13:07.279 --> 00:13:11.760
Um, but something that I could actually that could actually do the work itself.

s188
00:13:11.760 --> 00:13:15.200
And so I um prototyped this thing.

s189
00:13:15.200 --> 00:13:16.480
It's called the tech tree app.

s190
00:13:16.480 --> 00:13:17.600
And basically, it's similar to this.

s191
00:13:17.600 --> 00:13:28.959
It's a dependency graph, but each of these tasks um is a a coding agent that you can kick off and you can have your agent kind of like be running and doing them autonomously.

s192
00:13:28.959 --> 00:13:35.279
Um the project itself that's working on it's this little um this is just kind of like a demo app.

s193
00:13:35.279 --> 00:13:38.959
Um but this is Can I Yeah.

s194
00:13:38.959 --> 00:13:43.920
So this is this is a little fun, you know, multimodal input thing.

s195
00:13:43.920 --> 00:13:45.279
I haven't written any of the code for this.

s196
00:13:45.279 --> 00:13:47.920
This is um this is all written by agents.

s197
00:13:47.920 --> 00:13:52.560
But I can manage all of the work is being done in this desktop app uh or in this app here.

s198
00:13:52.560 --> 00:13:58.720
And so I can do something like I can see this one has has finished uh building some gesture controls for the canvas.

s199
00:13:58.720 --> 00:14:03.440
So I can open the PR uh and unfortunately sorry Jeffrey I am just going to merge this.

s200
00:14:03.440 --> 00:14:06.959
I'm not going to uh have it be explained to me.

s201
00:14:06.959 --> 00:14:08.234
But so this is

s202
00:14:08.234 --> 00:14:08.880
[snorts]

s203
00:14:08.880 --> 00:14:09.760
and so yeah great.

s204
00:14:09.760 --> 00:14:10.079
Awesome.

s205
00:14:10.079 --> 00:14:10.959
It looks good.

s206
00:14:10.959 --> 00:14:20.079
Um and then you know eventually this is going to get marked uh as as complete and this is also multiplayer which is really cool and you can have people working together.

s207
00:14:20.079 --> 00:14:20.320
Yeah.

s208
00:14:20.320 --> 00:14:21.199
So that's finished.

s209
00:14:21.199 --> 00:14:28.320
Um and I you can also prompt from like inside the the app you can draw and have a prompt.

s210
00:14:28.320 --> 00:14:35.040
So I can basically I can just take all of this uh and I can draw a little like so this is my prompt and I can wrap it in a task

s211
00:14:35.040 --> 00:14:50.720
um and I can you know call it facial animation canvas control and then I can assign that to cloud and I can just hit run and so now that's working as well and so this is kind of like you know this is kind of something similar to conductor or openai symfony where you're using a kind of like

s212
00:14:50.720 --> 00:15:01.120
one abstracted interface above uh what the actual in order to like manage your your multi- aent uh coordination and things like that And the thing I like about this also is that because this is multiplayer,

s213
00:15:01.120 --> 00:15:06.480
one of my colleagues can come and join and add tasks and uh edit things and see the work that's been going on.

s214
00:15:06.480 --> 00:15:10.720
So, it's much more collaborative than um like your own instance of something.

s215
00:15:10.720 --> 00:15:12.800
Um here's the moment of truth.

s216
00:15:12.800 --> 00:15:18.079
Let's see if that demo that I had it build in the beginning worked.

s217
00:15:18.079 --> 00:15:23.120
Um all right, it's it hasn't built the fluid simulation yet.

s218
00:15:23.120 --> 00:15:25.760
Uh it's been working for 13 minutes.

s219
00:15:25.760 --> 00:15:29.199
Um that's actually fine.

s220
00:15:29.199 --> 00:15:31.279
So basically this is the TL Draw desktop app.

s221
00:15:31.279 --> 00:15:35.680
Um something that's really cool here is that we have uh so this is running locally.

s222
00:15:35.680 --> 00:15:36.720
It's working on files.

s223
00:15:36.720 --> 00:15:37.680
We'll see if it finishes.

s224
00:15:37.680 --> 00:15:38.720
We'll we'll let this run.

s225
00:15:38.720 --> 00:15:46.160
Um but basically what this does is this uh this basically exposes the editor instance of the TLR app that's running here.

s226
00:15:46.160 --> 00:15:55.120
And it has a server that lets any agent uh for example my cloud code uh write just plain JavaScript against the against the editor.

s227
00:15:55.120 --> 00:16:08.240
and and it it basically it you know it's it's code mode if you've ever used code mode um but you can basically turn your teal desktop app into um a like scripting environment and

s228
00:16:08.240 --> 00:16:18.639
the one of my colleagues actually is u I'm going to this is this is the kind of off the rails bit of the canvas here um or of the of the talk.

s229
00:16:18.639 --> 00:16:22.320
So here's something my colleague made uh using the same thing.

s230
00:16:22.320 --> 00:16:28.399
So this is he has the teal draw desktop app in the corner here and he's using it as his uh window manager.

s231
00:16:28.399 --> 00:16:34.560
Um and what he did the way he did this was he just told claude code to because cloud code has access to your actual computer.

s232
00:16:34.560 --> 00:16:36.160
It's not locked into the canvas.

s233
00:16:36.160 --> 00:16:44.720
Uh it basically you know it it made some rectangles and it uh probably wrote some Apple script or something to actually re uh you know move the things around.

s234
00:16:44.720 --> 00:16:52.000
And so you can kind of make all of these uh like ephemeral UIs and have them actually be doing things in the real world.

s235
00:16:52.000 --> 00:16:59.920
Um, another really cool one that he did was, um, if this loads, it's Pong on the desktop.

s236
00:16:59.920 --> 00:17:04.079
Uh, let's hit it with a little refresh there and see if it works.

s237
00:17:04.079 --> 00:17:04.480
Yeah.

s238
00:17:04.480 --> 00:17:10.160
So, this is he's got in the corner here, you know, you you have uh you have TLR running.

s239
00:17:10.160 --> 00:17:14.959
This is the desktop app and it's using the Windows in order to uh play Pong.

s240
00:17:14.959 --> 00:17:24.240
And so again, like kind of crazy, but there's uh you know, maybe it seems a little silly, but let's see if this this worked.

s241
00:17:24.240 --> 00:17:25.039
Oh, it's still working.

s242
00:17:25.039 --> 00:17:26.559
Man, it was usually much faster.

s243
00:17:26.559 --> 00:17:36.000
Um, but I I think this stuff is so cool because this lets you kind of, you know, do all of the weird kind of like spatial interfaces that you can do uh on the canvas.

s244
00:17:36.000 --> 00:17:38.320
You get all of like the primitives of the canvas.

s245
00:17:38.320 --> 00:17:43.280
Um, but you can like you can have your agents working kind of like in the real world.

s246
00:17:43.280 --> 00:17:45.039
uh it has access to real data.

s247
00:17:45.039 --> 00:17:58.799
If I scroll up, I'll show that um if uh you know this is my cloud code and it found it it got the Gmail, it got the notion doc, it found the spec and it's going to to implement it.

s248
00:17:58.799 --> 00:18:11.840
Um but yeah, so to sum up, I think that agents working on the canvas is so cool and I think that there's like so much we can do if we use like the agent the canvas as a a place to work with agents.

s249
00:18:11.840 --> 00:18:22.320
And I think the place part of it is really important because you know when we do you know with remote work collaboration we do a lot of stuff online with each other and we collaborate with people on the canvas

s250
00:18:22.320 --> 00:18:31.520
and I think that the yeah the canvas can be a place where we uh collaborate with agents and I'm I'm I'm I'm vamping because I'm trying to see if this is finished but I don't think it's going to finish.

s251
00:18:31.520 --> 00:18:34.720
Um, but thank you so much

s252
00:18:49.232 --> 00:18:51.232
[music]