WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=tyFPqtyOK5w

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/tyFPqtyOK5w.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.639 --> 00:00:14.320
a little bit about my background.

s2
00:00:14.320 --> 00:00:30.480
Um, I'm the co-founder and CEO of a similar and this year is pretty much the like the 30th years that I've interacted with computers and now I'm trying to build a computers that's fully autonomous so I don't have to interact with that.

s3
00:00:30.480 --> 00:00:37.520
And if I look look back to my history, the first 10 years I'm I I I basically get to know computer when I was pretty young.

s4
00:00:37.520 --> 00:00:42.719
The first 10 years I focus on comparative programming like basically writing code every day.

s5
00:00:42.719 --> 00:00:54.559
And the the second decade I spent a lot of time uh training neuron nets basically published a lot of AI research papers uh during my PhD and later on in deep mind and the third

s6
00:00:54.559 --> 00:01:10.400
um uh the third decade so I joined deep mind in 2017 that's the beginning of the third decade so this year is 20 uh 26 almost 10 years so so when when I first joined deep mind we really try to solve this problem called AGI

s7
00:01:10.400 --> 00:01:14.880
artificial general intelligence And back then everyone is still confused.

s8
00:01:14.880 --> 00:01:24.560
Even now it's everyone's still confused about what's the definition of AGI but in general it's kind of like a general system kind of like like human uh humans like like a machine

s9
00:01:24.560 --> 00:01:30.640
can do all different kinds of things in there and uh hopefully next year we can solve the problem.

s10
00:01:30.640 --> 00:01:35.840
So I I can think about what to do what kind of problem to solve uh for the for the first decade.

s11
00:01:35.840 --> 00:01:36.159
Okay.

s12
00:01:36.159 --> 00:01:39.759
So I want to u start my talk now.

s13
00:01:39.759 --> 00:01:48.320
Um basically the reason I want to introduce that is because I spent 10 years on coding 10 years on neuronets 10 10 years on solving AGI.

s14
00:01:48.320 --> 00:01:58.079
So actually that gave me this this kind of bias that we we try to inject to our current company that we have this uh this kind of principle in mind

s15
00:01:58.079 --> 00:02:04.000
the general AI system needs to be neurosyolic basically code combined with neuronets.

s16
00:02:04.000 --> 00:02:11.120
So, so which I will I will show you quickly why uh like we have this kind of intuition in there.

s17
00:02:11.120 --> 00:02:18.480
So, the goal for this workshop I know we we're kind of short on time and I do want to give uh three things back to you.

s18
00:02:18.480 --> 00:02:30.080
First is basic knowledge about computer use u I understand 50% of you are still new to this domain and second is the tools we we have been working on as part of the infrastructure.

s19
00:02:30.080 --> 00:02:42.480
There are four different type type of components that we believe would be helpful to your future development of computers uh applications and I'll also show some applications that other people has been working on or other developers

s20
00:02:42.480 --> 00:03:00.319
and third which is important uh since uh what we have a product called sigh and this product is still closed beta right now so I want to give everyone invitation code at the end of the the workshop so you have a you know opportunity to play with the product also do let me know uh Feel free to

s21
00:03:00.319 --> 00:03:03.440
follow me on uh Twitter and join Discord.

s22
00:03:03.440 --> 00:03:05.280
Let me know if you need more credits.

s23
00:03:05.280 --> 00:03:08.879
So, so I definitely like give more credits in there.

s24
00:03:08.879 --> 00:03:13.680
So, the first question is what is a computer use agents, right?

s25
00:03:13.680 --> 00:03:23.680
So, uh normally what I do with the with my talk is I can take a selfie here with everyone.

s26
00:03:23.680 --> 00:03:25.360
I'll take half.

s27
00:03:25.360 --> 00:03:25.840
Yeah.

s28
00:03:25.840 --> 00:03:29.280
So, so this is what I do every day now.

s29
00:03:29.280 --> 00:03:32.959
So actually I'm I'm the CEO, so I don't use computer all day long.

s30
00:03:32.959 --> 00:03:34.799
I have my agents on my phone.

s31
00:03:34.799 --> 00:03:47.760
So I can text my agent, send a photo there and say post the photo to LinkedIn.

s32
00:03:49.760 --> 00:03:50.000
Yeah.

s33
00:03:50.000 --> 00:03:52.480
And then the agent will will be working on it.

s34
00:03:52.480 --> 00:03:54.319
So let's see.

s35
00:03:54.319 --> 00:03:59.920
So I do want to see if the agent is responding there.

s36
00:04:05.040 --> 00:04:07.920
I think I got some.

s37
00:04:13.360 --> 00:04:13.760
Okay.

s38
00:04:13.760 --> 00:04:16.239
So, let's see.

s39
00:04:17.680 --> 00:04:19.680
I do have my message there.

s40
00:04:19.680 --> 00:04:19.919
Okay.

s41
00:04:19.919 --> 00:04:21.359
The agent is responding.

s42
00:04:21.359 --> 00:04:24.160
So, he has my photo there.

s43
00:04:24.160 --> 00:04:27.199
And you can see the agent is starting asking me questions there.

s44
00:04:27.199 --> 00:04:32.639
So uh I uh I won't interact with the agent right now but I want to show you the basic interface.

s45
00:04:32.639 --> 00:04:43.680
So computer is agents means mean means the agent that has a computer which is a full Windows desktop in this scenario and if I continue interact with the computer the mouse and keyboard

s46
00:04:43.680 --> 00:04:53.280
is know it's going to work on this desktop and open a LinkedIn later and I'll go back to the presentation.

s47
00:04:53.280 --> 00:04:56.240
I just show you an example of having one agent.

s48
00:04:56.240 --> 00:05:05.520
So I can talk to the agent agent working on applications but actually you can have multiple that's what we are actually working on.

s49
00:05:05.520 --> 00:05:22.080
So uh the these are four agents you can see they have different names right so QA agents is basically like uh doing QA on the products testing product is one big use case in there they also research customer support go to the score channel and looking at all of the information there.

s50
00:05:22.080 --> 00:05:24.479
So the the actual agents are slower.

s51
00:05:24.479 --> 00:05:27.680
So it's actually this is recording for like hours.

s52
00:05:27.680 --> 00:05:36.160
So I just compressed that into 30 seconds for you to basically grab the you know the basic understanding of of how computer us agents is working in there.

s53
00:05:36.160 --> 00:05:46.960
So the general idea this is very different from um most of the agents that we talk about right now like most of the agent they're based on APIs and you don't actually see

s54
00:05:46.960 --> 00:05:49.039
visually what the agent is doing.

s55
00:05:49.039 --> 00:06:01.759
Computer use agents is another category and we believe is a is a is another bigger category which can do a lot more things that you you don't have APIs you can still let agents to work on it

s56
00:06:01.759 --> 00:06:15.919
and then I want to do a little bit ritual because we started a company in 2023 right and that was uh right after GT 3.5 and uh GT4V so I want to show you one example that two years ago we kind of we're kind of hopeless

s57
00:06:15.919 --> 00:06:18.639
two years ago on this problem in there.

s58
00:06:18.639 --> 00:06:21.759
I actually post it on on Twitter.

s59
00:06:21.759 --> 00:06:26.639
So this is is my tweet and let me pause a little bit.

s60
00:06:26.639 --> 00:06:28.319
This is my tweet two years ago.

s61
00:06:28.319 --> 00:06:38.639
So the the problems is like can we actually use GP4V to understand the screen and then click on the iMessage icon on the screen.

s62
00:06:38.639 --> 00:06:41.759
Very very simple trivial trivial problem.

s63
00:06:41.759 --> 00:06:50.080
And I basically use the model build up a iterative refinement of the the grounding.

s64
00:06:50.080 --> 00:06:52.800
You see the the most the first time is is not correct.

s65
00:06:52.800 --> 00:06:59.199
The second time is go to the mailbox icon and the third time is correct on the iMessage.

s66
00:06:59.199 --> 00:07:04.444
So this is this is the only one successful demo I did.

s67
00:07:04.444 --> 00:07:04.960
[laughter]

s68
00:07:04.960 --> 00:07:08.400
So so I asked I asked the agent to to click on Zoom.

s69
00:07:08.400 --> 00:07:10.240
It doesn't work at all.

s70
00:07:10.240 --> 00:07:18.560
So two years ago this problem is basically hopeless but we kind of have the intuition that uh financial model is going to be better and better over time eventually

s71
00:07:18.560 --> 00:07:32.479
this problem will be solvable but it's just like it's not possible within two years back our um that actually motivates us to think about what's the general strategy for solving this computer use agents uh problem

s72
00:07:32.479 --> 00:07:43.360
and we do have a name for that it's called autonous computer because you know like if the computer can do all different kind of things it's autonomous right and then The question is how do we solve autonomous computer?

s73
00:07:44.240 --> 00:07:48.080
So when I when I talk about it there are only two words autonomous and computer.

s74
00:07:48.080 --> 00:07:57.599
So it's become so natural two directions you think about one is the autonomy the other is computer right autonomy meaning like do we have a system that can actually understand

s75
00:07:57.599 --> 00:08:12.720
the context the environment of the of the computer and do all the reasoning planning can recover from its own mistake um like doesn't require human to babysitting the the agent and the second part is computer like what kind of computer

s76
00:08:12.720 --> 00:08:24.560
uh the agent can use so this is a core question but back then two years ago we feel computer is actually easy problem because everyone already have computers right so that's why we focus a lot on the autonomy part

s77
00:08:24.560 --> 00:08:40.320
so when we talk about autonomy this goes back to the the underlying principle we initially have is called neuro symbolic neuro means this also happens to self-driving car it's basically the the same concept for all different kinds of autonomous system they all have the same concept

s78
00:08:40.320 --> 00:08:51.680
you you you have a component that's neuro here is a VM vision language model VM agent and the Second part is symbolic and here is we call it coding agent back then

s79
00:08:51.680 --> 00:09:01.200
and basically we need to solve these two technology in order for us to make autonomy works and I just show you a demo two years ago VLM doesn't work at all

s80
00:09:01.200 --> 00:09:16.480
right so that's why we actually spent a lot of time two years ago working on the coding layer to ensure okay like we can actually control the computer in a more reliable way but the second problem is we we set up a research team two years ago to think about okay how can we make this

s81
00:09:16.480 --> 00:09:23.040
craft app VM better in terms of using the uh using the computers.

s82
00:09:23.040 --> 00:09:36.320
Um before I start talk about the the trajectory our journey on the VM agents I want to uh briefly talk about the the signals like you know like self-driving cars uh like we running on the road they have signals lighter

s83
00:09:36.320 --> 00:09:54.080
radar and uh and like camera right three different kind of signal computers had the same so I often tell people like okay we are we actually working on self-driving cars in the in computer in a digital world basically changing the steering wheel to your mouse and keyboard so we still three different kinds of signals.

s84
00:09:54.080 --> 00:09:56.880
One is screenshot, the other is accessibility tree.

s85
00:09:56.880 --> 00:09:59.040
So accessibility tree is very unique.

s86
00:09:59.040 --> 00:10:02.240
It's it's kind of rare, you know, in in our in our daily life.

s87
00:10:02.240 --> 00:10:05.360
DOM tree is pretty popular because we use browsers.

s88
00:10:05.360 --> 00:10:13.279
Accessibility tree is a kind of a uh still kind of like a dome tree, but this is this tied to each of the operating system.

s89
00:10:13.279 --> 00:10:20.800
Mac has it its own accessibility tree format and Windows has it like a separate protocol for accessibility tree.

s90
00:10:20.800 --> 00:10:30.079
But this is the basic the the the layer that operating system use to render all of the windows all of the icons on on your screen in there.

s91
00:10:30.079 --> 00:10:31.920
So this this part is very important.

s92
00:10:31.920 --> 00:10:42.399
It's because um yeah the these are the the effort we've been working on is called a so the whole idea is can we build out a harness.

s93
00:10:42.399 --> 00:10:44.000
So two years ago we don't call it harness.

s94
00:10:44.000 --> 00:10:45.680
Now day everyone call it harness.

s95
00:10:45.680 --> 00:11:04.880
So two years ago we we build out we try to finish this project called agents build out a harness b uh build upon all of the foundation models not just one so you can easily switch models it's model agnostic framework that allows you to produce the best uh computers capability in there

s96
00:11:04.880 --> 00:11:20.800
and the first generation was done in late 2024 and it's called agents uh we just call it agents because that that's the beginning and This agent achieved 20% success rate on OS word 20%.

s97
00:11:20.800 --> 00:11:26.079
And that was the state of art state-of-the-art back then in 2024 which is very interesting.

s98
00:11:26.079 --> 00:11:37.600
And this paper this paper also won the best paper in I clear 2025 agenda AI workshop you know that the the whole industry was moving so fast over the past year

s99
00:11:37.600 --> 00:11:47.440
and which is crazy and this the the choice for this harness is because first we know vision language model is so bad it's not going to work.

s100
00:11:47.440 --> 00:11:51.360
That's why we chose to use accessibility tree in the first generation.

s101
00:11:51.360 --> 00:12:02.880
Basically, we look at all of the the underlying protocols of the tree structure to describe the current environment and pass it to a large language model instead of vision language model

s102
00:12:02.880 --> 00:12:13.920
and use ID ID based grounding basically linearize all of the all of the buttons put them IDs and ask LM like which button what's the ID of the button where I need to click on right now.

s103
00:12:13.920 --> 00:12:14.720
It kind of works.

s104
00:12:14.720 --> 00:12:15.680
There's 20%.

s105
00:12:15.680 --> 00:12:22.123
which is super exciting because in the beginning this benchmark is like 5% success rate in the early

s106
00:12:22.123 --> 00:12:22.560
[laughter]

s107
00:12:22.560 --> 00:12:38.720
2024 in there and then early 2025 we started to see okay vision language model is getting better so the second generation of agents we actually uh completely dis discard the accessibility tree so it's a vision only

s108
00:12:38.720 --> 00:12:47.120
approach and the interesting part is this architecture we have multiple layers like managers and workers.

s109
00:12:47.120 --> 00:12:58.079
So it's kind of like human organization and we showed it kind of works and this work specifically was um a a lot of friends in summary actually talked to me about this work.

s110
00:12:58.079 --> 00:12:59.920
They they feel pretty excited about this work.

s111
00:12:59.920 --> 00:13:08.000
The reason is this master demonstrate that if you have two models you are better than one models.

s112
00:13:08.000 --> 00:13:14.320
If you have two foundation models within the same harness it's better than one uh foundation models.

s113
00:13:14.320 --> 00:13:19.360
And the second foundation model is a seven billion open source model.

s114
00:13:19.360 --> 00:13:28.160
So yeah, so all of the startup friends, they are pretty excited about it because okay, there's a there's our journey like I can train some small models and we can be 20%

s115
00:13:28.160 --> 00:13:43.200
better than uh like cloud or openi uh computer agents there and late 2025 we have our third generation agents and the whole idea of this part is can we actually do

s116
00:13:43.200 --> 00:13:54.000
test time scaling it's called inference time scaling is when you have a agent can you have multiple agents to try to attack uh solve the same task and eventually you have better performance

s117
00:13:54.000 --> 00:13:55.440
and with demonstrator it works.

s118
00:13:55.440 --> 00:14:06.320
So this is uh the figure for the total resource budget meaning the total number of workers versus uh times the total number of steps and this for like fair comparison

s119
00:14:06.320 --> 00:14:11.519
and you can see if you have 10 agents you actually scale better in terms of success rate.

s120
00:14:11.519 --> 00:14:25.839
So then in December we break the human performance uh record in OS world benchmark and these are comparison of all the trajectory between among uh agents and anthra and openi

s121
00:14:25.839 --> 00:14:39.040
uh both big labs they achieve the same human performance uh in March this year and that also uh it's uh last December was an exciting moment like we feel okay like computer user agents is finally

s122
00:14:39.040 --> 00:14:48.800
start to be somewhat useful but you know this benchmark is kind of research benchmark it doesn't mean like in real life it's already 100% it's already better than human

s123
00:14:48.800 --> 00:15:04.079
um a few days ago OS word has uh has released their second generation called OS world 2 I'm specifically excited about this new benchmark so um in the new benchmark the best performing model is only 20% in that benchmark

s124
00:15:04.079 --> 00:15:12.000
so so we are we are entering the second phase of computers uh agents in there I want to show show you one video.

s125
00:15:12.000 --> 00:15:18.480
Uh there was a very interesting um demo that one of the developer used.

s126
00:15:18.480 --> 00:15:21.680
So this this project actually won the cow hacks last year.

s127
00:15:21.680 --> 00:15:32.959
So it's it's a it's a one of the biggest hackathon in uh hosted in in Berkeley and they they built their applications based on agents the open source one.

s128
00:15:38.560 --> 00:15:43.040
I guess uh you probably don't I'll hear the voice.

s129
00:15:43.440 --> 00:15:48.000
What if you can control your computer from anywhere in the world just by it?

s130
00:15:48.000 --> 00:15:51.040
We built a multimodal agent with full control of the computer.

s131
00:15:51.040 --> 00:15:56.639
So instead of calling your mom to debug your two out of memory errors, you can just FaceTime the Mac instead.

s132
00:15:56.639 --> 00:15:58.480
Let me show you how it works.

s133
00:15:58.480 --> 00:16:04.959
Okay, so suppose that you're training an A model at home, but you already left and you just realized that you had a CUDA out of memory error.

s134
00:16:04.959 --> 00:16:06.720
So what's the easiest way to fix this?

s135
00:16:06.720 --> 00:16:11.360
Well, instead of driving all the way back home, you can just call your computer and tell it to do itself.

s136
00:16:11.360 --> 00:16:24.079
So, we can call it and text it to FaceTime me and then this will initiate the computer use agent right here to uh start a FaceTime call so that I can tell it what I want to do.

s137
00:16:24.160 --> 00:16:28.720
Yeah, they they did a very interesting thing.

s138
00:16:29.199 --> 00:16:30.720
Let me

s139
00:16:30.720 --> 00:16:32.880
So now I can see what the thing says.

s140
00:16:32.880 --> 00:16:36.320
Can we fix my CUDA out of memory error in VS Code?

s141
00:16:36.320 --> 00:16:39.920
Okay, the task is to fix the CUDA out of memory error in VS Code.

s142
00:16:39.920 --> 00:16:44.160
I will hit command space VS Code to open the application.

s143
00:16:46.000 --> 00:16:47.440
So these are all automatic.

s144
00:16:47.440 --> 00:16:48.399
Okay, I'm in VS Code.

s145
00:16:48.399 --> 00:16:50.800
I'm looking for the CUDA out of memory error.

s146
00:16:50.800 --> 00:16:53.680
It looks like this file has a CUDA out of memory error.

s147
00:16:53.680 --> 00:16:56.560
Based on the comments, it looks like the tensor size is too large.

s148
00:16:56.560 --> 00:17:00.320
So I should try to decrease this to something reasonable.

s149
00:17:07.520 --> 00:17:09.360
Yeah, I think I think you kind of get the idea.

s150
00:17:09.360 --> 00:17:18.400
So the idea is build a application that you can control your computer using FaceTime and use FaceTime to debug in VS code.

s151
00:17:18.400 --> 00:17:21.439
That was happening in October last year.

s152
00:17:21.439 --> 00:17:26.959
That's before like even open claw was started as a project in there.

s153
00:17:26.959 --> 00:17:32.559
Then we had to think about what's the second uh you know second phase of computer use agents in there.

s154
00:17:32.559 --> 00:17:36.640
So we already surpass human performance in the benchmark like what's next?

s155
00:17:36.640 --> 00:17:53.520
Then we realized okay there's actually a very interesting concept to called pass to the power of K. And in the past, everyone was measuring pass ad meaning is is pretty popular in in VIP coding like you you have a task you add you ask the agent to

s156
00:17:53.520 --> 00:18:03.360
uh produce the code right if the code doesn't work and then you you put some prompts in there and produce another another uh code in there one of the code

s157
00:18:03.360 --> 00:18:12.880
becomes successful is successful because you don't care about what's happening in the past you only care can I get a code that works right in computer use is very different because

s158
00:18:12.880 --> 00:18:26.559
Like we we started to get customers two years ago and all of the customer talk to me about this question like can I actually put your agents hosted in a server and run this workflow every day

s159
00:18:26.559 --> 00:18:33.200
and I don't have to you know keep looking at looking at looking at what workflows I don't have to debug in there.

s160
00:18:33.200 --> 00:18:45.919
So basically everyone was telling me about this metric pass to the power k. So this means I run this agent K times every time it needs to be successful

s161
00:18:45.919 --> 00:18:52.320
and this metric is not captured by any benchmark in there and then we started to research this problem.

s162
00:18:52.320 --> 00:19:02.799
So we took the best performing agents in OS word and their performance is pretty good in three different categories and then we run the stem agents three times.

s163
00:19:02.799 --> 00:19:19.360
We measure the performance of pass to about to about three right from this figure you can see all the agents has degraded performance in there they almost dropped 10% success rate even even the best performing agent in there

s164
00:19:19.360 --> 00:19:28.880
and the second part is on perturb environment meaning my querics could be a little bit different and environment could be a little bit different each time and it get even worse

s165
00:19:28.880 --> 00:19:39.039
so this this is still a problem in the computers agents nowadays Because you can see uh Claude has Codex has uh agents, claw has agents.

s166
00:19:39.039 --> 00:19:44.720
None of the agents actually produce reliable repeatable process for you.

s167
00:19:46.160 --> 00:19:50.720
So this is a recent paper we published uh like two months ago.

s168
00:19:50.720 --> 00:20:01.919
So basically discussing this problem and there's a little simple trick that we proposed in this paper that everyone can you know can borrow the ideas in the future if you want your agents to be more reliable.

s169
00:20:01.919 --> 00:20:10.880
The trick is basically ask the agent to to ask you clarification clarification questions when you give them the instruction.

s170
00:20:10.880 --> 00:20:24.240
So, so basically this paper was saying like suppose the agent always ask you some questions about ambi ambiguous part in your in your prompt the reliability of the agent will be better

s171
00:20:24.240 --> 00:20:33.280
and there's a prompt in the in a paper that everyone can look at it like basically it's a it's it's pretty simple it's just like okay like your goal is to minimize

s172
00:20:33.280 --> 00:20:45.520
u uh have minimum uh clarification that still preserving the nature this line is very important You put this prompt to the agent and the agent can perform better.

s173
00:20:45.840 --> 00:20:57.120
After you put this clarification prompt to the agent, you can see like the overall um reliability of the agent is getting better like dash line is the original performance

s174
00:20:57.120 --> 00:21:01.520
with no clarification and the solid line is the one with clarification.

s175
00:21:01.520 --> 00:21:01.919
Right?

s176
00:21:01.919 --> 00:21:10.880
You can see cloud is getting better and agent S with GPT5 which is interesting has the biggest improvements in there.

s177
00:21:10.880 --> 00:21:21.840
So, so the the purple line is model agnostic framework and the green line is is the you know like big labs computer agents in there.

s178
00:21:22.559 --> 00:21:22.960
Okay.

s179
00:21:22.960 --> 00:21:35.840
So, so basically this is our journey for solving the VM agent which seems to be work work pretty well and still like nowadays still people using the agents framework because model agnostic you can plug in any open source foundation models.

s180
00:21:35.840 --> 00:21:46.960
I'm also like looking forward to getting like results with the latest open source big model like GLM uh 5.2 tool and see how the performance looks like in there.

s181
00:21:46.960 --> 00:21:51.039
And then the second question is okay, it doesn't solve the reliability problem, right?

s182
00:21:51.039 --> 00:22:02.320
It like these these agent doesn't seem to be have as long as you're using a neuronet, as long as you use a VM, it's in the end at the end of the day, it's a probabilistic model.

s183
00:22:02.320 --> 00:22:05.120
It produce a probability each time.

s184
00:22:05.120 --> 00:22:12.240
So it's not going to give you the same performance uh if you run it for like 247 every day.

s185
00:22:12.240 --> 00:22:17.440
Then we had to think about what's the next thing to think about is basically symbolic layer.

s186
00:22:17.440 --> 00:22:23.679
We call it coding agent and we open source this uh language we call it simulan.

s187
00:22:23.679 --> 00:22:25.280
Uh it's a language for similar.

s188
00:22:25.280 --> 00:22:27.360
So we call it simulan there.

s189
00:22:27.360 --> 00:22:36.480
So this language is kind of like if you're familiar with RPA industry it's kind of like what RPA industry used to do like they ask engineers to write a code.

s190
00:22:36.480 --> 00:22:48.400
uh but the common problem for uh RPA is there's no single protocol that's shared across the whole community that everyone can use and we try to create this uh uh in browser automation we have playright

s191
00:22:48.400 --> 00:22:58.799
which is in a standard way but in computer use there's no really no uh industry standard there so that's why we we open source this we hope it's helpful to all the developers so you can

s192
00:22:58.799 --> 00:23:07.120
u actually like build your own square pin layer for different workflows that's reliable deterministic each time.

s193
00:23:08.320 --> 00:23:08.640
Yeah.

s194
00:23:08.640 --> 00:23:20.880
So, basically the goal is to allow you to run to create uh source code for Simulan and run this code you know you know terminal and you can also use it in claw code.

s195
00:23:20.880 --> 00:23:24.640
So I I do want to give you a quick demo there.

s196
00:23:24.640 --> 00:23:36.640
So let me Okay.

s197
00:23:36.640 --> 00:23:40.400
So now I'm in a cloud code.

s198
00:23:41.840 --> 00:23:51.360
So if you install um maybe I should show you simul Yeah.

s199
00:23:51.360 --> 00:23:57.840
So these are the the landing page and if you go there there's a simple oneline installer npm installer.

s200
00:23:57.840 --> 00:24:02.480
Once we install, you should be able to use it on your own terminal.

s201
00:24:02.799 --> 00:24:04.960
And I'm a claw code user.

s202
00:24:04.960 --> 00:24:08.799
So I like to see because no one is writing code nowadays, right?

s203
00:24:08.799 --> 00:24:13.360
It's important for you to use V coding to to do this kind of automation.

s204
00:24:13.360 --> 00:24:17.679
So once you install there's a skill that you can is paired with Simulan.

s205
00:24:17.679 --> 00:24:20.240
So you can just uh you know use the skill.

s206
00:24:20.240 --> 00:24:33.360
So I can say go to AIE website and summarize the events.

s207
00:24:34.080 --> 00:24:34.400
Okay.

s208
00:24:34.400 --> 00:24:37.520
So let's see how it works.

s209
00:24:41.919 --> 00:24:44.159
So now it's checking checking version.

s210
00:24:44.159 --> 00:24:46.960
So these are the logistic stuff.

s211
00:24:46.960 --> 00:25:00.000
But it does understand it's going to use simulan to open AI engineer words fail website continue.

s212
00:25:02.240 --> 00:25:08.240
So you can notice is it's try to use a simulant JS is a JavaScript uh language.

s213
00:25:13.840 --> 00:25:22.240
And now clock is gonna try to write some code using the simul protocol.

s214
00:25:26.720 --> 00:25:29.039
Yeah, I think it's because the the clock is new.

s215
00:25:29.039 --> 00:25:34.640
So u it needs to do lots of uh logistics stuff.

s216
00:25:35.840 --> 00:25:39.039
I allow reading for everything.

s217
00:25:44.080 --> 00:25:47.360
So you can still think about this this uh this is approach.

s218
00:25:47.360 --> 00:25:49.440
It's a different approach for computers.

s219
00:25:49.440 --> 00:25:55.760
It's basically reduce computers problem into a uh coding problem in there.

s220
00:25:55.760 --> 00:25:56.960
I have what I need.

s221
00:25:56.960 --> 00:26:06.240
Let me write a script that opens the website extract the content in there and then you can see the the agent is writing the code.

s222
00:26:06.240 --> 00:26:12.799
After that it opens the website automatically on your so so similar line is running on your own computer.

s223
00:26:12.799 --> 00:26:17.919
If you have a virtual machine you can also deploy virtual machine there.

s224
00:26:20.320 --> 00:26:32.640
So there's a and then you know after after clock run the code it recognize some errors and after it gets the errors it try to revise the code.

s225
00:26:32.640 --> 00:26:43.600
So this is pretty much if you enable you know like auto approval this is pretty much already um a self-improving uh computer agents there some level of self self-improving

s226
00:26:43.600 --> 00:27:08.720
there I do want to see the it's not there and you can see claw code is decided to use accessibility snapshot instead of the screen screenshot because you know it's a summarization

s227
00:27:08.720 --> 00:27:10.720
task.

s228
00:27:12.960 --> 00:27:14.320
Okay, it's coming.

s229
00:27:14.320 --> 00:27:19.120
It gets uh it actually summarize it did.

s230
00:27:19.120 --> 00:27:23.760
Yeah, it summarized all of the events for AI engineer in there.

s231
00:27:23.760 --> 00:27:24.000
Okay.

s232
00:27:24.000 --> 00:27:34.240
So, I'll pause here and I do want to show you the full code which could be interesting to look at.

s233
00:27:35.440 --> 00:27:35.679
Yeah.

s234
00:27:35.679 --> 00:27:36.720
So, these are the full code.

s235
00:27:36.720 --> 00:27:47.840
It looks like JavaScript uh just like any any other JavaScript and it does open a window take the snapshot and write to the file.

s236
00:27:51.200 --> 00:27:51.679
Okay.

s237
00:27:51.679 --> 00:27:56.480
So, let me go back to my presentation.

s238
00:28:00.080 --> 00:28:00.399
Okay.

s239
00:28:00.399 --> 00:28:04.480
So, so I I've been talking about like two completely different approach.

s240
00:28:04.480 --> 00:28:05.600
They're orthogonal.

s241
00:28:05.600 --> 00:28:07.679
They're not the same same approach.

s242
00:28:07.679 --> 00:28:11.360
But the question is do we actually need two frameworks?

s243
00:28:11.360 --> 00:28:11.840
Right?

s244
00:28:11.840 --> 00:28:16.240
So, so they they try to compete with each other.

s245
00:28:16.799 --> 00:28:30.559
And this goes back to like a like a very uh interesting and important topic in in the whole deep learning because uh when I was in deep mind I specifically focusing on this problem called continual learning.

s246
00:28:30.559 --> 00:28:39.919
So that's why we did a lot of study on like uh AI systems in there but in the end we realized okay there's no silver bullet like yeah the the

s247
00:28:39.919 --> 00:28:51.679
when whenever technology you have uh like you have still have to solve the two problems as a trade-off like the the learning any learning system needs to learn to new things and any of the learning

s248
00:28:51.679 --> 00:28:59.440
learning system needs to you know make sure you can accomplish the the task that you you you're already successful uh repeatedly.

s249
00:28:59.440 --> 00:29:03.840
So this become it's called stability and plasticity dilemma.

s250
00:29:03.840 --> 00:29:13.760
So you you want the learning system to be stable but at the same time you want the learning system to to have this capability to adapt uh to adapt to new environments in there.

s251
00:29:13.760 --> 00:29:18.320
It's never be the case that you you are both you you're good at both.

s252
00:29:18.320 --> 00:29:19.679
So that's the conclusion.

s253
00:29:19.679 --> 00:29:20.960
Yeah, it's always trade-off.

s254
00:29:20.960 --> 00:29:25.919
Then then the problem we can think about is how do we balance the two in it?

s255
00:29:25.919 --> 00:29:42.960
And if you look at the two system I just demonstrated these two systems are coding agent and VM agents right they're basically representing the two extreme of this learning system the coding engine produce the code and it's determinance all the time but it's kind of stubborn

s256
00:29:42.960 --> 00:29:53.360
right if you run a code every day if the environment changes then the code breaks so that that's the problem and the V engine is kind of adaptive but the problem is you cannot rely on the V agents to

s257
00:29:53.360 --> 00:30:04.000
uh you know like repeat the process all the So the core question is okay can we find a middle ground between the two because the north star is basically the best adaptivity

s258
00:30:04.000 --> 00:30:17.039
and the best reliability right can we also call this this curve we call this curve parental front and can can we actually find a the sweet spot there so this becomes the current

s259
00:30:17.039 --> 00:30:29.440
direction that our company is specifically focusing on like basically try to find a unified framework that we can combine these two into one single single principle which which we call neurosyic

s260
00:30:29.440 --> 00:30:40.640
right because there's a neuro then symbolic is there a way that we can actually like fuse these two technology into one single framework that it becomes self-improving all the time and becomes robust when when

s261
00:30:40.640 --> 00:30:55.679
reliable when it's needed becomes uh adaptive when environment changes so I wouldn't say we solved the problem I wouldn't say we solved the problem but uh I would say like we we are already at the initial phase of moving towards this direction

s262
00:30:55.679 --> 00:31:05.840
because there's a clear paradigm that we can do right now because we have we have a VM agent that's better than human in a small benchmark and we have

s263
00:31:05.840 --> 00:31:15.760
a coding protocol that we can actually write a code using coding agent and the protocol can produce deterministic workflows in there.

s264
00:31:15.760 --> 00:31:19.440
This is a simple way for us to do this new experiment.

s265
00:31:19.440 --> 00:31:34.320
So basically we we let the VM agent to run in the environment and the agent keep writing the code to a file and once this workflow is successful you have a you have a you have code that's that's that's

s266
00:31:34.320 --> 00:31:45.600
successful and the next time you run the agent you can run a code first instead of run a VM agent right if it doesn't work if the environment changes like when it breaks

s267
00:31:45.600 --> 00:31:53.039
uh as as I just demonstrated there's actually a healing loop that you can do with the coding agent.

s268
00:31:53.039 --> 00:31:59.519
So this this become very interesting and uh yeah I added this slide.

s269
00:31:59.519 --> 00:32:15.200
So basically another interesting uh story is this year's cow hacks the the Sam hackathon the first place also use our uh our agent but this time they they're not using the agent to build a product they actually use the agent to test the product

s270
00:32:15.200 --> 00:32:27.360
which is another interesting category that everyone can look at because all of us are developers right we write code and we know like nowadays there are more and more PRs more and more code then who's going to do the testing then computer

s271
00:32:27.360 --> 00:32:35.840
use agency is a solution for that So basically this is the agent that can just move in the mouse click on the uh on the dashboard and doing a testing for you.

s272
00:32:35.840 --> 00:32:36.880
I'm not sure if I have time.

s273
00:32:36.880 --> 00:32:52.640
If I have time I really want to uh show a demo about like the the testing part but I can I can try let me try um let me try to show you the loop

s274
00:32:52.640 --> 00:32:54.799
that we have in the product.

s275
00:32:54.799 --> 00:32:57.120
It's a it's an initial effort.

s276
00:32:57.120 --> 00:33:02.480
It works in a lot of cases, but sometimes there are still some hiccups in there.

s277
00:33:02.480 --> 00:33:12.000
So, um I do look I do look forward to everyone like testing the product and give us feedback in there.

s278
00:33:12.240 --> 00:33:12.640
Okay.

s279
00:33:12.640 --> 00:33:21.440
So, let me I'm not sure if this one is doing Oh, this one.

s280
00:33:21.440 --> 00:33:23.440
Okay.

s281
00:33:23.600 --> 00:33:26.559
So this is the value VM.

s282
00:33:26.559 --> 00:33:43.760
This is the agent is not you see I have I have a lot of VMs here and I I do put different names to to them.

s283
00:33:43.760 --> 00:33:48.640
Um so let me try this one though.

s284
00:33:50.159 --> 00:34:03.120
So in order for you to enable the neuros symbolic mode you need to come to this dashboard click on agent there are a lot of special there are a lot of special algorithm we do uh here like humanized

s285
00:34:03.120 --> 00:34:13.520
control like the trajectory is different uh clical integration so you need to look at the sim flow code replay so this is a specific experiment we're doing right now

s286
00:34:13.520 --> 00:34:21.280
so the whole idea is uh basically the diagram I just showed uh if you enable that we can try something.

s287
00:34:21.280 --> 00:34:23.440
Let me ensure the desktop in there.

s288
00:34:23.440 --> 00:34:24.720
Okay.

s289
00:34:24.720 --> 00:34:25.919
So, we can try this.

s290
00:34:25.919 --> 00:34:27.839
Yeah, we can still try the the simple thing.

s291
00:34:27.839 --> 00:34:31.040
Just want to give you a rough idea of how this works.

s292
00:34:31.040 --> 00:34:46.560
Go to AIE web page and um find the first speaker on Tuesday.

s293
00:34:51.040 --> 00:35:04.960
There's a always has problem during live demo in let me restart agent.

s294
00:35:12.079 --> 00:35:17.680
Let me see if my VPM is awake.

s295
00:35:21.280 --> 00:35:22.480
Yeah, while we're waiting.

s296
00:35:22.480 --> 00:35:30.640
So, let me show you another demo because I al I have a backup in there if live demo needs no longer time.

s297
00:35:30.640 --> 00:35:32.960
So, we can show another one.

s298
00:35:32.960 --> 00:35:35.599
So, this one is a collaboration with Microsoft.

s299
00:35:35.599 --> 00:35:43.040
Uh so Microsoft has uh Microsoft is also built in this computer infrastructure uh because because they are the creator of Windows.

s300
00:35:43.040 --> 00:35:45.040
So these are the VMs from Microsoft.

s301
00:35:45.040 --> 00:35:54.720
It's called Microsoft Windows 365 for agents and the idea is from this you can see like on the left side the chart.

s302
00:35:54.720 --> 00:36:03.839
So the idea is okay like use uh use our agents on Microsoft virtual machines to to fill out the form on a traditional ERP.

s303
00:36:03.839 --> 00:36:16.560
So this is one of the biggest area for computer computer use agents because uh there's still so many traditional industries and they don't have modern APIs they don't have like you know they still running all the old

s304
00:36:16.560 --> 00:36:31.520
um softwares on their Windows machines right so if you have if you build out any like applications that allow you to go to certain verticals because we are not vertical company and we are horizontal infrastructure if you're interested in building like towards different verticals

s305
00:36:31.520 --> 00:36:40.000
these are good opportunities Um like for example some of our customers they uh they work with insurance website they specifically working on insurance.

s306
00:36:40.000 --> 00:36:53.440
So basically they use the agents go to insurance website and uh fetch the data and some of them are healthcare u they work with doctors they have uh like doctors has windows uh machines with ERP system very old

s307
00:36:53.440 --> 00:36:56.720
and you can use computeriz agents to automate all the data entry.

s308
00:36:56.720 --> 00:37:02.079
So there's lots of ideas that uh people has been uh exploring in this direction.

s309
00:37:02.079 --> 00:37:04.079
So, um, yeah.

s310
00:37:04.079 --> 00:37:10.880
So, those are hopefully that's that that'll give you some insights there.

s311
00:37:12.320 --> 00:37:13.119
Oh, it works.

s312
00:37:13.119 --> 00:37:13.440
Yeah.

s313
00:37:13.440 --> 00:37:23.440
So, so let me because I guess I have some skills that's already there.

s314
00:37:25.040 --> 00:37:28.240
So, there's a skill here.

s315
00:37:28.720 --> 00:37:31.520
Let me remove it.

s316
00:37:34.560 --> 00:37:48.480
go to AIE website and see who's speaking first on Tuesday.

s317
00:37:51.599 --> 00:37:52.800
I already have website.

s318
00:37:52.800 --> 00:37:55.520
So, let me make a challenge.

s319
00:37:55.520 --> 00:37:58.320
I create a new one.

s320
00:38:02.800 --> 00:38:06.320
And you can see all of the logs here.

s321
00:38:09.359 --> 00:38:16.320
And the interesting part you can see this agent is writing uh almost like line by line simulan code.

s322
00:38:16.320 --> 00:38:23.839
And after that it's going to com compose all of the code into one big file.

s323
00:38:24.000 --> 00:38:28.160
Navigate into the schedule page.

s324
00:38:28.480 --> 00:38:38.960
Okay, so we get the get the information and if you notice the agent is actually it's not a screenshot.

s325
00:38:38.960 --> 00:38:47.040
So whenever you try to find information the agent actually decide should I take a screenshot or should I take accessibility tree this time actually take access accessibility tree.

s326
00:38:47.040 --> 00:38:51.680
So you can see all the information that beneath in one shot in there.

s327
00:38:51.680 --> 00:39:00.320
And what I can do now because I want to show you that uh save to a skill.

s328
00:39:01.920 --> 00:39:13.920
So if I enable that neurosy mode and I say save to skill, this skill will have all of the code that you just run in a in a previous session.

s329
00:39:13.920 --> 00:39:19.520
And then we can look at AI speaker on day.

s330
00:39:19.520 --> 00:39:23.839
Let me find a skill there.

s331
00:39:30.640 --> 00:39:33.520
I didn't find it though.

s332
00:39:37.119 --> 00:39:37.520
Okay.

s333
00:39:37.520 --> 00:39:40.480
So this skill is saved.

s334
00:39:40.480 --> 00:39:43.520
So I'll try a different query.

s335
00:39:43.520 --> 00:39:46.240
on a different day.

s336
00:39:47.520 --> 00:39:50.240
Let's try on Wednesday.

s337
00:39:53.680 --> 00:39:56.640
And you can see there's a replay mode showing up.

s338
00:39:56.640 --> 00:40:00.800
The interface a little bit different because it it does have the code.

s339
00:40:00.800 --> 00:40:04.240
It has the code and it changed the parameter of the code.

s340
00:40:04.240 --> 00:40:12.720
And if you click on the activity log, you can see the agent is actually producing a lot more code in the first step.

s341
00:40:12.720 --> 00:40:14.079
So these are the the code.

s342
00:40:14.079 --> 00:40:16.800
It has like different parameters in there.

s343
00:40:16.800 --> 00:40:21.680
U by default is Tuesday, right?

s344
00:40:21.680 --> 00:40:22.640
The default is Tuesday.

s345
00:40:22.640 --> 00:40:30.000
If you set it Wednesday and it's going to uh change the parameter there and there's a hiccup.

s346
00:40:30.000 --> 00:40:37.040
You can see there's a hiccup for the the agent need to filter in there.

s347
00:40:37.040 --> 00:40:41.280
It does find the information and replay successful in there.

s348
00:40:41.280 --> 00:40:49.680
there's a little hiccup and then after the hiccup the agent uh convert it to the neural mode which is a VM agent.

s349
00:40:49.680 --> 00:41:00.000
So the first time it always try to produce all of the code as much as possible because if you have a pretty good hit rate it's going to have a very determin

s350
00:41:00.000 --> 00:41:04.319
successful overflow and then the agent also have verification in the middle.

s351
00:41:04.319 --> 00:41:06.720
So every time it runs the code, you have verification.

s352
00:41:06.720 --> 00:41:19.839
If there's something wrong, stop the it's kind of like it's kind of when back when I was uh when I was in school, we we write everything that teacher says in the in the memo as a playbook, right?

s353
00:41:19.839 --> 00:41:26.960
And next time when I want to do something, I look at my memo and try to do all the, you know, the steps step by step.

s354
00:41:26.960 --> 00:41:34.240
And if the environment is different, I notice, okay, I I try to do this action, but the environment doesn't allow me to do that.

s355
00:41:34.240 --> 00:41:43.280
And then I have to like think more spend more time on you know like exploring the new environment and try to figure out like what's the best strategy to do.

s356
00:41:44.480 --> 00:41:44.800
Okay.

s357
00:41:44.800 --> 00:41:46.240
So let's move on.

s358
00:41:46.240 --> 00:41:57.760
Uh I believe that's a good direction for us to for even for the whole industry to think about like what's the what's the better um agents that we can produce with

s359
00:41:57.760 --> 00:42:02.000
this kind of balance of flexibility and reliability.

s360
00:42:02.000 --> 00:42:07.599
And we do see pretty good uh success in this this type of experiments.

s361
00:42:07.599 --> 00:42:20.640
It's like okay I don't I think the I think the main point is you don't have to spend the same amount of tokens each time anymore with this kind of technology because

s362
00:42:20.640 --> 00:42:38.560
uh because the second time you run the workflow maybe 50% of the effort is done by code without any LM request in there right and uh you can you can dramatically reduce your cost so that's why we also experimented in this product we also experiment with people

s363
00:42:38.560 --> 00:42:42.800
on like $500 unlimited task plan.

s364
00:42:42.800 --> 00:42:44.800
So the price and plan may also change.

s365
00:42:44.800 --> 00:42:47.920
It's not going to be like uh usage based forever.

s366
00:42:47.920 --> 00:42:51.119
Uh because you think about like as a human, I'm I'm a human.

s367
00:42:51.119 --> 00:42:56.720
When I try to do a task a 100 times, I become so familiar with that.

s368
00:42:56.720 --> 00:43:01.680
I can pretty much do all the things with my eyes closed, right?

s369
00:43:01.680 --> 00:43:04.960
But today's agents, they don't have this capability.

s370
00:43:04.960 --> 00:43:14.720
every time you do the same task, even you do a thousand times or 10,000 times for the same task, even the same query, you still pay the same amount of money to anthrop and open,

s371
00:43:14.720 --> 00:43:21.200
which doesn't seems to be like the the reasonable uh is it's not economical as well.

s372
00:43:21.200 --> 00:43:26.960
And yeah, so uh hopefully that that gave you a pretty good sense about what direction we're we're going forward.

s373
00:43:26.960 --> 00:43:36.800
And uh I do encourage everyone to try try it out because we we actively rapidly improving that neuros symbolic agents uh in in a in the coming months and

s374
00:43:36.800 --> 00:43:40.800
uh hopefully we can achieve some like a breakthrough in that direction.

s375
00:43:40.800 --> 00:43:46.079
And the second second problem we have to think about is okay like what's the what do we do with with the computer?

s376
00:43:46.079 --> 00:43:52.319
So two years ago we we work on this autonomy uh technology on your computer.

s377
00:43:52.319 --> 00:44:03.280
So basically we have a we we used to have a product that runs the agent kind of like called co-working on your own computer but we receive a lot of complaint from users because everyone has only one computer

s378
00:44:03.280 --> 00:44:08.560
and then when the agent is working on it it's in foreground you cannot you cannot do anything.

s379
00:44:08.560 --> 00:44:08.880
Yeah.

s380
00:44:08.880 --> 00:44:16.400
So so that actually motivates us to think about okay suppose we have a human level agent uh that works.

s381
00:44:16.400 --> 00:44:18.000
Suppose we have something that works.

s382
00:44:18.000 --> 00:44:19.839
It's not it's not there yet.

s383
00:44:19.839 --> 00:44:33.520
We we're very close but when that day comes we have agent that works then we have to think about what do we do with the computer like do you still need one computer or maybe you need 10.

s384
00:44:33.520 --> 00:44:42.240
So the current observation is whenever you have something that works, users always ask us to can you can you actually host this agent on a separate machine?

s385
00:44:42.240 --> 00:44:44.000
Can you host on a server?

s386
00:44:44.000 --> 00:44:49.040
So this is always always the same conclusion from most of the users.

s387
00:44:49.040 --> 00:44:53.280
And then they will start to ask for more machines.

s388
00:44:53.280 --> 00:44:54.560
So there's a trajectory.

s389
00:44:54.560 --> 00:44:55.520
There's a trajectory.

s390
00:44:55.520 --> 00:45:00.000
First we are in the in a in a phase that we really try to make agents works.

s391
00:45:00.000 --> 00:45:03.680
Once it works, you need 10 machines, 100 machines.

s392
00:45:03.680 --> 00:45:07.920
That's why it motivates us to work on the cloud computers.

s393
00:45:07.920 --> 00:45:11.200
Um I I briefly talk I already briefly talked about it.

s394
00:45:11.200 --> 00:45:22.960
I think that the whole point is if we have the cloud managed desktop uh it becomes so scalable and everyone in the end I hope everyone can have a 100 computers so you become the CEO of your own

s395
00:45:22.960 --> 00:45:24.800
AI organization, right?

s396
00:45:24.800 --> 00:45:31.119
So you have 100 AI employees in there and everyone become 100x productive.

s397
00:45:31.119 --> 00:45:35.680
So yeah, basically this year uh we we launched a product side in March.

s398
00:45:35.680 --> 00:45:40.160
Uh it's actually a pretty new product and together with that is called side box.

s399
00:45:40.160 --> 00:45:56.800
is the virtual computer cloud in there and from the from from that demo you can already see I have personally I have 20 machines uh running in the background in there and uh this is a general diagram s is actually the interface and sausent

s400
00:45:57.119 --> 00:46:10.319
which uh do which does is auton autonomy part does the planning and decision and then doing operations on the sidebox which is a cloud computer and okay and then we have a lot of friends who developers.

s401
00:46:10.319 --> 00:46:12.880
So they keep asking us so can you actually expose the API.

s402
00:46:12.880 --> 00:46:18.960
So today we actually uh releasing the API for uh accessing the uh the cloud computer.

s403
00:46:18.960 --> 00:46:21.920
I don't think I have time to demo but it's actually pretty simple.

s404
00:46:21.920 --> 00:46:35.359
It's called SAPI S API SAPI and basically you can do everything that I did in the web portal through your terminal using the CLI tools and you can also use there's also API tool in there

s405
00:46:35.359 --> 00:46:46.400
and uh we just released this website so feel free to check it out there's a lots of APIs CLI and once you sign up you have a machine in the remote

s406
00:46:46.400 --> 00:46:55.280
uh remote setting you can use this command line and API to uh control that computer and integrate to your own product.

s407
00:46:57.842 --> 00:46:58.560
[snorts]

s408
00:46:58.560 --> 00:47:00.400
Yeah, I'll skip the live demo.

s409
00:47:00.400 --> 00:47:09.680
So the the general vision for this direction is we want we want everyone to have one single agent and this agent can access multiple platforms not just Windows.

s410
00:47:09.680 --> 00:47:12.960
So we also have uh uh support on Mac OS.

s411
00:47:12.960 --> 00:47:14.480
The NUX is coming soon.

s412
00:47:14.480 --> 00:47:18.240
Lux is very uh economically friendly.

s413
00:47:18.240 --> 00:47:26.000
is it's pretty cheap uh in terms of our pricing uh like based on our internal evaluation and the interesting part is we also have Android.

s414
00:47:26.000 --> 00:47:27.839
So Android is also coming soon.

s415
00:47:27.839 --> 00:47:38.240
So I I do hope like in the future you have a couple Android phones in the cloud you have a couple windows and the agent actually can orchestrate among all of the device to doing all different task

s416
00:47:38.240 --> 00:47:42.800
uh in there and I want to show some interesting demos for the Android.

s417
00:47:42.800 --> 00:47:46.400
So these are all internal uh like evaluation.

s418
00:47:46.400 --> 00:47:50.160
So this one is a play in the Angry Bird game.

s419
00:47:50.160 --> 00:47:53.440
And the interesting part is actually drag.

s420
00:47:53.440 --> 00:47:54.079
Let me see.

s421
00:47:54.079 --> 00:47:56.079
Yeah.

s422
00:47:59.839 --> 00:48:03.040
Yeah, you actually drag, you know, it has a drag actions.

s423
00:48:03.040 --> 00:48:07.119
He drag drag the screen and doing a task in there.

s424
00:48:07.119 --> 00:48:11.359
And another demo I feel pretty cool is authenticator.

s425
00:48:11.359 --> 00:48:17.359
So you actually go to authenticator and uh it's the same interface inside.

s426
00:48:17.359 --> 00:48:22.240
Basically we swap Windows VM to Android one.

s427
00:48:22.240 --> 00:48:25.839
So let me make it faster.

s428
00:48:26.960 --> 00:48:41.680
Yeah, it goes to this M MFA authentication challenge website and fill out the forms there and then get the token.

s429
00:48:45.440 --> 00:48:45.680
Yeah.

s430
00:48:45.680 --> 00:48:46.640
And then success.

s431
00:48:46.640 --> 00:48:51.839
So, so this is this is one of the problem that people keep asking us like okay like can we do 2FA?

s432
00:48:51.839 --> 00:49:02.960
still possible and I do I do see there's a there's some value but we have to use it in with like uh c caution there and this is a example for

s433
00:49:02.960 --> 00:49:14.319
I believe the social media on the phone u like you can actually do different kind of task there yeah it's pretty exciting I I feel the Android is a

s434
00:49:14.319 --> 00:49:23.280
interesting area that like the current industry hasn't really like explored much you know there there should be a lot of applications that we can do with Android virtual machines.

s435
00:49:23.280 --> 00:49:25.520
So basically everything is converged right now.

s436
00:49:25.520 --> 00:49:36.319
So I started with autonomous computer and I decompose everything into autonomy and computer and then autonomy becomes two parts the neural part and symbolic part.

s437
00:49:36.319 --> 00:49:39.359
Computer become local machines and cloud machines.

s438
00:49:39.359 --> 00:49:41.200
So s actually serve for both.

s439
00:49:41.200 --> 00:49:51.280
So you can actually like integrate your own machines to be part of the you know the fleet in there and you can also buy our cloud computers and then everything become

s440
00:49:51.280 --> 00:49:53.520
site which is released uh in March this year.

s441
00:49:53.520 --> 00:50:03.599
I'm pretty excited about this new product and I hope uh that will bring a lot of new opportunities uh u because this is something like I didn't imagine like

s442
00:50:03.599 --> 00:50:13.920
two years ago like we work on this technology we don't know much about like what the vertical application we can do but we actually realized okay like for example there's a YC startup

s443
00:50:13.920 --> 00:50:24.319
uh saw me in one of the event and they tell me they actually use agents for their own vertical applications so I do see there's a huge opportunity that within like in the Next year

s444
00:50:24.319 --> 00:50:39.680
could be in the next year the tech technology will be uh relatively based on the the past estimation the next year will be very interesting to to see what's what's going to happen and then what I can imagine there are a lot of uh new applications

s445
00:50:39.680 --> 00:50:58.240
like blossom in in in this this domain in within a year yeah basically to summary we have four uh offering for all the developers side work is the basically the whole interface you can explore you go to the different algorithms in the developer mode

s446
00:50:58.240 --> 00:51:11.440
give it a try let us know how it works and we actively developing new technology in there agents simulan which is open sourced and everyone can use it's pretty much free uh free to use and sidebox is the API for the for the virtual desktop

s447
00:51:11.440 --> 00:51:23.599
and we just released today so expect some bugs uh do let us know uh and I I'll give you the invitation code uh for uh today is aie EWF

s448
00:51:23.599 --> 00:51:26.559
2026 and please give it a try.

s449
00:51:26.559 --> 00:51:28.079
Let me know uh the feedback.

s450
00:51:28.079 --> 00:51:38.640
We do have a credits program like uh happy to collaborate in a diff in different way different capacity and see how to you know like uh make your development better

s451
00:51:38.640 --> 00:51:39.839
uh in the future.

s452
00:51:39.839 --> 00:51:40.160
All right.

s453
00:51:40.160 --> 00:51:41.280
Thank you so much.

s454
00:51:41.280 --> 00:51:44.000
Uh I I want to leave some time for questions.

s455
00:51:44.000 --> 00:51:49.320
So feel free to uh ask me questions in there and thank you for your patience.

s456
00:51:49.320 --> 00:51:50.319
[applause]

s457
00:51:50.319 --> 00:51:53.200
I I'll make I'll make sure the photo is posted.

s458
00:51:53.200 --> 00:51:57.760
So, let me know if you want to be mentioned in there.

s459
00:52:01.680 --> 00:52:05.359
Yeah, feel free to ask any I I'll stay here like for a few minutes.

s460
00:52:05.359 --> 00:52:09.760
I understand like uh there's another events going there.

s461
00:52:10.800 --> 00:52:11.119
Yeah.

s462
00:52:11.119 --> 00:52:15.440
Question memory anyway.

s463
00:52:15.440 --> 00:52:16.400
Spatial memory.

s464
00:52:16.400 --> 00:52:17.040
Yeah.

s465
00:52:17.040 --> 00:52:23.440
for VM how it visually looks that

s466
00:52:23.440 --> 00:52:36.240
for now more most of the memory are reduced to text in there yeah and we do have as I mentioned in agent S2 we do have a model separate model so it's actually not a simple wrapper of a large

s467
00:52:36.240 --> 00:52:55.520
vision language model so we have other 7 billion open source models sometimes we also like fine-tuning that model and that model is specifically responsible for recognizing the screen and the interesting part is we know the foundation model is progressing so so fast and so good but it's still not very good at understanding the screen.

s468
00:52:55.520 --> 00:52:58.160
So that's an interesting yeah interesting conclusion.

s469
00:52:58.160 --> 00:53:05.119
Probably it's because the internet doesn't have many like screenshot data uh available.

s470
00:53:05.119 --> 00:53:09.119
So so it's it's not available for them to train the models in there.

s471
00:53:09.119 --> 00:53:13.599
Um but I can imagine the model get getting better and better.

s472
00:53:13.599 --> 00:53:16.240
But it's just like there's always a gap.

s473
00:53:16.240 --> 00:53:21.839
If if you have a model that can specifically for your own application, you can fine-tune that model.

s474
00:53:21.839 --> 00:53:28.640
is always going to be better than like uh like a general model from big labs in there.

s475
00:53:28.640 --> 00:53:28.960
Yeah.

s476
00:53:28.960 --> 00:53:29.599
Question

s477
00:53:29.599 --> 00:53:30.400
couple questions.

s478
00:53:30.400 --> 00:53:32.400
Yeah.

s479
00:53:44.319 --> 00:53:49.359
Have you experienced the process?

s480
00:53:50.319 --> 00:53:50.640
Yeah.

s481
00:53:50.640 --> 00:54:04.640
So, so let me answer answer your first qu first question first and uh the demo I showed is LM right generating the code and this is this is the approach I call a stateless programming.

s482
00:54:04.640 --> 00:54:16.079
So all of the past 20 years we all do stateless programming and the neurosyolic approach I call it stateful programming meaning when you every time you run one line of code you look at the environment

s483
00:54:16.079 --> 00:54:18.400
look at the state and the state actually change.

s484
00:54:18.400 --> 00:54:22.319
So this is I I believe this is a new paradigm for programming in the future.

s485
00:54:22.319 --> 00:54:37.839
So program will be more stable in the future and uh we we tried LM and the problem is this problem is essentially multimodel right so accessibility can have some cap capability of you know like some

s486
00:54:37.839 --> 00:54:48.480
but it's not it's still the same there's no silver bullet so some people argue you can do computer agent just for screenshots just using screenshots some people say accessibility

s487
00:54:48.480 --> 00:54:54.480
is good enough some people say DOM is good enough but in the And after all the evaluation there's no silver bullet on everything.

s488
00:54:54.480 --> 00:54:59.520
So you always has a corner cases of one which is not corner case of the second.

s489
00:54:59.520 --> 00:55:16.480
So, so in the end I think we are actually actually taking a whimo approach but you have lighter radar and camera at the same time right and some some company they may take the Tesla approach like I I just have camera and yeah but if you look at today's self-driving car

s490
00:55:16.480 --> 00:55:27.040
uh we do see a lot of whimo uh running on the road with better reliability there it's just that you have multiple systems that help each other to to make sure they don't make mistakes

s491
00:55:27.040 --> 00:55:35.680
like especially fatal mistake you don't want You don't want agent go to your bank and add another zero when you send a check to other people.

s492
00:55:35.680 --> 00:55:36.160
Second.

s493
00:55:36.160 --> 00:55:36.559
Yeah.

s494
00:55:36.559 --> 00:55:38.880
Second question.

s495
00:55:53.040 --> 00:55:53.440
Yeah.

s496
00:55:53.440 --> 00:56:01.359
So uh the good part for accessibility tree is that information is used to render the screen.

s497
00:56:01.359 --> 00:56:10.319
So basically that information has pretty much all of the information that's needed to render a screen and if you can use that information is definitely more reliable.

s498
00:56:10.319 --> 00:56:22.559
But there are some uh there still corner cases like the developer of that application can intentionally say this part do not expose in the in a tree.

s499
00:56:22.559 --> 00:56:27.040
So, so you can still capture that in a screenshot, but you don't capture in accessibility.

s500
00:56:27.040 --> 00:56:39.839
But you can imagine some applications like uh for example, LinkedIn has a lot of B detection there pretty much the I used to tell a joke with my friends like LinkedIn is is the Turing test for AGI.

s501
00:56:39.839 --> 00:56:40.144
Yeah.

s502
00:56:40.144 --> 00:56:40.640
[laughter]

s503
00:56:40.640 --> 00:56:41.359
Because Yeah.

s504
00:56:41.359 --> 00:56:54.640
If if LinkedIn doesn't recognize your agent as agent, then you pretty much achieve AGI because you pass the Turing test in because they they're in adversarial setting and uh they have a lot of techniques to detect

s505
00:56:54.640 --> 00:56:59.200
and like this kind of accessib could be one of them in there.

s506
00:56:59.200 --> 00:57:01.200
Yeah.

s507
00:57:01.599 --> 00:57:05.440
So that's why you still need multiple you know combination of the modality.

s508
00:57:05.440 --> 00:57:12.000
So yeah, there there's one person I I'll yeah I'll let that person ask question first.

s509
00:57:12.000 --> 00:57:13.359
Yeah, thank you.

s510
00:57:13.359 --> 00:57:14.559
Yeah, thank you very much.

s511
00:57:14.559 --> 00:57:16.799
First of all,

s512
00:57:17.839 --> 00:57:20.799
so we're from Germany.

s513
00:57:21.359 --> 00:57:36.720
We have a use case in Germany where you take information and then a lot of things with them.

s514
00:57:36.720 --> 00:57:50.960
But the question is how reliable is all of that that you have shown in the real world today when you compare it with the healthare sector for example.

s515
00:57:50.960 --> 00:57:56.640
So you need very accurate information and very accurate processes

s516
00:57:56.640 --> 00:57:57.839
right

s517
00:57:57.839 --> 00:57:59.359
what is your opinion on that?

s518
00:57:59.359 --> 00:57:59.599
Yeah.

s519
00:57:59.599 --> 00:58:04.160
So, so basically the question is so we need a very accurate information.

s520
00:58:04.160 --> 00:58:06.799
So basically we need everything to be 100%.

s521
00:58:06.799 --> 00:58:07.119
Right.

s522
00:58:07.119 --> 00:58:10.079
So even when you do data entry you need to be 100%.

s523
00:58:10.079 --> 00:58:16.720
So uh that problem is the biggest problem of the current foundation models but we do have a solution for that.

s524
00:58:16.720 --> 00:58:22.160
So it's basically uh as you can see everything is a loop right.

s525
00:58:22.160 --> 00:58:24.880
So all the all the fundamental principles everything is a loop.

s526
00:58:24.880 --> 00:58:30.079
when you do something you always verify criticize yourself and it's the same for humans.

s527
00:58:30.079 --> 00:58:38.799
So when I when I propose some ideas to to my friends they they criticize so I have the loop so I can like reason with less bios in there.

s528
00:58:38.799 --> 00:58:42.400
So the the accuracy problem can also solve by this approach.

s529
00:58:42.400 --> 00:58:51.839
So we we do have some experiments that I can tell you one solution we did is uh uh whenever you have something you fill out the form you take a form

s530
00:58:51.839 --> 00:59:02.799
reverse it and ask the model does that make sense right so ask the model to verify if the mo most of the time the model recognize something wrong in there

s531
00:59:02.799 --> 00:59:13.920
because verification is is much easier than generation generation general action and if there's something wrong you take the results back and ask the agent to do the same task again.

s532
00:59:13.920 --> 00:59:16.319
It actually improved the performance a lot.

s533
00:59:16.319 --> 00:59:29.599
And um another thing you can think about is the accessibility tree like for us to write the code and for the code to be deterministically replayed you can only do in accessibility tree.

s534
00:59:29.599 --> 00:59:36.640
You cannot do in a screenshot right because the tree is text can you can do 100% matching there.

s535
00:59:36.640 --> 00:59:42.559
So that kind of a 100% text matching is also important to ensure everything is uh reliable.

s536
00:59:42.559 --> 00:59:46.640
So in the end the conclusion is you have to have a system.

s537
00:59:46.640 --> 00:59:47.920
It's it's not going to be one model.

s538
00:59:47.920 --> 00:59:52.559
You put it there and then you can hope that's 100% it's not going to happen in there.

s539
00:59:52.559 --> 00:59:57.599
And uh but there's definitely a solution for your use case in there.

s540
00:59:57.599 --> 00:59:58.960
Yeah.

s541
00:59:58.960 --> 00:59:59.599
Sorry.

s542
00:59:59.599 --> 01:00:00.240
Your turn.

s543
01:00:00.240 --> 01:00:02.240
Yeah.

s544
01:00:13.520 --> 01:00:16.799
How do you kind of keep it?

s545
01:00:30.240 --> 01:00:30.559
Yeah.

s546
01:00:30.559 --> 01:00:32.720
So, so in the product side we do have a guard rail.

s547
01:00:32.720 --> 01:00:38.480
So whenever there's a action that recognize that irreversible action the guaro will say stop there.

s548
01:00:38.480 --> 01:00:41.760
So it's another system another system monitoring this agent.

s549
01:00:41.760 --> 01:00:47.680
If the agent want to do a certain action it'll basically cut off and say ask for approval.

s550
01:00:47.680 --> 01:00:57.359
But I believe your your problem is even more challenging is more at the low level of the operating system level like for example the agent will say open this applications

s551
01:00:57.359 --> 01:01:12.640
and you do want this application to be blocked in there right so this requires another layer uh on the OS level that's why we believe have the claw managed uh Windows machine is a better direction because we have we we have the VM

s552
01:01:12.640 --> 01:01:23.200
so if you need we can also produce uh the image the VM image for you so So in a VM image you have lots of opportunity to uh change the underlying

s553
01:01:23.200 --> 01:01:25.920
you know structure of the operating system.

s554
01:01:25.920 --> 01:01:29.680
So you can actually block some of the actions have the white list in there.

s555
01:01:29.680 --> 01:01:32.160
So I believe that's the that's the final direction.

s556
01:01:32.160 --> 01:01:34.960
Uh we are still early on the VM part.

s557
01:01:34.960 --> 01:01:45.839
I we we try to explore that direction and uh like yeah if you're interested like we can also think about some partnership in the future uh on how to have better guard rails.

s558
01:01:45.839 --> 01:01:51.520
Yeah, it's definitely like a security security guard is definitely like one important aspect in there.

s559
01:01:51.520 --> 01:01:52.079
Yeah.

s560
01:01:52.079 --> 01:01:53.440
Thanks.

s561
01:01:53.440 --> 01:01:53.920
Yeah.

s562
01:01:53.920 --> 01:02:02.400
Just the protection of an irreversible something.

s563
01:02:02.400 --> 01:02:02.640
Yeah.

s564
01:02:02.640 --> 01:02:03.040
Yeah.

s565
01:02:03.040 --> 01:02:09.359
So, so that that's that goes to the same principle that the first line I put in the so basically everything is neurosymbolic.

s566
01:02:09.359 --> 01:02:17.280
If you want everything to be reliable, everything needs to be neuros symbolic, you always go to the code first because code can guarantee 100%.

s567
01:02:17.280 --> 01:02:17.680
Right?

s568
01:02:17.680 --> 01:02:24.160
When the code is confusing, you always go to the flexible agent and then come back come back to the code.

s569
01:02:24.160 --> 01:02:26.640
So that's why the code is still super important.

s570
01:02:26.640 --> 01:02:30.160
It's the same principle for the design for the guar.

s571
01:02:30.160 --> 01:02:32.319
So the guar is also like code first.

s572
01:02:32.319 --> 01:02:49.680
there lots of rules happening there and if the rule doesn't capture it goes to agent and then goes to human for verification and then comes back uh it's a it's a design principle that we try to follow in all of the system inside uh even every components uh in the in the system

s573
01:02:49.680 --> 01:02:50.640
thanks question

s574
01:02:50.640 --> 01:02:55.520
yeah so I mean you know

s575
01:02:58.640 --> 01:03:05.760
repeating cycles Does it get more?

s576
01:03:06.480 --> 01:03:07.119
Right.

s577
01:03:07.119 --> 01:03:07.359
Right.

s578
01:03:07.359 --> 01:03:07.599
Right.

s579
01:03:07.599 --> 01:03:09.760
So, so that that's the trajectory.

s580
01:03:09.760 --> 01:03:15.359
So, so basically we want the token consumption to eventually become zero.

s581
01:03:15.359 --> 01:03:15.920
Yeah.

s582
01:03:15.920 --> 01:03:22.160
event should become zero and you can see sometimes you can see the code actually start to have more branch

s583
01:03:22.160 --> 01:03:32.960
show showing up when you will see different scenarios you see more it's just like human learning right when you see uh you have more data samples then you are

s584
01:03:32.960 --> 01:03:39.839
less biased in terms of decision making yeah it's the same for this approach yeah

s585
01:03:40.400 --> 01:03:54.799
oh so question has proprietary program that they for legacy programs, right?

s586
01:04:02.319 --> 01:04:06.480
On the initial prompt, you have to be very specific because it's never seen this program.

s587
01:04:06.480 --> 01:04:23.200
Go to go to this check this box in how do you like especially a long process is that a long promp or do you actually show it how to do something?

s588
01:04:23.200 --> 01:04:35.359
Yeah, for for this version you can see there's a prompt and the agent doing the task uh well maybe in a couple months we we also think about it's it's uh it's pretty common in research community.

s589
01:04:35.359 --> 01:04:51.520
So basically if you can replicate like people talking about word models right if you can replicate this applications uh that you have better control on not doing if you have guard rails that the the agent is not going to do like bad things and then you can do exploration we call exploration in the environment.

s590
01:04:51.520 --> 01:05:02.319
So basically you let the agents do free exploration on the envir environment for different uh like maybe a 100 times and then the agent become more familiar with the environment

s591
01:05:02.319 --> 01:05:12.720
and then your basically your prompt will be less you you don't need very accurate problem like whether this button or that button you you can also al also like succeed in terms of like

s592
01:05:12.720 --> 01:05:24.559
uh finding the first basically the goal is to finding the first successful trajectory once you found it it can be successful all the And you already know that process you can already tell

s593
01:05:24.559 --> 01:05:25.680
the exact process.

s594
01:05:25.680 --> 01:05:26.880
Yeah.

s595
01:05:26.880 --> 01:05:27.039
Yeah.

s596
01:05:27.039 --> 01:05:27.200
Yeah.

s597
01:05:27.200 --> 01:05:27.440
Yeah.

s598
01:05:27.440 --> 01:05:27.760
Yeah.

s599
01:05:27.760 --> 01:05:28.000
Yeah.

s600
01:05:28.000 --> 01:05:31.200
For for many companies they call SOP.

s601
01:05:31.200 --> 01:05:33.839
So yeah they have internal they have SOP.

s602
01:05:33.839 --> 01:05:35.760
So we we actually did a interesting experiment.

s603
01:05:35.760 --> 01:05:47.520
We feed one SOP directly to the uh coding agent right and one shot in one shot everything and it's 100% successful directly.

s604
01:05:47.520 --> 01:05:56.720
So it's pos is there definitely possible but I I hold the uh the possibility that like everything now is still kind of demo phase.

s605
01:05:56.720 --> 01:06:03.920
We we want a technology that's general enough that every customer coming here and we don't have to worry about anything breaks in there.

s606
01:06:03.920 --> 01:06:08.720
So there's still lots of corner case that we have to solve going forward.

s607
01:06:08.799 --> 01:06:14.799
I remember there's one gentleman raised hand before.

s608
01:06:15.280 --> 01:06:15.839
Oh okay.

s609
01:06:15.839 --> 01:06:16.640
Yeah.

s610
01:06:16.640 --> 01:06:17.280
All right.

s611
01:06:17.280 --> 01:06:17.680
All right.

s612
01:06:17.680 --> 01:06:18.799
Thank you so much.

s613
01:06:18.799 --> 01:06:21.440
Oh, one more.

s614
01:06:32.640 --> 01:07:06.720
And actually your opinion on that actually software but you can see like we are actually the pixels certain things at the end.

s615
01:07:09.520 --> 01:07:28.400
So this computer don't even see it working on it.

s616
01:07:28.400 --> 01:07:42.480
I understood that you want to agent working but is it something you Yeah.

s617
01:07:42.480 --> 01:07:51.839
Um uh I I so uh my impression is kind of like a map and then you try to let the agent to walk through the map.

s618
01:07:51.839 --> 01:08:10.720
So he can just go if you ask us to read screenshot go there figure out just how we should go like what it doesn't f

s619
01:08:10.720 --> 01:08:22.080
yeah yeah yeah so that that's the whole idea of the coding layer so the coding layer basically encode all of the trajectory into the code right and I think I think the problem we try to solve is

s620
01:08:22.080 --> 01:08:33.120
uh suppose there's legacy software then it's not going to change then this this approach work but if we talk about like websites right for social media or like different websites

s621
01:08:33.120 --> 01:08:41.759
and they actually change and when they change then your path is not going to work in there is there a way for us to actually adapt adapt the agent

s622
01:08:41.759 --> 01:08:43.440
uh during that scenario

s623
01:08:43.440 --> 01:09:02.319
I mean my answer to that problem was if your main agent again like check it out.

s624
01:09:03.199 --> 01:09:06.799
So you have continuous improving.

s625
01:09:13.199 --> 01:09:13.440
Yeah.

s626
01:09:13.440 --> 01:09:13.679
Yeah.

s627
01:09:13.679 --> 01:09:13.839
Yeah.

s628
01:09:13.839 --> 01:09:14.640
I get your point.

s629
01:09:14.640 --> 01:09:21.759
So, so basically his solution is okay, every time there's a change, I have a swarm of agents doing the exploration in there.

s630
01:09:21.759 --> 01:09:25.359
So, I always know the the best uh

s631
01:09:25.359 --> 01:09:25.520
Yeah.

s632
01:09:25.520 --> 01:09:26.560
the path in there.

s633
01:09:26.560 --> 01:09:29.920
You have to like actually

s634
01:09:30.560 --> 01:09:30.719
Yeah.

s635
01:09:30.719 --> 01:09:30.880
Yeah.

s636
01:09:30.880 --> 01:09:31.120
Yeah.

s637
01:09:31.120 --> 01:09:34.080
That that's exactly my response to that gentleman.

s638
01:09:34.080 --> 01:09:41.600
So, basically the idea is if you have legacy system and you have is the zero cost for exploration, right?

s639
01:09:41.600 --> 01:09:45.839
Because for example, it's if it's a bank, then there's a cost for exploration.

s640
01:09:45.839 --> 01:09:48.000
I cannot explore for all of the actions.

s641
01:09:48.000 --> 01:09:51.759
But if you don't you don't have any like cost for doing exploration.

s642
01:09:51.759 --> 01:10:00.960
You just do exploration like put agents in there and like let them try different things and map all of the trajectory there and it's actually a simpler problem to solve.

s643
01:10:00.960 --> 01:10:07.679
So basically I I think 80% of the the applications is actually living in the legacy system right now.

s644
01:10:07.679 --> 01:10:13.760
And if we have you can like it's it's actually easier problem to the general uh system.

s645
01:10:13.760 --> 01:10:18.719
So that can also already capture a lot of economic value in the current industry there.

s646
01:10:18.719 --> 01:10:19.120
Yeah.

s647
01:10:19.120 --> 01:10:20.719
Thanks for your feedback.

s648
01:10:20.719 --> 01:10:21.199
Yeah.

s649
01:10:21.199 --> 01:10:22.719
Further furthermore

s650
01:10:22.719 --> 01:10:31.679
you think enough data you can signal enough data.

s651
01:10:31.679 --> 01:10:31.920
Yeah.

s652
01:10:31.920 --> 01:10:39.679
So, so that the question is suppose we have enough data can we actually train uh trend the agents to be very successful.

s653
01:10:39.679 --> 01:10:44.159
So there was a there was a lot of discussion around that.

s654
01:10:44.159 --> 01:10:52.964
So basically my conclusion is you need more exponentially more money than open eye and anthropobic to do it

s655
01:10:52.964 --> 01:10:53.600
[laughter]

s656
01:10:53.600 --> 01:11:06.560
because uh vision data is two dimensional tax is one dimensional so we are talking about like something like much bigger right so so you uh if you have if you can give me like

s657
01:11:06.560 --> 01:11:18.320
couple trillion dollars we can we can certainly consider that yeah but it's challenging maybe it's possible with the with a new paradigm of but given the current paradigm of deep learning

s658
01:11:18.320 --> 01:11:24.640
it's not something that's uh like I don't think it's achievable in the near term right now.

s659
01:11:24.640 --> 01:11:25.040
Yeah.

s660
01:11:25.040 --> 01:11:27.679
Question final question.

s661
01:11:29.520 --> 01:11:31.520
Right.

s662
01:11:41.360 --> 01:11:44.640
There's a lot of Right.

s663
01:11:45.280 --> 01:11:49.120
How does it do?

s664
01:11:57.120 --> 01:12:02.239
Oh, eventually your code will have a lot of eels logic in there.

s665
01:12:02.239 --> 01:12:07.360
So, so you can imagine the code is the code is not going to be manageable by humans for sure.

s666
01:12:07.360 --> 01:12:19.600
Uh but the code is still be able the agent is still be able to write that code with lots of eels while loop uh complex logics in there and it'll still work

s667
01:12:19.600 --> 01:12:23.440
and yeah that that's the eventual goal of this system in there.

s668
01:12:23.440 --> 01:12:24.239
Yeah.

s669
01:12:24.239 --> 01:12:27.920
But you can also chop it chop it to make a simple problems each time.

s670
01:12:27.920 --> 01:12:32.719
But then you have a boundary problem like okay how do you define a smaller problem like what's the boundary?

s671
01:12:32.719 --> 01:12:34.000
What's the input and output?

s672
01:12:34.000 --> 01:12:34.400
Right?

s673
01:12:34.400 --> 01:12:39.760
So sometimes you if you have a simple solution that's that's actually even better.

s674
01:12:40.159 --> 01:12:40.560
All right.

s675
01:12:40.560 --> 01:12:43.120
Thank thank you everyone.

s676
01:12:56.760 --> 01:12:58.760
[music]