WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=9dYcwOkpCE8

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/9dYcwOkpCE8.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:12.639 --> 00:00:15.280
Hey everyone, thanks for coming.

s2
00:00:15.280 --> 00:00:16.080
I'm Andrew.

s3
00:00:16.080 --> 00:00:24.400
I'm the chief of software at Verscell and I'm here to talk to you about how we solved agent building at Verscell.

s4
00:00:24.400 --> 00:00:25.680
I'm the chief of software.

s5
00:00:25.680 --> 00:00:34.480
So I work on a mix of internal engineering, external experimentation, and generally being at the frontier and building new libraries, frameworks, and technologies.

s6
00:00:34.480 --> 00:00:41.760
For those of you that don't know Verscell, Verscell builds a gentic infrastructure so people can build what's next.

s7
00:00:41.760 --> 00:00:49.200
We get started in the web world helping people ship websites and web apps without having to worry about the infrastructure that doesn't make their app any better.

s8
00:00:49.200 --> 00:00:52.719
It can scale to a million and scale down to zero effortlessly.

s9
00:00:52.719 --> 00:00:55.600
But we're seeing a change in what people want to build.

s10
00:00:55.600 --> 00:00:59.680
You know, people started by building pages, but now we see them want to build agents.

s11
00:00:59.680 --> 00:01:07.520
And we've been embarking on a similar journey to make it easy for people to build agents and agentic applications easier.

s12
00:01:07.520 --> 00:01:09.439
We built this thing called the AIDK.

s13
00:01:09.439 --> 00:01:18.560
So instead of needing to switch out 300 400 lines of provider specific code, you're just going to switch out one line of code and we have the same model interface underlying

s14
00:01:18.560 --> 00:01:21.119
for all these different providers.

s15
00:01:21.119 --> 00:01:32.880
We built a lot of other tools to make it easier to have model fallbacks, secure code execution, better pricing when it's inactive and waiting for responses, as well as for durability and resumability.

s16
00:01:32.880 --> 00:01:45.759
And I'm here to talk to you about how I went on this crazy experiment roughly a year ago that led to a aentic explosion at Verscell and led to a really cool thing that we built recently

s17
00:01:46.159 --> 00:01:48.479
about uh this is 1980.

s18
00:01:48.479 --> 00:01:55.680
Uh, Bill Gates before my time had this quote saying he imagined there would be a computer on every desk and in every home.

s19
00:01:55.680 --> 00:02:02.159
You know, that was probably pretty contrarian then and today it seems like very normal to have that happen.

s20
00:02:02.159 --> 00:02:10.879
And me and the CTO had this thought, you know, instead of a computer on every desk, could we potentially have an agent on every desk?

s21
00:02:10.879 --> 00:02:22.400
You know, today we only really use agents for coding and technical workloads, but we're starting to see expansion into things like design, product management, and other verticals.

s22
00:02:22.400 --> 00:02:30.640
And this was maybe about a year ago, so I would say I'm pretty early to this, but that was when it was like sonnet 4 and things weren't as sophisticated as they were today.

s23
00:02:30.640 --> 00:02:34.720
And I tried to actually explore this out, see what we could do about it.

s24
00:02:34.720 --> 00:02:44.400
I went around to various job functions at Verscell, marketing, sales, finance, legal, and I asked them, what do you hate most about your job?

s25
00:02:44.400 --> 00:02:50.080
And the most compelling use case I heard was that the data team, they were growing.

s26
00:02:50.080 --> 00:02:53.599
They were a very lean team, but Versel was growing faster.

s27
00:02:53.599 --> 00:02:59.200
You know, they had so much more data from customers, analytics, metrics, sales.

s28
00:02:59.200 --> 00:03:05.200
They just had to keep on aggregating and keep on making available for themselves to use.

s29
00:03:05.200 --> 00:03:14.080
And at this time, if you think about what the data science people ever have to do, whenever someone from marketing or sales has a question about a customer or product,

s30
00:03:14.080 --> 00:03:22.319
the data science team has to drop everything they're doing, write the query, process it, do an analysis, and come back with some recommendation on what to do.

s31
00:03:22.319 --> 00:03:25.040
And this was really killer to productivity.

s32
00:03:25.040 --> 00:03:29.280
You know, the data team did not want to drop everything and just write queries all day.

s33
00:03:29.280 --> 00:03:36.000
And so I worked with our VP of data to try to build a better way for them to operate this way.

s34
00:03:36.000 --> 00:03:43.840
And so if you think about the very first thing you would ever do if you want to try to use AI to solve a problem, you may just build like a huge mega prompt.

s35
00:03:43.840 --> 00:03:49.360
You know, you just have a question, you pass into an LM, you have it respond, and that's it.

s36
00:03:49.360 --> 00:03:51.599
You know, this was how the first version really looked.

s37
00:03:51.599 --> 00:03:55.840
Honestly, I asked them for a dump of of the snowflake schema.

s38
00:03:55.840 --> 00:04:03.280
I pasted it into a system prompt with a question and then when it generated SQL I actually copy and pasted that in and just ran it myself.

s39
00:04:03.280 --> 00:04:10.720
You know I just want to see are the models good enough today in order to write valid SQL given some decent structure.

s40
00:04:10.720 --> 00:04:22.720
And I would say this gave us a little bit of confidence that you know models today aren't that good but maybe we can harness engineer or make the context around it a little better and give us some more guardrails to operate a little better.

s41
00:04:22.720 --> 00:04:33.600
And so if you actually think about what a data scientist actually needs to do when they get a question, you know, they have to process the question, they may have to explore the semantic layer and actually figure out what the join patterns are.

s42
00:04:33.600 --> 00:04:36.080
They will actually go and execute the SQL.

s43
00:04:36.080 --> 00:04:40.639
They may go back and do that again if the SQL did not execute or was too expensive.

s44
00:04:40.639 --> 00:04:48.080
And they'll eventually report on it, including visualize the data, maybe write some paragraphs, maybe do a retro, maybe do some other stuff.

s45
00:04:48.080 --> 00:04:55.120
And so if you think about those different phases, me and the VP of data tried to sit down and map those out into specific agent workloads.

s46
00:04:55.120 --> 00:04:59.520
And so the second version of this data science agent uh called D0.

s47
00:04:59.520 --> 00:05:03.199
I'm going to reference D0 from now on is you ask a question.

s48
00:05:03.199 --> 00:05:11.840
We have a query agent that passes on a query to the planning agent that will then have an execution agent etc. And if you chain all of these together,

s49
00:05:11.840 --> 00:05:20.560
you actually get something that looks like this where each agent has a very dedicated system prompt focused to what that does with tools scoped to exactly that function.

s50
00:05:20.560 --> 00:05:29.840
So example here, you can see that for the first one, the planning agent has a read entity YAML and a and a search schemas tool.

s51
00:05:29.840 --> 00:05:38.800
And so it will only use those capabilities until it has an answer to pass on to the planning agent and then to the SQL agent and then to reporting.

s52
00:05:38.800 --> 00:05:40.320
And this was getting better.

s53
00:05:40.320 --> 00:05:46.240
You know, we were able to get away from having to copy and paste a SQL and have to come back and report on it.

s54
00:05:46.240 --> 00:05:51.600
It was now actually doing like the end to end loop from question to answer.

s55
00:05:51.600 --> 00:05:55.759
But we started hitting some walls with this architecture.

s56
00:05:55.759 --> 00:06:05.280
And around this time we came to the conclusion that you know what you actually need is you need one agent with all the mega context within it and for it to sort of manage its own memory.

s57
00:06:05.280 --> 00:06:15.600
You know this was around the time when we realized that you want to actually have the agent be able to look back on what it's done sort of reflect and figure out the steps that got to get here.

s58
00:06:15.600 --> 00:06:24.240
And with the previous model you may have noticed that the only thing that the next agent gets is a summary and a small snippet of the previous thing that was done.

s59
00:06:24.240 --> 00:06:29.600
Now this way you can imagine that you have one mega agent and internally it manages its own state.

s60
00:06:29.600 --> 00:06:35.120
At some points it's planning, some points it's building, some points it's executing and some points it's reporting.

s61
00:06:35.120 --> 00:06:37.039
And this is sort of what it looked like.

s62
00:06:37.039 --> 00:06:48.080
You know you have one big AI call maybe max steps 100 and you give it the ability to manage its own state based on where it's at inside of its execution journey.

s63
00:06:48.080 --> 00:07:00.319
And so you can see similar tools, you can see a similar shape, but the best part about this is if it ever ran to an error when executing or joining, it could go back and explore more or it could go and read more and figure out what it was doing wrong.

s64
00:07:00.319 --> 00:07:02.080
And it was very good at this point.

s65
00:07:02.080 --> 00:07:08.800
We were pretty confident in the actual system at hand and we actually spread it to a few trusted members ever.

s66
00:07:08.800 --> 00:07:15.199
You know, this is a very powerful tool and we didn't really want to put in the hands of the wrong people or people that were using very critical workloads.

s67
00:07:15.199 --> 00:07:18.800
So, we got to a few people's hands and the immediate response was it was awful.

s68
00:07:18.800 --> 00:07:20.080
You know, we thought we were cooking.

s69
00:07:20.080 --> 00:07:26.880
We thought this was, you know, nailing 30% of our evals, but we couldn't have anticipated some of the questions that were being asked.

s70
00:07:26.880 --> 00:07:35.199
And for us to spend more time manually mapping out some of these scenarios, it didn't seem like a very scalable way to do this.

s71
00:07:35.199 --> 00:07:39.599
And then claude code and opus 4.5 came out.

s72
00:07:39.599 --> 00:07:44.800
Well, more like Opus 4.5 came out and it in tangent with claw code which is so powerful.

s73
00:07:44.800 --> 00:07:57.280
You know they sort of unlocked the concept of a file system agent and we on the side were like wow clawed code and Opus 4.5 is basically AGI compared to what we had before.

s74
00:07:57.280 --> 00:08:05.440
You know it would answer most of our questions without even without even missing a beat um compared to the handgrown agent we had.

s75
00:08:05.440 --> 00:08:14.160
And when we tried to step back and wonder what we were doing wrong and why this was so much better, we realized that the big unlock was that it was just a file system.

s76
00:08:14.160 --> 00:08:24.479
You know, we it had a very minimal set of tools, list file, read file, run bash, and we gave a few more here for uh our own data agent use case.

s77
00:08:24.479 --> 00:08:32.479
But the biggest thing was it was able to use the tools that agents are well trained on and was able to explore and write work where it needs to.

s78
00:08:32.479 --> 00:08:37.279
you know, we weren't giving it claw code was not giving it a very prescriptive set of tools.

s79
00:08:37.279 --> 00:08:41.279
It was sort of just letting it go wild and explore emergent behavior.

s80
00:08:41.279 --> 00:08:44.959
And so from this, we learned that you can really just use a file system.

s81
00:08:44.959 --> 00:08:51.200
You know, we we saw the learnings from claw code and how powerful it was given that it just executes locally.

s82
00:08:51.200 --> 00:08:55.040
And we tried to rebuild it in a way that was very cloud codeesque.

s83
00:08:55.040 --> 00:08:57.600
You know, it was now going to run in a sandbox.

s84
00:08:57.600 --> 00:09:00.640
That sandbox would dump the whole semantic layer into it.

s85
00:09:00.640 --> 00:09:12.000
You could the agent would be able to grab, bash, read file, write file all around to figure out what it needs and we would just sprinkle a few tools on top to make sure it could do everything that is versel specific.

s86
00:09:12.399 --> 00:09:14.720
And this was actually the biggest unlock ever.

s87
00:09:14.720 --> 00:09:26.320
You know, the leap from single agent to cloud code SDK and then from cloud code SDK to file system agent in general, fine-tuned or purpose-built for our use case

s88
00:09:26.320 --> 00:09:27.600
was an amazing leap.

s89
00:09:27.600 --> 00:09:32.640
At this point, we were starting to get ready to give it away to more people at Versell.

s90
00:09:32.640 --> 00:09:36.560
And at this point, the eval score basically doubled.

s91
00:09:36.560 --> 00:09:39.040
And I wrote this uh this is basically how it looks.

s92
00:09:39.040 --> 00:09:40.080
Um it's very simple.

s93
00:09:40.080 --> 00:09:41.360
You just give it a bash tool.

s94
00:09:41.360 --> 00:09:50.480
We have a nice helper called bash tool on npm and you attach it to a sandbox and you can attach files to the sandbox for it to read, write and execute.

s95
00:09:50.480 --> 00:09:58.000
And after this revelation and after I saw that we were passing so many of the questions that we failed to do before, I wrote this banger blog post.

s96
00:09:58.000 --> 00:10:00.160
It's uh it's actually up today.

s97
00:10:00.160 --> 00:10:05.760
And the week that I wrote this, it was responsible for 70% of our versel.com traffic.

s98
00:10:05.760 --> 00:10:07.760
So you know it's a banger.

s99
00:10:07.760 --> 00:10:14.800
And after that, the next logical step was that we want to figure out the common use cases we had.

s100
00:10:14.800 --> 00:10:25.040
So by then we've already sort of let a leash on all of our cell and we were getting thousands of queries a day from people wanting everything from customer metrics sales metrics

s101
00:10:25.040 --> 00:10:33.600
number metrics npm downloads and it turns out that a lot of these queries are actually the same in shape you know there's only so many ways you can do an aggregation

s102
00:10:33.600 --> 00:10:43.200
only so many ways you can look up a product only so many ways you can do billing info and so we actually have a recurring job that takes the most recent queries and tries to distill them into a skill

s103
00:10:43.200 --> 00:10:50.959
and right now we have roughly 100 skills that do a mix of aggregation all the way through looking up specific data about certain people.

s104
00:10:50.959 --> 00:10:56.640
And we found this very effective because if you think about every new agent run, it sort of just starts from nothing.

s105
00:10:56.640 --> 00:11:01.760
You know, there's really no pre-established context besides, you know, the semantic layer and the system prompt.

s106
00:11:01.760 --> 00:11:08.959
But with a skill, it already starts off with a lot of contextual knowledge that has otherwise already been done.

s107
00:11:09.200 --> 00:11:10.959
And this is roughly how it looks.

s108
00:11:10.959 --> 00:11:15.440
It's very similar to the previous one, but the inclusion of a skills folder is actually very powerful.

s109
00:11:15.440 --> 00:11:18.240
Um, we also built this tool at Verscell called Skillsh.

s110
00:11:18.240 --> 00:11:23.279
It's the most popular way to find agent skills and run them yourself.

s111
00:11:23.360 --> 00:11:35.279
And I I'm saying all this because this journey is something that most of you may hit once in a while where you start from something simple and you gradually add complexity and you eventually hit a system in which you can ship to prod.

s112
00:11:35.279 --> 00:11:46.880
And I'm telling you this because at every step along building this agent, someone Everell was agent curious and they tried to fork off of my DZero agent and build their own.

s113
00:11:46.880 --> 00:11:51.760
And at every step, we sort of had a better way to do something that was not previously known.

s114
00:11:51.760 --> 00:12:00.959
And we were wondering like what if people today could start from the very last insight and not have to ever start from just a simple prompt or from reinventing

s115
00:12:00.959 --> 00:12:04.640
best principles from first principles.

s116
00:12:04.640 --> 00:12:09.200
And so we actually thought what if we built the Nex.js for agents.

s117
00:12:09.200 --> 00:12:18.320
For those that don't know, Nex.js is a popular web framework that Verscell built that invented this thing of file system uh framework defined infrastructure.

s118
00:12:18.320 --> 00:12:20.160
You don't have to worry about where things go.

s119
00:12:20.160 --> 00:12:25.920
You just have to write files in the right conventions and it automatically declares where they should go.

s120
00:12:25.920 --> 00:12:27.600
Your pages go to the CDN.

s121
00:12:27.600 --> 00:12:29.440
Your serverless functions go there.

s122
00:12:29.440 --> 00:12:31.680
Your caching goes in the middle.

s123
00:12:31.680 --> 00:12:34.560
And we thought, you know, building agents should be this simple.

s124
00:12:34.560 --> 00:12:40.320
You should only have to create a skills folder, a tools folder, a channels folder, and you should be able to just declare these very easily.

s125
00:12:40.320 --> 00:12:45.040
And the framework should know exactly how to make an agent out of it.

s126
00:12:45.040 --> 00:12:47.519
And that's why two weeks ago we released Eve.

s127
00:12:47.519 --> 00:13:04.079
Eve is a agent framework like the next.js GS for agents where it's very easy from just starting with a sample template to having a fully agent ready and being able to add in your own custom knowledge, your own custom tools and even integrated into the channels that you are familiar with.

s128
00:13:04.160 --> 00:13:06.639
This is roughly what we think an agent actually looks like.

s129
00:13:06.639 --> 00:13:09.360
You know, an agent has a runtime and it has channels.

s130
00:13:09.360 --> 00:13:11.839
And in that runtime, you're going to have durability.

s131
00:13:11.839 --> 00:13:14.000
You're going to want to run things in an isolate environment.

s132
00:13:14.000 --> 00:13:15.920
You're going to want to call into different models.

s133
00:13:15.920 --> 00:13:18.000
And you're going to want to have connections.

s134
00:13:18.000 --> 00:13:20.079
And we built this with open source in mind.

s135
00:13:20.079 --> 00:13:28.639
You know, we built Eve so you can plug in your own open source adapters for Postgress, OpenAI's uh responses API, Docker, other connectors.

s136
00:13:28.639 --> 00:13:31.279
But we also made it incredibly easy to deploy in Verscell.

s137
00:13:31.279 --> 00:13:39.440
The only thing here you see different is that everything here is using a Verscell product that we've been building over the years in order to make it easy to build these experiences.

s138
00:13:39.440 --> 00:13:50.480
Versell workflows for durability, sandbox for secure execution, and Verscell connect, something we just released to make it easy to generate short-lived ODC tokens for connections.

s139
00:13:51.040 --> 00:13:59.680
And we actually rewrote the whole D0ero agent in Eve as we were building Eve and from the convoluted structures behind the scenes that you did not see from the code.

s140
00:13:59.680 --> 00:14:01.839
Um, this is roughly how the file system looks.

s141
00:14:01.839 --> 00:14:02.560
It's very simple.

s142
00:14:02.560 --> 00:14:12.079
You have a bunch of system instructions, a couple skills, a couple tools, and it's very easy to compose this into a real agent, and it's very easy to iterate on.

s143
00:14:12.079 --> 00:14:17.760
We actually gave this out to a few beta customers before we actually fully released it two weeks ago at our London event.

s144
00:14:17.760 --> 00:14:20.560
And this one company that partners closely with us, Aura.

s145
00:14:20.560 --> 00:14:26.160
They've rebuilt their agent that's sort of like a mini claw to go and test people's services.

s146
00:14:26.160 --> 00:14:29.839
It goes to websites, installs them, it tries to use them.

s147
00:14:29.839 --> 00:14:38.880
And they've seen incredible success on building their own agent from the ground up using Eve compared to using an off-the-shelf cloud code.

s148
00:14:38.880 --> 00:14:44.079
Fewer steps, better successes, as well as better insights.

s149
00:14:45.040 --> 00:14:49.760
And when you deploy Eve to Verscell, you get observability observability out of the box.

s150
00:14:49.760 --> 00:15:00.320
You can see here that you get all the agent runs, you see all the tool calls, you see each step it takes as well as maybe some estimated costs and some optimizations you could potentially take.

s151
00:15:00.399 --> 00:15:03.040
And you can get start today at eve.dev.

s152
00:15:03.040 --> 00:15:07.680
You can just clone it and you can just start a template, deploy easily, self-host if you need.

s153
00:15:07.680 --> 00:15:15.040
And the reason why I bring this up is because I hope that there will be more and more business specific use case agents.

s154
00:15:15.040 --> 00:15:26.959
You know, before we built Ezero, we actually battle tested a lot of the industry well-funded startups that were doing these vertical agents that were dedicated to taking your Snowflake instance

s155
00:15:26.959 --> 00:15:31.440
and making it so their agent could run Snowflake queries against it.

s156
00:15:31.440 --> 00:15:39.040
But we found out that what really makes this agent good is it has a lot of very specific uh company knowledge.

s157
00:15:39.040 --> 00:15:41.839
You know, the way that Versel is a web- based company.

s158
00:15:41.839 --> 00:15:45.440
We have a lot of customers that have websites and web properties.

s159
00:15:45.440 --> 00:15:51.600
That goes a lot deeper into when you should query for what and what things link to what.

s160
00:15:51.600 --> 00:15:54.320
And so a lot of these off-the-shelf agents, they're great.

s161
00:15:54.320 --> 00:16:03.759
They're good to try, but I think if you really want to get the most juice out of a squeeze, you should really try to build your own agent and add in as much company specific knowledge as you can.

s162
00:16:03.759 --> 00:16:20.399
Today, you know, we've had 20 roughly decently PMF agents adversel that range from anything from marketing retros to figure out who to reach out to to the first ever red line of a contract when legal sees a new negotiation

s163
00:16:20.399 --> 00:16:24.959
all the way to my data science agent helping with with data queries.

s164
00:16:24.959 --> 00:16:28.800
And that goes to show that we ever have been very agent-filled.

s165
00:16:28.800 --> 00:16:33.279
You know, all of this stuff is actually saving us a lot of time.

s166
00:16:33.279 --> 00:16:35.040
The data team has never been more productive.

s167
00:16:35.040 --> 00:16:46.720
They have more time to go and improve the performance of Snowflake, to add new data sources that were missing, to fill in the gaps that they previously did not have time to because they were so busy writing queries.

s168
00:16:46.720 --> 00:16:58.320
And I think it's never been easier for you at your big, small, medium-sized company to sort of automate away some of the things that you do not want to do or some of the things that you're spending too much time doing.

s169
00:16:58.320 --> 00:17:04.880
You know, I think a lot of HR, finance, sales can be somewhat automated with agents.

s170
00:17:04.880 --> 00:17:09.600
And I think Eve is the best way to build said agents today.

s171
00:17:09.679 --> 00:17:11.120
And these are my socials.

s172
00:17:11.120 --> 00:17:12.959
Thank you all for coming and listening.

s173
00:17:12.959 --> 00:17:17.360
I'm Andrew and I'll be around if you want to chat outside.

s174
00:17:31.942 --> 00:17:33.942
[music]