WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=sM1iYgz93HI

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/sM1iYgz93HI.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.719 --> 00:00:14.240
Hi everyone, I'm Ben Kuss.

s3
00:00:14.240 --> 00:00:28.800
I'm CTO of Box and today I'm going to be talking about uh building for change and specifically around uh AI agents and how to continue to adapt uh your infrastructure as we are all in the middle of this journey.

s4
00:00:28.800 --> 00:00:35.520
Um, so before I get too far, I will quickly sort of set a little bit of like who I am and uh sort of what I do.

s5
00:00:35.520 --> 00:00:42.640
Um, for Box, I'm CTO and one of my jobs in my job for my whole career has been to build enterprise software.

s6
00:00:42.640 --> 00:00:50.320
And so if today um you're from a consumer company or you're uh not involved in enterprise, I hope that a lot of it is still relevant.

s7
00:00:50.320 --> 00:00:56.000
But in many cases um a lot of the lessons I've learned are enterprise uh specific.

s8
00:00:56.160 --> 00:01:12.880
So um I sort of will highlight um when I'm thinking and talking about infrastructure when I'm talking about uh the kind of challenges that we face um I'm typically talking about things that are sort of in a scale of like uh like for box we have over an exabyte of data

s9
00:01:12.880 --> 00:01:25.119
not a gigabyte not a terabyte not a pabyte but an exabyte um and then oftentimes we're I'm thinking in the tens of of millions of users uh the hundreds of billions of things in our case files or content or unstructured content

s10
00:01:25.119 --> 00:01:29.920
um and then the new stat that is sort of the the one that we talk about is uh tokens.

s11
00:01:29.920 --> 00:01:35.600
Um so we are now in the ballpark of trillion tokens probably will be 10 trillion tokens sometime soon.

s12
00:01:35.600 --> 00:01:42.799
Um and this of course is uh uh some of the new and interesting challenges that this kind of scale brings.

s13
00:01:43.040 --> 00:01:51.119
Um so in my career and um I think maybe many of us here um we've kind of lived through these this these technology changes.

s14
00:01:51.119 --> 00:02:06.719
And so taking a quick step back um I started uh my career when the internet was sort of becoming a thing uh lived through mobile and sort of this idea of like you know carrying these different devices move to the cloud where you could kind of store and maintain all your data and then of course we're all

s15
00:02:06.719 --> 00:02:16.400
in the middle of this AI change and I think when you see these kind of technology disruptions when you're sort of thinking about this idea of like all of these kind of have changed all of our lives

s16
00:02:16.400 --> 00:02:24.000
um and then uh and you're thinking about it from the perspective of a technology leader or a startup or a engineer.

s17
00:02:24.000 --> 00:02:28.560
Um, you kind of see that like these are where like major companies are born.

s18
00:02:28.560 --> 00:02:32.080
Um, you see that like big companies adapt or die.

s19
00:02:32.080 --> 00:02:33.920
Um, small companies are here to disrupt things.

s20
00:02:33.920 --> 00:02:34.720
They're here to take bets.

s21
00:02:34.720 --> 00:02:35.760
They're here to grow.

s22
00:02:35.760 --> 00:02:37.040
Um, I've had two startups.

s23
00:02:37.040 --> 00:02:38.000
I've been acquired twice.

s24
00:02:38.000 --> 00:02:40.000
Once in IBM, once in a box.

s25
00:02:40.000 --> 00:02:44.800
Um, and um, so that we're in the middle of this kind of major opportunity.

s26
00:02:44.800 --> 00:02:54.080
Um but for um a long tu uh no matter what you kind of come from and what area you're at I uh t typically if you were to ask my advice

s27
00:02:54.080 --> 00:03:05.599
um and about kind of what makes it you successful as an company as an engineering organization as a technology startup or as a like a company who has a a technology division

s28
00:03:05.599 --> 00:03:15.599
um I would say no matter what there's kind of three things the first is you need to build scalable reliable uh platforms and select the technology that you care about.

s29
00:03:15.599 --> 00:03:19.040
Meaning that like there's a lot of ways to do things, but get good at something.

s30
00:03:19.040 --> 00:03:23.680
Get good at that technology, at that system, at that stack, and then and then keep going with that.

s31
00:03:23.680 --> 00:03:27.760
Um, and then you leverage this technology so that you do more for your customers.

s32
00:03:27.760 --> 00:03:37.040
Um, you build your better product, you develop the capabilities, and then you optimize it, make it better, make it faster, make it cheaper, make it uh more capable.

s33
00:03:37.040 --> 00:03:43.920
And this was sort of the generic enterprise advice, uh, the generic energy advice that many many people would follow.

s34
00:03:43.920 --> 00:03:52.080
And I think this works really well except now I don't know if this is good advice.

s35
00:03:52.080 --> 00:03:55.920
It has been across all these major disruptive changes over time.

s36
00:03:55.920 --> 00:03:56.640
Unclear.

s37
00:03:56.640 --> 00:04:06.799
In fact, I don't think it's good advice right now because there's a funny thing happening right now which that didn't happen in those previous trends which is that the rate of change is dramatically higher.

s38
00:04:06.799 --> 00:04:16.400
Um I and you might say look technology always is changing like you know there's other trends things change a lot but not that much um the internet is still based on HTTP

s39
00:04:16.400 --> 00:04:30.080
the uh uh mobile devices are still um iOS and Android based and so on and so but nowadays um other than the fact that like uh generative AI exists most things that power it are changing and changing dramatically.

s40
00:04:30.800 --> 00:04:40.800
So last year I was here at this uh at the AI engineering world fair and I gave a speech and I said uh after spending a lot of time on this and thinking through this

s41
00:04:40.800 --> 00:04:45.919
I think there's a key to this which is a gentic uh uh graph-based approach.

s42
00:04:45.919 --> 00:04:52.720
The idea was um uh you have a large language model and these nodes and then you sort of put them together and you have the sort of the AI

s43
00:04:52.720 --> 00:04:55.440
traverse this graph that you set up you build the graph.

s44
00:04:55.440 --> 00:05:05.520
This is the key approach that and if you use this this is going to really help you sort of build agents because what is anything that we do in life it's a agent it's a workflow

s45
00:05:05.520 --> 00:05:15.600
um uh it's a it's a way and then if you have an intelligent uh uh a agent it can basically traverse this is the key approach and I believe that at the time and a lot of people did

s46
00:05:15.600 --> 00:05:32.479
and I still love this approach but nowadays it's sort of a little bit out of date in fact I remember um uh a guy came up to me after my speech last time and he was like the problem you talked about the the answer is just this exactly you were speaking to me thank you so much and

s47
00:05:32.479 --> 00:05:42.960
and I was happy I I you know I gave a good speech and gave somebody some good advice um and then I remember when we made a change I was like I wonder what happened to that guy I wonder if he's here uh it's uh uh so

s48
00:05:42.960 --> 00:05:54.880
um but the problem is is that not that it was wrong that that was the best approach but a new way emerged um in fact I started to like look through like all of the last year's

s49
00:05:54.880 --> 00:06:05.280
uh events and actually go to other conferences like what what did people talk about a year ago and most of them were again nothing much wrong all good speeches all all good ideas but

s50
00:06:05.280 --> 00:06:19.600
most of them have now a better way there um and uh so um and this is be sort of the gist of of the challenge so if you look at our journey of the technologies the kind of things that we care about

s51
00:06:19.600 --> 00:06:33.199
um I'll just kind of rapid file here like so let's say that you want to utilize AI models and let's just look the last couple years a long a while ago probably distant memory now like people would say train your own my models or maybe fine tune them like nah that doesn't that's too why bother just use

s52
00:06:33.199 --> 00:06:44.800
a frontier model use something from openi use something from anthropic use something from from Gemini um and then that's great but it's kind of expensive okay great let's just use openweight models they're pretty close you can use them you can host them yourself you can get some good GPUs

s53
00:06:44.800 --> 00:06:57.520
um but then uh some companies will come to you and they'll be like look we just did this big deal with open orthropic like can we use our own key or bring our own model like sure you can do that too but then nowadays is probably the best approach is to do an adaptive model selection where you

s54
00:06:57.520 --> 00:07:04.639
basically are picking the big and smaller models what which model does well and this is kind of the cool new thing maybe I could give a talk on that

s55
00:07:04.639 --> 00:07:13.680
um or let's say you're building agents like we lot lot of things here like um I mean the word agent hasn't really been around for that long but in that time it used to be like a singleshot

s56
00:07:13.680 --> 00:07:21.199
uh LLM response call that an agent if you feel like it then you have to say no okay we're chain of thought reasoning now we're going to make a graph-based agent system

s57
00:07:21.199 --> 00:07:26.720
like I presented last year uh no then it turns that why are you bothering to make graphs when you could actually have an agent just figure out what to do?

s58
00:07:26.720 --> 00:07:27.199
Make a plan.

s59
00:07:27.199 --> 00:07:28.000
That's the new approach.

s60
00:07:28.000 --> 00:07:32.000
That's kind of the way that um Claude sort of uh laid the approach there.

s61
00:07:32.000 --> 00:07:33.280
And you're like, okay.

s62
00:07:33.280 --> 00:07:41.360
And then now it's like, well, you if you want to use dedicated sub agents, maybe, but then maybe why not just make a generic agent and have it recursively work and then give it skills.

s63
00:07:41.360 --> 00:07:42.240
Skills are very generic.

s64
00:07:42.240 --> 00:07:43.199
They're super helpful.

s65
00:07:43.199 --> 00:07:52.560
Um and then maybe now it's maybe the idea is not just to do that, but to do it with a agent sandbox so the agent can write code and execute it because that's super useful because agents are great programmers.

s66
00:07:52.560 --> 00:07:54.479
I'd have them sort of just live in their own computer.

s67
00:07:54.479 --> 00:08:00.319
Um, and then arguably now that's the best approach or maybe even we're in the world now of like don't even bother with any of that.

s68
00:08:00.319 --> 00:08:13.840
Just bring your own harness like like like let people select if they want to use one of these other systems and then you know not even just building agents but the technology around context retrieval things like um you know in the old world we were like BM25 and keyword search that's the way to do things but that's

s69
00:08:13.840 --> 00:08:14.960
like distant memory.

s70
00:08:14.960 --> 00:08:21.599
Uh obviously the future is is retrieve augmented generation embeddings approximate nearest neighbor that's how we're going to find data.

s71
00:08:21.599 --> 00:08:24.800
Turns out that doesn't really scale well and it kind of almost mimics randomness as you keep going.

s72
00:08:24.800 --> 00:08:26.639
So then maybe it's about graphs.

s73
00:08:26.639 --> 00:08:27.759
It's difficult to get working well.

s74
00:08:27.759 --> 00:08:28.720
It's probably not the best.

s75
00:08:28.720 --> 00:08:30.000
Um so then it's about hybrid.

s76
00:08:30.000 --> 00:08:34.479
You want a lexical and you want to do semantic search and rank fuse those together.

s77
00:08:34.479 --> 00:08:35.360
Arguably not.

s78
00:08:35.360 --> 00:08:40.800
Arguably agents are actually way better at finding data because they can find things and apply their intelligence to get to it.

s79
00:08:40.800 --> 00:08:48.399
So each of these things I just mentioned is arguably the leading approach for that moment over time.

s80
00:08:48.399 --> 00:08:51.200
If you asked me to give a speech right now on any one of these, I would pick the last.

s81
00:08:51.200 --> 00:08:57.839
I'd pick the adapted models with dedicated RM style agent and agentic search powered by hybrid.

s82
00:08:57.839 --> 00:09:01.360
But uh is this the end of this journey?

s83
00:09:01.360 --> 00:09:04.240
This is not that long of that time here.

s84
00:09:04.240 --> 00:09:10.800
And so um my guess is the stuff that you're learning today likely won't last that long.

s85
00:09:10.800 --> 00:09:17.519
Not that it's not wrong, not that it is not the best answer right now, but probably something's going to change.

s86
00:09:17.519 --> 00:09:23.120
The big thing that changed last year was in my mind Opus 40 to Opus 45.

s87
00:09:23.120 --> 00:09:28.320
When you did that, suddenly you got to a model that could do instruction following and really high scale.

s88
00:09:28.320 --> 00:09:32.320
This is kind of to me the beginning the epic of like the new agent models.

s89
00:09:32.320 --> 00:09:35.360
Uh also uh hardware is getting better, faster, cheaper.

s90
00:09:35.360 --> 00:09:36.800
Maybe we'll start to use more tokens.

s91
00:09:36.800 --> 00:09:39.519
Like token usage is off the charts of course and is that good or bad?

s92
00:09:39.519 --> 00:09:40.800
What's going to change there?

s93
00:09:40.800 --> 00:09:49.839
Enterprises are adopting things differently whether or not a company has decided to go all in on one agent to rule them all sort of like claude or maybe codec style agent

s94
00:09:49.839 --> 00:09:55.040
um or maybe they want to utilize uh agents from different platforms and different systems or both.

s95
00:09:55.040 --> 00:10:02.880
This is going to affect your lives um in addition to things like just the new techniques new interesting uh approaches new technology to power these things.

s96
00:10:02.880 --> 00:10:15.120
So the fact that everybody in is working so hard on this trillions of dollars investment is actually leading to a lot of this change and again it's happening way faster than than I've ever seen for sure.

s97
00:10:15.120 --> 00:10:19.839
Um, now if you look back, um, it's not this way with everything else.

s98
00:10:19.839 --> 00:10:24.240
Like if you go see some of these other discussions, like I've given a speech on some of these topics.

s99
00:10:24.240 --> 00:10:26.079
I looked at them come some of a few years old.

s100
00:10:26.079 --> 00:10:26.800
They're pretty good.

s101
00:10:26.800 --> 00:10:27.920
I still think they're very relevant.

s102
00:10:27.920 --> 00:10:38.320
You want to talk about large scale databases about uh identity access controls, how to scale engineering teams, how to do multi cloud storage, probably um these are still relevant things today.

s103
00:10:38.320 --> 00:10:44.720
These do not change as fast despite being high-scale uh interesting powerful uh technologies.

s104
00:10:45.040 --> 00:11:00.000
So the previous wisdom of saying optimize for specific technologies go deep switch rarely right this what this the reason you do that is because switching is hard migrations suck whenever you migrate you break something every time no matter what

s105
00:11:00.000 --> 00:11:01.557
um it's always harder than you think even

s106
00:11:01.557 --> 00:11:01.600
[music]

s107
00:11:01.600 --> 00:11:12.480
if you know that um and uh and the switching cost is basically high so basically don't do it for most things you're kind of uh just because something's better out there that's not the answer for most infrastructure

s108
00:11:12.480 --> 00:11:20.160
so typically If you say half life of an agent infrastructure, 3 to 5 years, reevali, see what's out there.

s109
00:11:20.160 --> 00:11:22.880
We've been using databases like my SQL databases for a long time.

s110
00:11:22.880 --> 00:11:25.040
It's still pretty good and probably need to replace it soon.

s111
00:11:25.040 --> 00:11:40.959
But um but now with AI technologies, arguably the halflife is measured in months, meaning a few months after you've adopted what might be the best possible thing, there's a significant chance that you're going to have to replace it coming soon.

s112
00:11:40.959 --> 00:11:44.079
And this is I think shocking.

s113
00:11:44.079 --> 00:11:48.320
Maybe I see from some of your reactions that like you're kind of like have experienced this a little bit.

s114
00:11:48.320 --> 00:11:53.760
Um but this is a different aspect of the way that you build technology.

s115
00:11:53.760 --> 00:12:02.160
So if you're an engineer, this really sucks because the thing that you just learned and that you're making is now probably going to be out of date soon.

s116
00:12:02.160 --> 00:12:04.640
Uh no engineer I know likes this.

s117
00:12:04.640 --> 00:12:07.360
Um uh as a startup, you bet on something.

s118
00:12:07.360 --> 00:12:08.240
You're like, we're going to go all in.

s119
00:12:08.240 --> 00:12:20.240
We're going to go on the technology this approach and then we're going to basically uh uh disrupt somebody which probably will but then you see and see now like the first phase of AI companies are starting to get disrupted by the next phase.

s120
00:12:20.240 --> 00:12:33.519
If you're a technology buyer, you're a leader of a company, you buy technology, you you you select uh open source models, you select vendors, there's a significant chance that whatever you just bought is not going to be the approach you're going to invest.

s121
00:12:33.519 --> 00:12:38.160
That's you know, good luck doing a three-year deal like on on things about about this kind of stuff.

s122
00:12:38.160 --> 00:12:49.040
Or if you're VC, uh maybe the coolest best thing that everybody agrees is the greatest opportunity is no longer going to be the opportunity soon um because everything's changing.

s123
00:12:49.040 --> 00:12:50.880
So here's my advice.

s124
00:12:50.880 --> 00:12:58.800
Get good at It's almost silly to say because you know obviously technology changes.

s125
00:12:58.800 --> 00:13:02.160
Obviously it's something that is um you know built in.

s126
00:13:02.160 --> 00:13:02.959
Of course we're all going to change.

s127
00:13:02.959 --> 00:13:04.320
We've done this for a long time.

s128
00:13:04.320 --> 00:13:05.519
Um it's hard.

s129
00:13:05.519 --> 00:13:09.760
I think it's really hard and the faster that you do it the harder it is.

s130
00:13:09.760 --> 00:13:16.720
Uh when I was going through that uh like oh yeah we switched from the graph based agent to a um to the more looping style deep style agent.

s131
00:13:16.720 --> 00:13:19.040
I remember very well the conversation with the engineer.

s132
00:13:19.040 --> 00:13:20.000
He just he's like, "I did it.

s133
00:13:20.000 --> 00:13:22.639
I got a tic search working and this approach does deep research.

s134
00:13:22.639 --> 00:13:25.040
It does all this stuff just like you asked."

s135
00:13:25.040 --> 00:13:28.720
Okay, we're going to switch rebuild it again in this new technology.

s136
00:13:28.720 --> 00:13:29.920
And he's like, "Wait, what?"

s137
00:13:29.920 --> 00:13:31.120
Like, "It's working.

s138
00:13:31.120 --> 00:13:32.000
You did what you're talking."

s139
00:13:32.000 --> 00:13:33.600
Yeah, but it's not as capable as we wanted it to be.

s140
00:13:33.600 --> 00:13:34.320
Like, what do you mean?

s141
00:13:34.320 --> 00:13:35.120
You didn't tell me that before.

s142
00:13:35.120 --> 00:13:38.160
Like, and then and then so convince him like, "Okay, this is a new approach."

s143
00:13:38.160 --> 00:13:41.040
And then, you know, he does it and it's good.

s144
00:13:41.040 --> 00:13:45.760
Two months later, uh, we we're we're actually shipping the product uh on on Tuesday.

s145
00:13:45.760 --> 00:13:50.240
And then I was like, "Okay, uh, guys, on Wednesday we're going to rebuild it again on the new approach."

s146
00:13:50.240 --> 00:13:51.519
And they're like, "What are you talking about?"

s147
00:13:51.519 --> 00:13:56.000
Like, like, um, then they'll say, like, it's it's almost hard on everybody like, "Wait, wait, give me more time.

s148
00:13:56.000 --> 00:13:57.920
I'll I'll make the new way the old way do it better."

s149
00:13:57.920 --> 00:14:00.480
Um, uh, like, and then also they're skeptical.

s150
00:14:00.480 --> 00:14:03.760
Like, now you say that, but like this is going to change again, right?

s151
00:14:03.760 --> 00:14:05.839
Like, who are you to like make these choices?

s152
00:14:05.839 --> 00:14:08.480
And the answer is, yeah, I'm pretty sure it's going to change again.

s153
00:14:08.480 --> 00:14:11.120
So, this is, I think, a leadership problem.

s154
00:14:11.120 --> 00:14:12.720
It's a technology problem.

s155
00:14:12.720 --> 00:14:14.000
It's a morale problem.

s156
00:14:14.000 --> 00:14:15.120
It's a team problem.

s157
00:14:15.120 --> 00:14:19.440
It's a company problem and if you're not careful, it is actually can destroy you.

s158
00:14:19.440 --> 00:14:22.560
It can destroy a lot of things because people lose faith, they lose morale.

s159
00:14:22.560 --> 00:14:24.639
It's a problem.

s160
00:14:24.639 --> 00:14:31.040
So, um if my advice is change um and be ready for change, how are you going to do it?

s161
00:14:31.040 --> 00:14:33.440
Three things to give you.

s162
00:14:33.440 --> 00:14:35.760
One, um you just got to prepare people.

s163
00:14:35.760 --> 00:14:37.920
Well, this is a kind of a people challenge.

s164
00:14:37.920 --> 00:14:44.720
So when you build your teams, when you talk to them, when you prepare them, if they're in AI world, you got to tell them like expect change.

s165
00:14:44.720 --> 00:14:45.600
It's normal.

s166
00:14:45.600 --> 00:14:46.720
It's not a problem.

s167
00:14:46.720 --> 00:14:48.399
It's not that you did something wrong.

s168
00:14:48.399 --> 00:14:55.120
This is is weirdly like um like helps people like I have a a technology review team and and then and then they're like we can't like change.

s169
00:14:55.120 --> 00:14:55.440
We don't know.

s170
00:14:55.440 --> 00:14:55.920
We're not sure.

s171
00:14:55.920 --> 00:14:58.320
We can't tell you that in two years from now this is going to be best.

s172
00:14:58.320 --> 00:14:59.519
Like that's okay.

s173
00:14:59.519 --> 00:15:01.839
Uh we're gonna we're gonna build these things that change.

s174
00:15:01.839 --> 00:15:02.800
So just go with it.

s175
00:15:02.800 --> 00:15:04.240
You have to pick something.

s176
00:15:04.240 --> 00:15:09.040
Um, also whenever possible if you can build an abstraction so that it lets you swap out what's underneath.

s177
00:15:09.040 --> 00:15:18.000
We have an agent extraction in box and you're able to go through and be like uh like select things underneath and the agent still works the same for the customers but it it's better underneath.

s178
00:15:18.000 --> 00:15:29.040
Um, and the idea is that change is not a mistake and and I highlight like that's very hard for most people and and I and I would sort of just you just I tell them all the time change is not a mistake.

s179
00:15:29.040 --> 00:15:31.199
You wouldn't nobody knew six months ago.

s180
00:15:31.199 --> 00:15:32.959
Nobody today will know six months from now.

s181
00:15:32.959 --> 00:15:35.360
It's Seems very true.

s182
00:15:35.360 --> 00:15:39.120
So uh at Box we are now in the habit of reviewing every six months no matter what.

s183
00:15:39.120 --> 00:15:40.160
This is great technology.

s184
00:15:40.160 --> 00:15:40.959
We love it.

s185
00:15:40.959 --> 00:15:46.320
Review in six months like because uh which is just completely crazy for everything else that we're doing.

s186
00:15:46.320 --> 00:15:48.480
Everything else is like three years.

s187
00:15:48.480 --> 00:15:54.560
Um also even though change is critical you um you need to define what you mean when change.

s188
00:15:54.560 --> 00:15:57.680
If you just change all the time there's a new paper it's awesome.

s189
00:15:57.680 --> 00:16:01.279
You know our CEO Aaron is very active on all the newest things.

s190
00:16:01.279 --> 00:16:02.240
He's like check this out.

s191
00:16:02.240 --> 00:16:04.079
Like don't change just because of that.

s192
00:16:04.079 --> 00:16:05.759
Like don't change just because it's a trend.

s193
00:16:05.759 --> 00:16:07.839
Change because you know it matters.

s194
00:16:07.839 --> 00:16:09.519
And how do you know it matters?

s195
00:16:09.519 --> 00:16:10.880
Probably pitch you on eval sets.

s196
00:16:10.880 --> 00:16:14.480
If you're building agents, if you're building AI, make sure that you know what people have.

s197
00:16:14.480 --> 00:16:17.040
You have the ability to give the same input, expect certain output.

s198
00:16:17.040 --> 00:16:19.600
Grade that cost, speed, quality, capabilities.

s199
00:16:19.600 --> 00:16:22.639
These are the things that you probably are going to to be wanting.

s200
00:16:22.639 --> 00:16:24.800
So for us, it's easy.

s201
00:16:24.800 --> 00:16:28.560
Does the new approach work better for our eval sets?

s202
00:16:28.560 --> 00:16:29.839
What the customer cares about?

s203
00:16:29.839 --> 00:16:31.839
If the answer is yes, strongly consider switching.

s204
00:16:31.839 --> 00:16:38.560
If the answer is no, don't bother like or or keep working on a little bit of work to see if you can make sure that you you've fully explored it.

s205
00:16:38.560 --> 00:16:44.160
Um and then so the idea is uh build a system that lets you be able to change.

s206
00:16:44.160 --> 00:16:52.240
And then the third uh and final piece of advice here is um almost certainly none of us can keep up with everything.

s207
00:16:52.240 --> 00:16:53.680
It is very hard.

s208
00:16:53.680 --> 00:16:59.040
Um I think I heard uh Andre Kaparthy uh he he was like everything changes so fast I can't keep up.

s209
00:16:59.040 --> 00:17:04.799
and you're like you're sort of quite famously good at keeping up and so like what's the hope for everybody else if if that's the case.

s210
00:17:04.799 --> 00:17:08.079
Um and so but then so what you do is you rely on somebody else.

s211
00:17:08.079 --> 00:17:12.319
You rely on a technology, you rely on a vendor, you rely on a platform.

s212
00:17:12.319 --> 00:17:22.400
Um you when you select it and um and then here I think very use I mean like whenever you whenever anybody's bought technology in the past I would had advised them like

s213
00:17:22.400 --> 00:17:24.000
look at what they do now.

s214
00:17:24.000 --> 00:17:25.199
Double check the road map.

s215
00:17:25.199 --> 00:17:25.839
Make sure it's good.

s216
00:17:25.839 --> 00:17:28.880
Make sure it's on the path you want but just focus on what's available now.

s217
00:17:28.880 --> 00:17:45.280
But I think something else here is um should do that of course that's most important thing but like look back how have they handled change what's their attitude towards change how can what can you when you talk to them when you read about their stuff like what happened six months ago what happened a year ago how did

s218
00:17:45.280 --> 00:18:01.200
they handle that transition many of the vendors that I really like right now have reinvented themselves three times in the last year and I now trust that if something else comes along they're very good at this they understand agent technologies they understand the the eval sets they understand the the observability systems and then you can say ah

s219
00:18:01.200 --> 00:18:09.120
okay good I hope that they keep up and then I now my sort of thing I need to do is just evaluate whether or not that's a good platform

s220
00:18:09.120 --> 00:18:24.320
so um making sure that you have this sort of platforms that do well is is is critical um and um if anybody's interested in unstructured content and AI associated with it uh Box has a booth downstairs happy to talk to you about those kind of things

s221
00:18:24.320 --> 00:18:33.840
um and then um I I'll leave you with this is um I actually I fully bet and I believe that um a company that's born this year was born last year

s222
00:18:33.840 --> 00:18:44.400
um will or maybe even a company a medium-sized company or a big company will will they they'll shoot very high the company that will dominate tomorrow is is is now born today.

s223
00:18:44.400 --> 00:18:52.160
Uh but I kind of bet you that the technology approach that they have right now is probably going to change multiple times before they do that.

s224
00:18:52.160 --> 00:19:05.679
So interestingly it's like the challenge the advice the thought here is build for change adaptability arguably that's the moat that you have until that changes.

s225
00:19:05.679 --> 00:19:08.228
Okay thank you everyone.

s226
00:19:08.228 --> 00:19:10.228
[applause]