WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=7A65O-0lvKE

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/7A65O-0lvKE.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.680 --> 00:00:14.240
Welcome everybody.

s3
00:00:14.240 --> 00:00:17.360
Sorry for everybody who was already here and missed the Coinbase guy.

s4
00:00:17.360 --> 00:00:20.400
I have no idea where he went or why he didn't come.

s5
00:00:20.400 --> 00:00:23.720
I was actually pretty excited to hear about his his comments.

s6
00:00:23.720 --> 00:00:28.160
But today I'm going to talk about which AI startups actually win enterprise contracts.

s7
00:00:28.160 --> 00:00:32.279
So to begin I thought this was going to be a different audience.

s8
00:00:32.279 --> 00:00:35.720
I didn't realize this was going to be mostly people on the leadership track.

s9
00:00:35.720 --> 00:00:38.440
I thought I was going to be speaking to more AI engineers.

s10
00:00:38.440 --> 00:00:45.000
So maybe just by show of hands, how many of you represent like the engineering or startup or like seller side?

s11
00:00:45.000 --> 00:00:47.360
And then how many of you represent maybe like the buyer side?

s12
00:00:47.360 --> 00:00:50.280
Like you're in the enterprise, you're trying to get these tools in.

s13
00:00:50.280 --> 00:00:51.840
Okay, so we got a good mix.

s14
00:00:51.840 --> 00:00:53.960
I'm going to try to balance that out today.

s15
00:00:53.960 --> 00:00:57.400
I work on product stuff at Millennium, which is a hedge fund.

s16
00:00:57.400 --> 00:00:58.720
We build a lot of stuff.

s17
00:00:58.720 --> 00:01:00.160
I can't talk about any of it.

s18
00:01:00.160 --> 00:01:04.199
So I'm going to talk about stuff that we we look at and evaluate.

s19
00:01:04.199 --> 00:01:05.720
It's going to be pretty generic.

s20
00:01:05.720 --> 00:01:11.280
I tried to make it as interesting as possible while still getting my compliance department to be okay with me doing this.

s21
00:01:11.280 --> 00:01:15.040
But also need to say legally that I'm speaking as an individual.

s22
00:01:15.040 --> 00:01:18.720
I am not representing my company and all opinions are my own.

s23
00:01:18.720 --> 00:01:21.880
So with that we can dive in.

s24
00:01:21.880 --> 00:01:25.000
So I ran through this with my parents last week.

s25
00:01:25.000 --> 00:01:27.440
I don't I grew up not that far from here.

s26
00:01:27.440 --> 00:01:31.480
And my mom basically said, "Why are you spending your time teaching vendors how to sell to you?

s27
00:01:31.480 --> 00:01:33.720
Aren't you busy enough already?"

s28
00:01:33.720 --> 00:01:37.920
And the real answer to that question is I really like how stuff works.

s29
00:01:37.920 --> 00:01:39.920
I like seeing stuff come together.

s30
00:01:39.920 --> 00:01:42.360
My bachelor's degree was in economics.

s31
00:01:42.360 --> 00:01:44.960
And I really love seeing things just work well.

s32
00:01:44.960 --> 00:01:51.080
So AI's been really interesting because it's kind of a whole new paradigm of how businesses are doing work.

s33
00:01:51.080 --> 00:01:54.480
That's the whole point of this track, this AI native enterprise track.

s34
00:01:54.480 --> 00:02:02.560
And so even though I don't need more people DMing me on LinkedIn, um I'm actually really excited to talk about this.

s35
00:02:02.560 --> 00:02:10.200
So, my hypothesis in short is basically at current model intelligence, most of the value available is already being left on the table.

s36
00:02:10.200 --> 00:02:14.400
Um this is not a hot take for most people, I think who work in enterprise.

s37
00:02:14.400 --> 00:02:16.160
You've probably seen this problem.

s38
00:02:16.160 --> 00:02:26.160
This little stat at the bottom is uh pretty heavily uh repeated for a lot of people who work inside of business circles, and they all kind of like laugh, and they're like, "Yeah, yeah, you know, all these AI tools,

s39
00:02:26.160 --> 00:02:27.959
how much are they actually doing?"

s40
00:02:27.959 --> 00:02:29.800
Um and I want to talk about why.

s41
00:02:29.800 --> 00:02:33.920
So, there's kind of two sides to this becoming gen AI native.

s42
00:02:33.920 --> 00:02:41.520
Um you have models and products, which are one side, that's the seller side, and then you also have systems and all of what's inside of the enterprise, that's the buyer side.

s43
00:02:41.520 --> 00:02:44.560
So, that's the side that I deal with a lot.

s44
00:02:44.560 --> 00:02:47.800
Um so, we're going to talk first about the seller side, and then we're going to talk about the buyer side.

s45
00:02:47.800 --> 00:02:53.400
So, per pain point, uh a lot of my job is kind of go around the company and figure out like what are the pain points?

s46
00:02:53.400 --> 00:02:55.080
Uh what are we trying to solve for?

s47
00:02:55.080 --> 00:02:55.920
Uh can we buy it?

s48
00:02:55.920 --> 00:02:57.640
Can we build it?

s49
00:02:57.640 --> 00:03:03.440
So, let's say for a given pain point, maybe I identify 10 to 15 startups that look really interesting.

s50
00:03:03.440 --> 00:03:07.239
Like, "Huh, maybe these guys can solve our problem for us, we don't have to build it."

s51
00:03:07.239 --> 00:03:14.519
Um of those, after doing a little bit of due diligence on my own, I might schedule two to three demo calls.

s52
00:03:14.519 --> 00:03:19.200
Of those, we probably will land zero or one pilots.

s53
00:03:19.200 --> 00:03:25.640
And of those, probably one in four of those longer term will actually end up with a contract.

s54
00:03:25.640 --> 00:03:26.280
So, what does this mean?

s55
00:03:26.280 --> 00:03:31.560
This means about 5% of all of our demo calls actually end up in a signed contract.

s56
00:03:31.560 --> 00:03:32.959
Um and this tracks with the industry.

s57
00:03:32.959 --> 00:03:42.480
I had no idea that this was actually a benchmark, um but it turns out that there's quite a bit out there that indicates that this is really similar across the board.

s58
00:03:43.239 --> 00:03:55.680
So, I want to talk about what enterprise ready actually means from the inside, uh because we have a lot of startups that tell me what enterprise ready means, and And we go through all of our requirements, and then we have a very different idea of what enterprise ready actually means.

s59
00:03:55.680 --> 00:03:59.640
Um so we're going to talk about what breaks down and why.

s60
00:03:59.640 --> 00:04:04.800
So 40% of this is efficacy, so just value, uh commercial issues.

s61
00:04:04.800 --> 00:04:06.920
Then there's a lot that dies in security.

s62
00:04:06.920 --> 00:04:12.200
There's other things that die in reliability, and then there's some stuff that dies in legal.

s63
00:04:12.200 --> 00:04:19.480
So we're going to start with the requirements that we put forward and then some of the things that we've seen go wrong across various AI companies that we work with.

s64
00:04:19.480 --> 00:04:24.919
So our requirements for efficacy, maybe unsurprisingly, the product actually needs to solve the problem.

s65
00:04:24.919 --> 00:04:29.080
Um that seems pretty clear, but that's not always super clear.

s66
00:04:29.080 --> 00:04:33.919
Uh the next one is pricing models that need to reflect real value.

s67
00:04:33.919 --> 00:04:38.480
Clear demonstration of integrations on day one, not a hypothetical.

s68
00:04:38.480 --> 00:04:42.760
And we define the success criteria, not the vendor.

s69
00:04:42.760 --> 00:04:45.840
So things we've seen go wrong, uh vaporware in short.

s70
00:04:45.840 --> 00:04:52.560
Um we've had a lot of startups who come in, they pitch us an idea, uh and it's something that our platform team can rebuild in about 6 weeks.

s71
00:04:52.560 --> 00:04:59.960
So this is not a knock, uh this is actually just what's going on in the industry everywhere, um on all sides of the equation.

s72
00:04:59.960 --> 00:05:06.040
Um sometimes it's actually better for us to build, and sometimes it is still better for us to buy even if we could rebuild.

s73
00:05:06.320 --> 00:05:08.280
Upside down pricing, so this one's crazy.

s74
00:05:08.280 --> 00:05:18.960
Um We had a startup just recently tell us, "Hey, um we know that all of the LLM traffic that we're using for our wrapper is passing through your LLM gateway,

s75
00:05:18.960 --> 00:05:29.120
but we want you to report your gateway telemetry to us so that we can then price a huge margin on top of that, even though none of it's running through our infrastructure."

s76
00:05:29.120 --> 00:05:31.640
Um that did not work.

s77
00:05:31.640 --> 00:05:35.640
Another one is promises in demo calls, but no ETAs after 2 months.

s78
00:05:35.640 --> 00:05:36.919
Um this is pretty common.

s79
00:05:36.919 --> 00:05:40.000
Um not a lot to say here.

s80
00:05:40.000 --> 00:05:42.480
Um and then repitching features we've already declined.

s81
00:05:42.480 --> 00:05:47.840
So if you're a salesperson, um my best advice to you is listen to your customers.

s82
00:05:47.840 --> 00:05:51.480
It's not novel, but uh it still seems to be a struggle for some.

s83
00:05:51.480 --> 00:05:54.280
Uh it's really just better to address the things that we've asked for.

s84
00:05:54.280 --> 00:05:58.520
So, the other thing I want to point out at the very bottom of this slide is the pilot window collapsing.

s85
00:05:58.520 --> 00:06:07.520
So, uh I've been at Millennium for a little over 2 years, and when I started, a lot of these pilot timelines that people were used to were like, "Oh, maybe we'll run a pilot for 6 months."

s86
00:06:07.520 --> 00:06:12.280
And then, not that long after that, it was like, "Oh, maybe we only need it for 3 months."

s87
00:06:12.280 --> 00:06:15.840
And anymore, it's like, "Maybe we can do this pilot for 2 weeks."

s88
00:06:15.840 --> 00:06:19.640
Uh because it's just accelerated so rapidly.

s89
00:06:20.000 --> 00:06:21.760
Um so, then moving on to security.

s90
00:06:21.760 --> 00:06:22.600
Uh this is a huge one.

s91
00:06:22.600 --> 00:06:28.040
I'm not a security expert, but I do run kind of frontline defense on talking to a lot of startups about security.

s92
00:06:28.040 --> 00:06:31.400
And so, these are a lot of the things that that come up over and over.

s93
00:06:31.400 --> 00:06:32.280
Uh ZDR.

s94
00:06:32.280 --> 00:06:33.919
So, this is a really hot topic.

s95
00:06:33.919 --> 00:06:42.640
Obviously, a lot going on with Fable, uh mandatory data retention requirements, uh and then a whole other battleground around customer-managed encryption keys.

s96
00:06:42.640 --> 00:06:45.600
So, ZDR is always best, of course.

s97
00:06:45.600 --> 00:06:52.280
If that's not possible, customer-managed encryption keys and, with a big parentheses, that don't break the product.

s98
00:06:52.280 --> 00:06:54.600
Um there are a lot of things that people are like, "Oh, yeah, it's fine.

s99
00:06:54.600 --> 00:06:56.880
It'll work with customer-managed encryption keys."

s100
00:06:56.880 --> 00:06:58.480
And then, it breaks the product.

s101
00:06:58.480 --> 00:07:03.080
Uh so, that's a big product uh issue that we have to work through with people.

s102
00:07:03.080 --> 00:07:05.080
Other requirements, bring your own gateway.

s103
00:07:05.080 --> 00:07:09.560
We prefer to route all of our own traffic through our own gateway and BYO infrastructure.

s104
00:07:09.560 --> 00:07:15.640
Uh we would prefer to host it in our own cloud infrastructure and have something that's deployable in our systems.

s105
00:07:15.720 --> 00:07:16.919
This is another really big one.

s106
00:07:16.919 --> 00:07:18.720
Um SCIM-tied RBAC.

s107
00:07:18.720 --> 00:07:29.360
So, for all of you who who get that jargon, um it's really important that we can tie our AD groups or other permission and entitlement groups to role-based access control.

s108
00:07:29.360 --> 00:07:33.240
We want to make sure that we don't just turn on features for everybody across the board.

s109
00:07:33.240 --> 00:07:35.720
A lot of people don't think about this when they're designing their systems.

s110
00:07:35.720 --> 00:07:37.320
They're like, "Oh, this is a great feature.

s111
00:07:37.320 --> 00:07:39.160
We should just turn it on for everybody."

s112
00:07:39.160 --> 00:07:43.800
Um when you work at a a enterprise, that's not something that people want to do.

s113
00:07:43.800 --> 00:07:51.200
Um there are usually different groups who should have different access at different times, and most of all we want it to be configurable via API.

s114
00:07:51.440 --> 00:07:56.480
Um for smaller companies, we want to see at least one real security hire.

s115
00:07:56.480 --> 00:07:58.160
So, this is something that's really important.

s116
00:07:58.160 --> 00:08:01.720
We know that security is not the first thing that people hire for.

s117
00:08:01.720 --> 00:08:11.040
Um but in the age of AI, this is a very real problem, and we need to make sure that the startups we're working with actually have somebody who can understand what's going on from the security standpoint.

s118
00:08:11.040 --> 00:08:22.120
Uh so, some of the things we've seen go wrong, um outright people just sending data to their vendors, uh cloud servers, and not following any of what we've asked for.

s119
00:08:22.120 --> 00:08:24.240
Um this has been a problem in pilots.

s120
00:08:24.240 --> 00:08:28.400
Uh thankfully, all of our pilots run non-production data.

s121
00:08:28.520 --> 00:08:32.360
Another one, like we kind of talked about, um read write all default scopes.

s122
00:08:32.360 --> 00:08:36.760
So, there's a lot of really cool tools out there, integrations, features.

s123
00:08:36.760 --> 00:08:38.280
They're really flashy.

s124
00:08:38.280 --> 00:08:41.159
You can click a button, and it'll integrate with everything.

s125
00:08:41.159 --> 00:08:49.640
And then you get a little bit deeper and find out the only way that it'll work is if you literally give it read write all to everything, uh which is a huge problem.

s126
00:08:49.640 --> 00:08:55.160
Another one, uh kind of along the same lines, all or new beta features on by default with each release.

s127
00:08:55.160 --> 00:09:00.200
So, if you're an enterprise, you don't want everything just turned on with each release.

s128
00:09:00.200 --> 00:09:08.680
Um so, being able to control that, and then the line that we hear a lot, which is we'll get you the security architecture diagram next week.

s129
00:09:08.680 --> 00:09:12.520
Uh we do weekly check-in calls during a pilot, and then we hear this over and over.

s130
00:09:12.520 --> 00:09:15.600
Uh it's not usually a great sign.

s131
00:09:17.280 --> 00:09:24.280
Uh the question that we often have our CISO end up asking, which is um what are you going to do if there's a breach?

s132
00:09:24.280 --> 00:09:27.520
And we get this response, well, we haven't had a breach yet.

s133
00:09:27.520 --> 00:09:31.640
Uh with the subtext of we don't know what we would do if we did.

s134
00:09:31.640 --> 00:09:33.800
Um okay, reliability, this is another one.

s135
00:09:33.800 --> 00:09:36.240
So, a control plane that actually works.

s136
00:09:36.240 --> 00:09:39.560
We want to see every admin setting available via API.

s137
00:09:39.560 --> 00:09:41.920
We want to see audit logs on config changes.

s138
00:09:41.920 --> 00:09:51.680
So, if there are five different people who are given admin access and somebody accidentally changes something or does it because uh maybe it was really late at night and maybe they had too many drinks.

s139
00:09:51.680 --> 00:09:54.160
Uh we actually want to see what happened.

s140
00:09:54.160 --> 00:09:57.640
Uh we want to be able to control the rollout on these changes.

s141
00:09:57.640 --> 00:10:00.600
Uh we want to see real SLAs and a reachable support engineer.

s142
00:10:00.600 --> 00:10:02.760
That goes a very, very long way.

s143
00:10:02.760 --> 00:10:14.600
So, uh things we've seen go wrong, a lot of apps that are rapidly prototyping, they're shipping so quickly that they are maybe shipping updates multiple times a day and there's a really attractive little button that says relaunch to update

s144
00:10:14.600 --> 00:10:16.640
and it happens across 3,000 people.

s145
00:10:16.640 --> 00:10:18.680
We have no way of tracking what's going wrong.

s146
00:10:18.680 --> 00:10:28.520
Maybe then like SSL certificates break in one of the new releases and then we have no way of tracking because everybody's on a different version um and we have no way of being able to deploy at scale.

s147
00:10:28.520 --> 00:10:30.720
Um that's really challenging.

s148
00:10:30.720 --> 00:10:32.600
No documentation versioning.

s149
00:10:32.600 --> 00:10:42.680
So, support articles with new terms or risks that are not actually in the legal contract but show up in the website somewhere in a random support page and then we have no way of tracking what they were before versus after

s150
00:10:42.680 --> 00:10:44.880
and it just says updated yesterday.

s151
00:10:44.880 --> 00:10:46.320
All of these are real examples, by the way.

s152
00:10:46.320 --> 00:10:47.800
I am not naming and shaming.

s153
00:10:47.800 --> 00:10:49.240
Um I'm just shaming.

s154
00:10:49.240 --> 00:10:53.760
So, uh maybe if any of you are familiar, you can put it together.

s155
00:10:53.760 --> 00:10:57.800
Um core API's down for multiple hours during a busy trading day.

s156
00:10:57.800 --> 00:11:02.040
Uh that is a really big problem for us because we run production systems.

s157
00:11:02.040 --> 00:11:04.440
We are trading billions of dollars.

s158
00:11:04.440 --> 00:11:07.040
Um this is a really big issue for us.

s159
00:11:07.040 --> 00:11:10.320
And then lastly, no SLA roadmap or status page.

s160
00:11:10.320 --> 00:11:12.880
Um the status page is a big one.

s161
00:11:12.880 --> 00:11:14.560
Okay, last, legal issues.

s162
00:11:14.560 --> 00:11:20.440
So, we don't want anybody training on our data regardless of what type of feature or product it is.

s163
00:11:20.440 --> 00:11:24.400
Uh we also want to see a lot of transparency in the sub processors.

s164
00:11:24.400 --> 00:11:28.080
Um any fourth-party risk becomes our risk.

s165
00:11:28.080 --> 00:11:31.560
We want to see IP indemnification with reasonable liability caps.

s166
00:11:31.560 --> 00:11:37.920
Uh we do not control the models, so if there's output that is IP infringing, we don't want to be held liable for it.

s167
00:11:37.920 --> 00:11:48.840
So, we have seen in pilots that people claim they have ZDR, they have it legally, but then they find out or we find out later that they actually retain some of our data because they say, "Hey, we were looking at something and we noticed this thing."

s168
00:11:48.840 --> 00:11:50.520
And we're like, "How did you notice that?

s169
00:11:50.520 --> 00:11:52.360
You weren't supposed to have this data."

s170
00:11:52.360 --> 00:11:54.440
And they're like, "Oh, yeah, you're right."

s171
00:11:54.440 --> 00:11:55.920
Um so, that's not great.

s172
00:11:55.920 --> 00:11:58.560
If you say ZDR, do ZDR.

s173
00:11:58.560 --> 00:12:04.040
Um next, every feature that is conveniently beta with permissive data retention clauses.

s174
00:12:04.040 --> 00:12:10.840
So, we've seen some vendors who they will stop releasing new features in general availability.

s175
00:12:10.840 --> 00:12:19.040
They will only make them beta, and then the beta comes with a secret little clause that says that they're allowed to retain our data, which is a very sneaky way of trying to get our data.

s176
00:12:19.040 --> 00:12:20.320
We don't like that.

s177
00:12:20.320 --> 00:12:22.320
Um not great.

s178
00:12:22.320 --> 00:12:29.160
Another one kind of similar is fourth-party risk that's tucked away on a random website page that's not listed in the contract.

s179
00:12:29.160 --> 00:12:33.760
Uh this is a really big problem for us managing risk.

s180
00:12:34.120 --> 00:12:37.600
So, um it was the best of times, it was the worst of times.

s181
00:12:37.600 --> 00:12:49.160
As a recap, the best startups have security architecture that actually works, support engineers who respond, an admin API from the beginning, a 90-day plan that deploys into our infrastructure and cloud,

s182
00:12:49.160 --> 00:12:51.520
and success criteria that we write.

s183
00:12:51.520 --> 00:13:02.160
The worst AI startups don't have any security architecture diagrams, no path to a support engineer, no deployment control or audit logs, no ETAs, and salesmanship over solid product building.

s184
00:13:02.160 --> 00:13:08.320
Um this is really just kind of a recap of like what I have been through over the last 2 years.

s185
00:13:08.320 --> 00:13:15.560
Um I actually don't think that any of this is novel, um but it is codifying a lot of what I feel like is good and best practice.

s186
00:13:15.560 --> 00:13:22.000
Um okay, so a new frontier model comes out on average every 11 days, but your architecture might be a decade or more old.

s187
00:13:22.000 --> 00:13:28.040
So, you've got a bunch of cool new models, there's some amazing capabilities is there, and then you have profitability on the other side of it.

s188
00:13:28.040 --> 00:13:29.200
And what's in the middle?

s189
00:13:29.200 --> 00:13:35.280
Maybe it's your legacy architecture, probably a lot of security and privacy issues, and a lot of change management.

s190
00:13:35.280 --> 00:13:37.880
Um ChatGPT has only been out for 43 months.

s191
00:13:37.880 --> 00:13:42.480
There are a lot of companies who are still doing an ERP migration that might have been from 5 years ago.

s192
00:13:42.480 --> 00:13:45.600
Um so, the timelines are very asymmetric.

s193
00:13:45.600 --> 00:13:49.120
Uh and I think that sometimes we forget about that.

s194
00:13:49.240 --> 00:13:49.800
Um okay.

s195
00:13:49.800 --> 00:13:57.480
So, my thesis again, half or more of getting to AI native is unsexy and has absolutely nothing to do with AI.

s196
00:13:57.480 --> 00:14:00.920
Um AI models and products today can't fix your legacy architecture.

s197
00:14:00.920 --> 00:14:04.560
Although, if any of you are startup people, that's a great one to go for.

s198
00:14:04.560 --> 00:14:07.120
Um and it also can't run your change management.

s199
00:14:07.120 --> 00:14:15.120
These are unscientific numbers that I'm putting up here, but I hypothesize that 40% of getting to AI native is AI models and products.

s200
00:14:15.120 --> 00:14:25.800
The other 60% is all the other stuff that no one really likes talking about anymore, uh which is like data hygiene, clean architecture, having good integration, strong enablement, and change management.

s201
00:14:26.440 --> 00:14:30.000
Um I really look at AI as a flashlight, not a band-aid.

s202
00:14:30.000 --> 00:14:34.720
Um I really think that AI shines a light on a lot of what's already working or not working.

s203
00:14:34.720 --> 00:14:39.839
It can accelerate what's working really well, and it breaks down very quickly when things don't work well.

s204
00:14:39.839 --> 00:14:48.080
Um I don't think that it's a band-aid, and I think that for everybody who's in tech leadership, it's really important to remember that if you have issues in your technology estate,

s205
00:14:48.080 --> 00:14:53.160
those need to be addressed before trying to plug in AI and just having everything rip.

s206
00:14:53.160 --> 00:14:55.640
Um it's it's not going to work.

s207
00:14:55.640 --> 00:15:03.080
Um so, hehehe again, maybe an unpopular message, but I really believe that we all need to start with the boring 60%.

s208
00:15:03.080 --> 00:15:07.839
I think that's where we all need to start to get to the other side of the road.

s209
00:15:07.839 --> 00:15:12.000
So, what did we learn as we shine the flashlight internally?

s210
00:15:12.000 --> 00:15:16.960
Again, not revealing anything super proprietary, but I do think these are big picture lessons.

s211
00:15:16.960 --> 00:15:18.280
Number one, entitlements.

s212
00:15:18.280 --> 00:15:20.560
Entitlements need a new paradigm.

s213
00:15:20.560 --> 00:15:25.400
Uh there are a lot of people in a lot of large enterprises who are over entitled, under entitled.

s214
00:15:25.400 --> 00:15:35.800
The entitlements model and how it works and how it's managed, all of that breaks down when you think about agents and how quickly you want agents to work and what you want them to work on and their ability to exercise judgment.

s215
00:15:35.800 --> 00:15:38.840
Um the entire paradigm just shifts.

s216
00:15:38.840 --> 00:15:41.760
Another one is cross-platform integration moved up the stack.

s217
00:15:41.760 --> 00:15:46.080
So, AI is only as good as what it reaches and we want it everywhere.

s218
00:15:46.080 --> 00:15:53.000
Uh so, having things that can integrate across platforms is really important uh even more than it already was.

s219
00:15:53.000 --> 00:15:54.440
Another one is centralized knowledge.

s220
00:15:54.440 --> 00:16:03.160
So, this is something that um Emil brought up this morning in his keynote, which is that basically we need thinner agents and a smarter substrate.

s221
00:16:03.160 --> 00:16:05.000
Um centralized knowledge is really key to that.

s222
00:16:05.000 --> 00:16:13.680
So, all of your documentation, all your support articles, everything that's going on inside of your company that's making it work, um all of that needs to be centralized and easily consumable.

s223
00:16:13.680 --> 00:16:17.760
Even better if AI can help write that in real time in a feedback loop.

s224
00:16:17.760 --> 00:16:21.200
Uh that's something we've been talking about with some of our vendors.

s225
00:16:21.200 --> 00:16:24.120
Another one is a separate ecosystem for experimentation.

s226
00:16:24.120 --> 00:16:27.000
Um some companies may need to get here.

s227
00:16:27.000 --> 00:16:39.520
That gap between your legacy architecture and where you want to go might be so vast that you actually just decide, "Hey, maybe we need a separate ecosystem to do a lot of this work, figure out what does work and what doesn't and then kind of go from there."

s228
00:16:39.520 --> 00:16:42.240
Um and that's something that we thought about as well.

s229
00:16:42.240 --> 00:16:48.360
So, to just put a finer point on the agents and the entitlement thing, uh agents inherit your foundations.

s230
00:16:48.360 --> 00:17:01.120
So, I strongly recommend that everybody fix their entitlements if they are not working really well now um because this is something that if you think about the problems that you run into when things go rogue, processes go rogue, people go rogue,

s231
00:17:01.120 --> 00:17:03.839
agents are going to like 100X that problem.

s232
00:17:03.839 --> 00:17:08.360
Um so, this is really something that's worth figuring out now.

s233
00:17:08.760 --> 00:17:20.880
So, to kind of recap uh as I wrap up here, the recipe, if you are one of the people in the first half who are raising your hand on like, "What do I need to do if I'm a startup and I want to work with a really difficult large customer?

s234
00:17:20.880 --> 00:17:23.040
Millennium's got like 8,000 people.

s235
00:17:23.040 --> 00:17:27.439
We have very, very tight security, compliance, regulatory requirements.

s236
00:17:27.439 --> 00:17:30.200
Um this is the stuff that we care about.

s237
00:17:30.200 --> 00:17:35.920
And we want to see more startups doing work that allows us to work with them.

s238
00:17:35.920 --> 00:17:38.960
Um I really view this as like one of the highest bars.

s239
00:17:38.960 --> 00:17:43.120
We're probably not the highest, um although we're probably pretty close.

s240
00:17:43.120 --> 00:17:50.000
Um and I think if you can architect your startup to work with companies like this with this kind of architecture, um you're probably going to be able to satisfy

s241
00:17:50.000 --> 00:17:52.440
basically everybody else.

s242
00:17:52.440 --> 00:18:01.640
Um on the other side for anybody who's buying, uh these are the things that I think again that kind of that boring 60% that really deserves a lot of work.

s243
00:18:01.640 --> 00:18:12.200
Um off entitlements, governance, audit logging, etc. Um these are the things that I think we need to have in terms of systems to get it working on the other side of the equation.

s244
00:18:12.200 --> 00:18:14.080
So, that's it.

s245
00:18:14.080 --> 00:18:21.480
Um my only motivation here is to getting stuff working better and having better enterprise grade AI.

s246
00:18:21.480 --> 00:18:25.440
Uh that's a QR code to my LinkedIn and I appreciate all of your time.

s247
00:18:25.440 --> 00:18:26.962
Thank you.

s248
00:18:26.962 --> 00:18:28.962
[applause]

s249
00:18:40.717 --> 00:18:42.717
[music]