An AI Future Without the Lock-In — Remy Guercio, Tailscale

AI Engineer · 19 min · 194 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Well, hi folks.
00:14Uh, thanks for joining me here for the last talk on this stage uh of the uh AI engineer world's fair this year.
00:23So, um, yeah, thank you for joining me.
00:25My name is Remy.
00:27I am on the strategic projects team at Tailscale.
00:30Uh and more specifically I actually work on our AI gateway aperture and today I want to talk a little bit about you know what it means or you know what it's going to take uh to have an AI future without the lockin.
00:42I think a lot of the locking that we've seen uh over the last 12 months and yeah so if we take a look real quick at the you know the last 12 months you know I think it can really be
00:51you know described quite a bit by uh the phrase token maxing and I I don't mean that in the kind of you know negative sense that you might kind of hear outside of the you know kind of AI enthusiast circle.
01:04I mean that in the sort of you know it's been a year of trying to figure out you know what we can do with million token context windows uh with the ability to get context from anywhere right not just you know MCP being everywhere but you know also having coding tools right that are great at finding context
01:22using the CLI bringing that context into the model uh bringing it you know bringing it into you know everything that you need and and and going from there right it's it's been pretty cool right I mean we've gotten
01:34It's really made truly agentic coding possible having all of this right.
01:38So yeah, it's I mean it's it's quite cool and you know we're going to see uh not just agentic coding but you know other agentic use cases continue to take off
01:46uh as we as we do this.
01:49So um you know token maxing is cool but right it has it has problems right there's obvious problems right it's very expensive right it's it's it's an extremely expensive endeavor
02:02uh you know I think the last 12 months are you know have maybe been marked by hey we have you know unlimited AI budget uh and that's changed here in the last few months
02:12uh in terms of you know what it what it means to you know what it means to use AI um it also I mean you know it's expensive like just generally if you're doing it, you know, right, but it also it does incentivize
02:22what I would call like wasted tokens by default.
02:25I mean, I I work on an AI gateway.
02:27We use our AI gateway.
02:28I get to see all of the usage from the AI gateway.
02:30You would not believe the number of sessions that are hundreds or thousands of messages long where nobody clears the context.
02:38They just kind of keep going.
02:40I mean, it's helpful sometimes, but you know, it's it's it's pretty surprising.
02:44you will have you know tens of millions of tokens of a session uh where it's the cash is basically maxed out maxed out the entire time uh it gets really expensive
02:54um and the other thing is right so you know this has been an exciting time right people are doing new things building new things uh but this has led to
03:01you know in many cases very vertically integrated tool chains uh cloud code is obviously you know a big one codeex right everybody's kind of building up their you know for all the way from the model you know all the way all the way up to the top uh you know building their
03:14uh you know building their agents, building their tool chains.
03:16So um all right there's a lot of problems there and so you know my question right in terms of this is uh you know how do we address these problems in particular how do we address the lock in and you know how do we actually
03:27improve uh how we use AI such that you know we can use it for things that aren't just you know kind of essentially frontier task or where we're just kind of wasting
03:35wasting money wasting tokens uh you know that sort of thing.
03:39Um, but first I want to talk about how not to address the problem.
03:44Uh, and this is a super super common thing.
03:47So, uh, over the last 6 months, we've been talking to I've literally been talking to hundreds and hundreds of folks about their AI spend, their AI usage, right?
03:55That's the kind of benefit of working on an AI gateway.
03:58Folks come to you, they're like, I don't know what's going on.
04:00I know we're using a lot of AI.
04:02I get, you know, uh, I get an anthropic bill every day.
04:05Uh, I get a codeex bill every day.
04:08Um uh so you know I it's not just that you know I see it all the time.
04:12So you know what do I do?
04:13They usually come with one kind of solution in mind uh and that is oh we're just going to like uh everybody seems to be using cloud code right now.
04:20So we're just going to like consolidate on a single vendor maybe two vendors right maybe two vendors.
04:25So that's uh they're like I need to get a handle on this.
04:27How do I get a handle on this?
04:28How I get you know oh okay yeah I will say everybody's going to use this right we're just going to use this thing.
04:33um that is not not a great idea.
04:38uh you know I might seem obvious sitting here it's maybe not a great idea but you know when you're in that panic zone of we've spent a lot of money and
04:46we don't know how exactly uh and we you know finance is talking to you it is being like what are you doing uh um you know like we said to you know use AI but maybe not use that much AI
04:57uh and you know it it's a pretty natural response to be okay I'm going to consolidate everything down right okay I'm gonna have a control on it I'm gonna have you know I'm gonna have control on this uh control of this Um
05:08and you know I think this is kind of peeking at what we're going to see the next 12 months and what people are actually trying to do uh at least at the you know kind of frontier is uh what I would like to call ROI maxing right we're not going to be token maxing we're going to be
05:20ROI maxing uh and you know it's it's really just a necessity right it's not uh um and I don't mean that when I say ROI maxing I don't just mean cutting costs right I mean actually
05:32proving some sort of value uh to the thing that you're doing uh and not only that but also kind of opening things up, right?
05:40So if the only thing you have is like two frontier models and their entire stacked ecosystem, that's a certain cost level where you you it just makes things certain things untenable, right?
05:50You can't actually expand AI usage into the company if that's the only way you can use AI.
05:55Um and so you know when we think about ROI maxing, we first need to kind of think about like what are the components of an internal AI deployment.
06:02I'm not talking about like a hey, we've built this new AI feature in our app.
06:06I mean, you know, internal internal to your organization.
06:09And so, you know, there's four things, right?
06:11There's the LLM or LLMs, right?
06:13There's that layer.
06:14Uh there's sort of the data connector layer.
06:16I mean, this is MCP for a lot of people, but in reality, it's, you know, CLI or APIs or really just any way that you get data could be reading a file, right, into your uh into your agentic system.
06:26Uh there's the interface.
06:27I think people kind of forget about this sometimes, but you know, it's started with the chat interface.
06:32We got two, you know, we kind of got a uh we got, you know, idees with chat.
06:36We got, you know, kind of a whiplash back from two into, you know, I'm going to do it on my go to my phone.
06:42Uh, you know, to, oh, now the, you know, the hot thing is, you know, claude tag and like I'm going to use, you know, I'm going to use we're just going to interface with agents via Slack, right?
06:50So, uh, um, you know, it's another core component and it is a component that you can break out.
06:55Uh and then the last one is sandboxes just the environment in which the agent runs right so those are your four kind of uh you know four components really
07:04uh and to be ROI maxing uh means to be maxing your you know every opportunity that you have uh for choice to make a choice at one of those layers right you don't have to necessarily have a unique thing at you know oh I've made my bespoke
07:19special choice at every one of those layers but it's really important to be able to make a choice at any given layer that you need at any given time.
07:27Uh, and it might be different for different teams or different folks, right?
07:29It's, you know, it can be pretty different across the organization.
07:32I don't totally imagine like an entire engineering team just coding via, you know, slack thread.
07:39Might be fun, but I don't totally imagine that, right?
07:41Uh, um, but, you know, the entire marketing team might be working entirely uh, you know, in a slack thread, asking data questions, doing other things like that.
07:48So it's really important to you know to be able to uh kind of separate these things and right so I I work at tail scale right I work on an AI gateway
07:56uh you know maybe no surprise that I'm here to talk a little bit about why an AI gateway uh will help you do that can help you do that and I don't just mean necessarily using our AI gateway I think it's nice but you know just any AI gateway is the um you know is important for this
08:09uh so our g AI gateway is uh called aperture and um what it does uh is right it's maybe No surprise here, right?
08:18But it let you it sits in between your agents and that interface of that agent uh and you know all of your endpoints uh as well both like LLM endpoints and data you know kind of MCP endpoints API endpoints anything that you have uh and sits in the middle and let you both control and monitor what's happening
08:36from a that can be from a like security perspective that can be from a cost perspective that can be from a which we do a lot I want to learn about
08:45what somebody is doing or what an agent is doing uh perspective.
08:48It's actually really cool when you have logs uh and you can tell your agent to hey uh I it maybe it ran out.
08:55It kind of just kind of you know oh it it didn't reach an answer.
08:58It just kind of spiraled out of control.
09:00Uh maybe happens less now.
09:02Definitely used to happen quite a bit.
09:03Happens with you know maybe cheaper models more often too.
09:06Uh they're usually not great at stopping themselves.
09:09Uh but they are great at analyzing where they went wrong after the fact.
09:12And so when you have something like an AI gateway, it's really easy to point it back at what you were doing and say, "What happened?
09:18Why did this not why did this not, you know, work the way I was expecting it to?"
09:22Uh, and so, you know, if you've used an AI gateway, how many people like have an AI gateway, have deployed an AI gateway, use an AI?
09:29Yeah.
09:29So, if you've used one before, you're probably used to, okay, so I've put all my keys on this gateway.
09:34Now, I need to go create synthetic keys for everything that I have, right?
09:38So, it's like, okay, I've traded some keys for a bunch of other keys.
09:42uh we get to do something a little special.
09:44So Aperture is built on tail scale and if you're not familiar with tail scale, tail scale is an identity based mesh network.
09:51It's built on wire guard.
09:53Uh and what you get to do with tailscale is uh every connection that is made over a tail.
10:01So it's a tail scale network uh you know you install the tail net agent everywhere that you need.
10:05It can be containers, laptops, GPU servers, doesn't matter.
10:09um you install it everywhere and every connection that's now made over that tail net uh comes with the identity of the machine or the user on the other side.
10:18So that can be a tag, right?
10:20So that can be like hey this is a machine ID.
10:21This is a PR review bot, right?
10:23Uh and then it would get a separate node ID underneath.
10:25This is the instantiation of that PR review bot, right?
10:28So you can do that or it can be the user, right?
10:30So it's just me using cloud code on my laptop.
10:32Uh that can come with it as well.
10:34uh in addition to if you're using something like Entra or Octa uh we can actually sync all the groups in uh so you can see the group that that person is associated with.
10:42So something on the other side is able to see all of that.
10:46We have this really cool library.
10:47It's called TSNet.
10:48It's an open source library.
10:49You can use it to build these sorts of apps.
10:51That is what we use to build aperture.
10:53That is what is actually happening here.
10:55So it is able to see the uh it is able to see the identity of that user or that agent on the other side of the thing on the other side of the connection.
11:03And so you know again just reminder components of a you know an internal you know AI deployment uh right so there's the LLM side of things there's the data side of things the interface side of things and the actual sandboxes
11:15uh sandbox and environment in which things run uh side of things.
11:18So, um, when it comes to using Aperture, uh, you know, as far as LLMs go, this is pretty similar, I think, to, you know, what you might expect from other other AI gateways,
11:29except now, you know, here I have a, you know, a configured gateway.
11:32Uh, it's got anthropic, open eye, versel, you know, it's got even anthropic oath, uh, you know, bedrock codex, everything, uh, that you can imagine.
11:40I put all those keys on that gateway and then now when I want to actually connect to that LLM provider as an agent or a user there's no I don't have there's if I had this expanded there's there's no extra screen for create these API keys create these identities create this once I connect to the once I
11:57actually connect to the gateway it knows who I am it knows what I am and it's like oh you have access to this models this provider right you can use GLM 5.2
12:06uh you get this budget with GLM 5.2 too, right?
12:09You get unlimited budget with a cheap model and you get an extremely narrow budget with a Frontier model, right?
12:14Those sorts of things.
12:15Uh there's no extra setting here.
12:17It's just add your add your API key and go.
12:20So again, we've taken all the LLM layer and we've put it into a you know, put it into one place that you can then, you know, kind of expand out within your organization.
12:28Again, doesn't matter if you're trying to switch models or providers, you can do that.
12:32Uh the next thing is the data side of things.
12:34So, uh, in addition to, you know, being an an LLM gateway, and this is actually why I said AI gateway at the beginning.
12:41So, I know that seems kind of nebulous, uh, but I'm actually I very specifically chose that term because we're both an LLM gateway and an MCP gateway and we even do off like, uh, injection for just standard APIs and reexpose those as MCPs, too.
12:56So, uh, you know, kind of intentionally choosing the broader term here.
13:00Um, but with this, as you can see here, uh, in Aperture as an admin, you can go in and configure whatever MCP servers you want.
13:07We have kind of built-in ones where we've, you know, checked them out, make sure they're easy to easy to work with, all those sort of things.
13:12If you've ever deployed an MCP or used MCPS within your organization, you'll know that, uh, there's a standard, but everybody follows a different version of that standard.
13:23Uh, you know, you can tell exactly when an MCP server was built by exactly how they decided to do off.
13:28Uh yeah, it's it's a great uh you know kind of archaeology lesson uh when you go through when you go through setting up MCP servers, but we try to make that a very simple process here, right?
13:38We've kind of gone through this.
13:39Uh so you as an admin can say, "Hey, I've configured these.
13:41These are the ones that are allowed."
13:43You can use that same identity information to say, "Hey, the marketing team, you know, they get access to Google Calendar and Gmail, but the engineering team, they'll also get access to GitHub,
13:51right?
13:52uh you can even say things like oh the engineering you know this side of the engineering team gets access right access to GitHub or readonly access to GitHub you can do all all sorts of things here and what's really nice and again talking about decoupling
14:06is all of the authentication happens here so if you have folks that want to use cursor or they want to use cloud code or they want to use codeex or they want to use some special agent framework that they're you know some new hot thing that they've you know they've discovered or built or whatever
14:21uh all they have to do is hit that endpoint and if they're on the tail net everything's authenticated as them or as whatever the agent or as whatever you know the tag device is.
14:31Uh there's no like oh I have to go reth into everything and then reauth into everything and then reth into this new tool.
14:36Uh we make the switching cost right of like actually trying out tools and trying different workflows and different experiences a lot a lot easier with this sort of thing.
14:44It's all those little points of friction uh that lead to use cases where like you know people just don't try new use cases.
14:50They don't try to uh you know oh maybe I should experiment maybe I can make this cheaper maybe I can make this faster.
14:57Um you know next thing talking about the interface so uh with Aperture we actually have a built-in chat UI surprise uh you know everybody seems to have a chat UI we have a built built-in chat UI
15:07um however it is built on top of APIs that you can also leverage yourself.
15:12Uh so our whole goal with Aperture is to actually provide you with the same sort of you know tools and interfaces and or tools to build interfaces that we can.
15:20So whether or not you want to build a Slackbot or you want to build uh chat UI or you want to build you know some new thing that nobody's thought of some mobile app kind of thing you know it's voice only interface right
15:30uh you can you can do that.
15:32The whole point is you're going to be able to do that with Aperture.
15:34We want to support all of those different individual uh use cases.
15:38But again you can see it has connectors.
15:40Those are all again set up through Aperture itself.
15:43They're not set up in the chat UI.
15:44You're just getting them uh by way by way of Aperture.
15:49Uh and then the last thing, so this is uh you know, so we talked about interfaces, you know, a little bit.
15:54It's, you know, it's again, it's very important uh that you be able to kind of use all of that.
15:57Uh actually one other thing that's coming very soon here is um uh if you've ever I don't know if anybody's ever tried to set up like open web UI or
16:05you know or like Libra like or any of the like kind of like any of the like oh I'm going to set up like a hey this like you know this interface or this interface
16:13uh one big thing is you always lose identity information and then suddenly all of the you know everything's coming from the UI right everything's it's like oh everything's just coming from the system uh we're actually going to be adding a full like pass through O
16:24uh so you'll be able to actually see all of the like user identities and machine identities if for some reason they're using whatever interface that you have.
16:32Uh you should be able to set it up so that you can pass through the O and you still get all of the same visibility that you're kind of expecting from a from an AI gateway as well.
16:40Um and then yeah the last thing is right this is actually the I think the area in which um folks are doing pretty good.
16:46There's actually a lot of choice already people are trying to do this.
16:49Uh I think that's because uh the definition of a sandbox I'm sure I could poll everybody here and would have a different everybody would have a different definition of exactly what a sandbox is
17:00uh or what they need from a sandbox or what they want from a sandbox.
17:03Uh and so you know there's we're kind of in that let a thousand flowers bloom phase of you know there's a sandbox provider for every sort of whim and need and you know and want out there.
17:14Um but uh you know and folks are doing a pretty good job of integrating them but again it's a really important thing.
17:18We're actually going to be adding kind of an orchestration layer as well for sandboxes to a gateway.
17:24uh again trying to use that term AI gateway very loosely intentionally very loosely because in order to again kind of make the best choices for all of the different endpoints that you're you know environments LLMs
17:35data connectors all that sort of thing uh you really do need uh to kind of have a place to sort of see them all uh and then a place that brings them all together such that you can then use any other one so right kind of a almost a universal translation layer in a way
17:48um but yeah so that's that's mostly that and you know like I said is a reminder here uh is you know ROI maxing really is all about maximizing choice
17:58uh to prevent lock in right so you know even if you don't use Aperture I do highly recommend you start considering an AI gateway you look at an AI gateway
18:07it's good just beyond like actual you know oh you know I don't have to you know uh have three different providers configured or switch between them this way you know uh it's good beyond that right it's great for just you know understanding what agents are doing even if it's just you as an individual
18:21we have so many people who or just as an individual they have their usage behind an AI gateway just to have an idea as to what their agents are doing because you know if you're running
18:32uh very commonly uh is people are running something like openclaw with on tail scale and it's on their tail net and they and it's just running in the background and they're like oh my god it did something I'm not sure what it did it did something it was not supposed to do with something like an AI gateway
18:47you can actually go back and you can see and you can look and you can actually diagnose where it went wrong what happened Um so you know it's it's not just lock in right it's actually just experimentation
18:56and understanding as a whole uh is you know is very important to use something uh like a like an AI or LLM gateway.
19:02So um that's it uh for my talk just short and sweet uh thank you very much for attending I this is the the last talk in here of the day so
19:11uh I hope everybody's had a really great conference I know I love the AI engineer world fair uh and I hope you do too.
19:17So thank you