WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=lMCxVorb9wM NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/lMCxVorb9wM.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:00.000 --> 00:00:02.000 Hi, thank you so much for tuning in. s2 00:00:02.000 --> 00:00:04.960 I'm Ravi, I'm one of the co-founders of Scale Grid. s3 00:00:04.960 --> 00:00:17.800 Today, I'm going to talk about how you need to think architecturally from the ground up about building your applications APIs, your MCP servers for agents, and how the human-focused s4 00:00:17.800 --> 00:00:20.360 architecture doesn't scale well for agents. s5 00:00:20.360 --> 00:00:33.760 So, a while back, we were looking at our performance and latency numbers, and one thing that kind of jumped out at us was how our latency was spiking every 15 minutes in a rhythmic manner. s6 00:00:33.760 --> 00:00:36.800 Nothing harmful, but just a curious thing for us to analyze. s7 00:00:36.800 --> 00:00:39.120 What we noticed was very interesting. s8 00:00:39.120 --> 00:00:47.480 So, in our identity and authentication infrastructure platform, we have this little timestamp that we mark for every user to say, "Hey, when was the user last seen?" s9 00:00:47.480 --> 00:00:51.840 or "When was the user last active or last acted in our system?" s10 00:00:51.840 --> 00:00:55.840 so that we can predictively say, "Hey, this user is an active user. s11 00:00:55.840 --> 00:00:57.280 This user is not so active." s12 00:00:57.280 --> 00:01:08.640 But, one thing that we realized was this system was primarily built for humans, but when agents started hitting our APIs in the last 12 months or so, we realized that s13 00:01:08.640 --> 00:01:19.520 this last seen update is happening 60 times faster than what it would, and that is creating unnecessary pressure in our DB write system. s14 00:01:19.520 --> 00:01:22.320 So, of course, it's a harmless thing. s15 00:01:22.320 --> 00:01:24.240 We were able to fix it very easily. s16 00:01:24.240 --> 00:01:30.440 We would just batch the update at a second level and not at every single time we had to update it. s17 00:01:30.440 --> 00:01:32.120 That kind of took us down a rabbit hole. s18 00:01:32.120 --> 00:01:39.200 So, the assumption that broke was how often would our system have to update this timestamp on every row, and that's okay. s19 00:01:39.200 --> 00:01:51.280 It's just about speed, it's about latency, etc. But, what I was worried about is, "Hey, what if some of our assumptions that we made about authentication, authorization need to be rewired and rethought completely s20 00:01:51.280 --> 00:01:56.760 when it comes to agents because we would have designed earlier for humans as actors in mind?" s21 00:01:56.760 --> 00:02:10.240 Now, just to give you a context, I worked on identity and authentication operations for the last 10 years building an identity platform at Freshworks, which is being used by millions of daily users, hundreds and thousands of customers all over the world. s22 00:02:10.240 --> 00:02:12.520 But, this is predominantly human users, right? s23 00:02:12.520 --> 00:02:15.440 Or at best, APIs. s24 00:02:15.440 --> 00:02:22.040 But, the way I think about it is APIs are also accessed by machines that are written by humans. s25 00:02:22.040 --> 00:02:24.080 That's not too bad, right? s26 00:02:24.080 --> 00:02:32.400 But, what I realized is the fundamental picture has changed drastically in the last three, four years or so. s27 00:02:32.400 --> 00:02:46.520 We have a unique ringside view to see how developers nowadays are building agents and how they're giving context to these agents with data from third-party applications like Salesforce or or Databricks or HubSpot or Notion. s28 00:02:46.520 --> 00:03:00.280 What we have realized is most of the agents our customers are building have way too permissions and scopes than the agent's responsibility or the agent's job is. s29 00:03:00.280 --> 00:03:09.952 Again, it's not because the developers who are building the agents are careless, but somehow this became a default pattern of giving s30 00:03:09.952 --> 00:03:10.200 [snorts] s31 00:03:10.200 --> 00:03:18.920 the agents what they need access to, and the existing primitives that we have don't let us give extremely fine-grained permissions to the agents. s32 00:03:18.920 --> 00:03:21.680 Now, I'll tell you how we ended up here, right? s33 00:03:21.680 --> 00:03:27.920 We predominantly have two slots, and neither of the slots was built for agents in mind. s34 00:03:27.920 --> 00:03:39.200 There's a human who's acting the application, either a web application or a mobile application, or their own little script that they wrote, and they give it their API key so that their program can access data from the application. s35 00:03:39.200 --> 00:03:48.680 This is all the fundamental principle here is it's the same user who is authenticating, and it's the same user who's acting, right? s36 00:03:48.680 --> 00:03:59.760 And the second slot is the traditional service account scenario or end-to-end account scenario where you create a service account, you give it certain permissions, and then say this machine has its own identity. s37 00:03:59.760 --> 00:04:09.560 That's where the likes of SPIFFE and and OAuth and all of that came into picture, but you would give them certain credentials and say, "Hey, now this machine has access to s38 00:04:09.560 --> 00:04:12.840 whatever data that it needs at any single point of time." s39 00:04:12.840 --> 00:04:15.480 And this is the existing pattern, right? s40 00:04:15.480 --> 00:04:21.640 So, the fundamental philosophy that we've always maintained is whoever is authenticating is the one that is acting. s41 00:04:21.640 --> 00:04:30.120 Every action the program or the human takes is based on fixed set of permissions that actor was granted at some time. s42 00:04:30.120 --> 00:04:40.840 If you take traditional authentication mechanisms for humans, including password, you just say, "Hey, if an identity has the same password that it was set at the time of registration, if they come back s43 00:04:40.840 --> 00:04:45.800 and if they present the same password again, then you say, "Okay, this is how I validate the identity. s44 00:04:45.800 --> 00:04:48.000 This is how I authenticate the human." s45 00:04:48.000 --> 00:04:51.760 And every action subsequently is tied to that human identity. s46 00:04:51.760 --> 00:04:58.880 Again, the same is the case with API key or the same is the case with web session tokens or even the same case for service account. s47 00:04:58.880 --> 00:05:08.680 You define the permissions at the time of registration, and then every single time it acts based on the registration time permissions and scopes. s48 00:05:09.080 --> 00:05:25.840 Now, this is okay all this while because for decades the service account and OAuth principle even is working fine even though there are their own problems, but it's still working fine because these machines are s49 00:05:25.840 --> 00:05:33.680 using a program in a deterministic way by the way the human developer wrote that program. s50 00:05:33.680 --> 00:05:44.400 So, there is absolute guarantees about what the program could or what the program won't do, but it is still intentional based on what the human wrote, right? s51 00:05:44.400 --> 00:05:57.200 In this particular case, again, if it is using API keys, then that actor and the principal is the same, then there is some sort of a delegated permission for the program to act based on what consent the user has granted. s52 00:05:57.200 --> 00:06:04.200 But, the second one is the most important part, which is it's a deterministic program, and it always stays in its own lane. s53 00:06:04.200 --> 00:06:07.800 It can never do what it was not programmed to do. s54 00:06:07.800 --> 00:06:13.280 And you could inspect the code to say, "Okay, is the program doing what it's supposed to do?" s55 00:06:13.280 --> 00:06:19.760 Even if you apply for a Google developer account, and then ask for client ID, and you need to access these scopes, you have to go through a security review. s56 00:06:19.760 --> 00:06:23.520 So, what they're doing is they're looking at your code base to see, "Are you doing enough checks? s57 00:06:23.520 --> 00:06:25.000 The appropriate practices in place?" s58 00:06:25.000 --> 00:06:27.160 So, these programs are deterministic. s59 00:06:27.160 --> 00:06:32.040 These programs behave the exact same way a developer programmed them to work. s60 00:06:32.040 --> 00:06:35.800 But, agents fundamentally break this assumption. s61 00:06:35.800 --> 00:06:36.200 Right? s62 00:06:36.200 --> 00:06:40.280 First of all, in the case of agents, the principal is not the same as an actor. s63 00:06:40.280 --> 00:06:45.440 You need to give delegated access, so that the agent can act on behalf of the user. s64 00:06:45.440 --> 00:06:47.760 Agent can access the user's Gmail. s65 00:06:47.760 --> 00:06:51.800 Agent can access the user's Salesforce data, and whatever the case may be. s66 00:06:51.800 --> 00:06:57.680 But, again, unfortunately, not a lot of systems, even today, support OAuth. s67 00:06:57.680 --> 00:07:01.120 So, here again, there is no on behalf of principal that is working. s68 00:07:01.120 --> 00:07:07.919 So, you don't even know if there is a program that is acting on behalf of the user, or the user acting by themselves. s69 00:07:07.919 --> 00:07:08.280 Right? s70 00:07:08.280 --> 00:07:09.680 That's a fundamental problem. s71 00:07:09.680 --> 00:07:16.280 The second problem is even more dangerous, which is right now, the program is not written by human. s72 00:07:16.280 --> 00:07:22.080 There is no determinism baked in to say what the agent will do or won't do. s73 00:07:22.080 --> 00:07:22.280 Right? s74 00:07:22.280 --> 00:07:33.280 Just because an agent does certain things today, you can't be 100% certain that the agent can't do the same or the exact same thing tomorrow, day after, or even if it's the next immediate run. s75 00:07:33.280 --> 00:07:33.760 Right? s76 00:07:33.760 --> 00:07:42.360 Because of this non-deterministic nature of agent, we usually pick one of the two lanes, right? s77 00:07:42.360 --> 00:07:50.280 You give a specific identity to the agent, which is what we call as client ID in the context of OAuth, and then say, you act on behalf of this particular user, s78 00:07:50.280 --> 00:07:57.080 or we go back to the agent acts as the user, which is even worse. s79 00:07:57.080 --> 00:08:08.960 The fundamental reason why I'm harping on the same thing is because when an agent is acting on behalf of user one versus user two or user three, the agent needs s80 00:08:08.960 --> 00:08:12.840 specific permissions based on the user's context. s81 00:08:12.840 --> 00:08:22.960 Now, the OAuth solved this perfectly fine by saying, hey, the user will grant specific scopes or permissions to the agent. s82 00:08:22.960 --> 00:08:30.280 So, when the agent is acting on behalf of the user, it can't do everything, but the agent will only do certain things. s83 00:08:30.280 --> 00:08:43.360 Now, in the kind of the world that we're living in, most of the MCP servers that we've worked with don't actually limit the tool context access to the agent based on which user authorized the agent. s84 00:08:43.360 --> 00:08:57.200 They typically surface all the tools that the user has access to, or all the tools that the application can even support, and then let the agent determine what they are can or cannot do. s85 00:08:57.200 --> 00:09:00.680 And now the agent ends up picking the wrong tool, doing wrong things. s86 00:09:00.680 --> 00:09:10.000 Maybe there is some runtime check in the application that prevents some of these things, but the agent is still seeing the same surface regardless whom it is acting for. s87 00:09:10.000 --> 00:09:12.000 Now, two things that we need to solve for. s88 00:09:12.000 --> 00:09:18.280 One is the actor, in this case, an agent, has to be bound to the principal at all times. s89 00:09:18.280 --> 00:09:21.160 And the agent should have its own identity. s90 00:09:21.160 --> 00:09:26.680 Agent should have extremely fine-grained credentials, not the OAuth scopes that we are seeing today. s91 00:09:26.680 --> 00:09:36.560 If you inspect the scopes for some of these applications, like even very popular applications like Gmail, it'll say, can this client send emails on your behalf? s92 00:09:36.560 --> 00:09:41.400 There's no extremely fine-grained scoping to say can this agent act at this hour? s93 00:09:41.400 --> 00:09:45.280 Can this agent read emails only from these senders? s94 00:09:45.280 --> 00:09:48.520 Can this agent send emails to only this recipients? s95 00:09:48.520 --> 00:10:04.560 The reason why that is important is because again, we spoke about this earlier in the context of non-deterministic agent workflows, it's extremely important that the agent should have permissions for limited amount of time that they're operating in number one. s96 00:10:04.560 --> 00:10:06.720 Every agent has a goal, every agent has a job. s97 00:10:06.720 --> 00:10:15.680 So, you should be able to deterministically say that this agent should have access only to those tools or only to those jobs it has access to. s98 00:10:15.680 --> 00:10:30.120 So, gone are the days when the broad scoped auth scopes that we defined is okay because in that case developer was writing a deterministic application and you can review the code to make sure that he's not doing anything sinister. s99 00:10:30.120 --> 00:10:33.880 But in the case of agents is extremely non-deterministic, it is probabilistic. s100 00:10:33.880 --> 00:10:37.560 Agents are bound to do things whatever they can get a hold of. s101 00:10:37.560 --> 00:10:46.480 So, in the context of when you're giving access to the agents, you should be in a position to give extremely fine-grained scopes. s102 00:10:46.480 --> 00:10:51.920 It should be at an attribute level scoping, it should be context level scoping, it should be principal level scoping. s103 00:10:51.920 --> 00:10:54.160 So, all of that is extremely important. s104 00:10:54.160 --> 00:11:05.120 Again, I think everyone agrees that agents should be least privileged by default and they should be able to ask for just-in-time authorization if they want elevated scopes. s105 00:11:05.120 --> 00:11:08.720 Now, the reason why we're talking about this is because it's not some futuristic thing. s106 00:11:08.720 --> 00:11:09.520 It is happening today. s107 00:11:09.520 --> 00:11:13.600 We have seen enough incidents where agents end up doing rogue things. s108 00:11:13.600 --> 00:11:15.720 They end up deleting production databases and stuff like that. s109 00:11:15.720 --> 00:11:23.320 So, how do you put deterministic guardrails in place is an important problem to be solved right now. s110 00:11:23.320 --> 00:11:27.640 Again, one of our customers, ref.tools, they don't even have humans as actors. s111 00:11:27.640 --> 00:11:33.760 Their predominant product is about how to give context to coding agents so that they can do their job effectively. s112 00:11:33.760 --> 00:11:35.840 So, they built the entire OAuth scoping. s113 00:11:35.840 --> 00:11:37.880 How do you do things the right way and things like that. s114 00:11:37.880 --> 00:11:44.400 So, the reason why I give this example is not to say that this is a warning shot, but this is a problem of today and not for tomorrow. s115 00:11:44.400 --> 00:11:57.720 Before I go, you have to have absolute visibility into what your agent can do, every action that's taken in your system, who took it, on behalf of whom, and who authorized it, when was the authorization s116 00:11:57.720 --> 00:12:03.760 given, what authorization was given, how long is it given for. s117 00:12:03.760 --> 00:12:16.600 If you don't have visibility into all these actions at every single time, and if you can't deterministically control what your agent can or cannot do, then you're just praying that agent doesn't end up doing what it's not supposed to do. s118 00:12:16.600 --> 00:12:19.440 And praying is not a strategy, as we all know. s119 00:12:19.440 --> 00:12:20.520 One last thing to take away. s120 00:12:20.520 --> 00:12:33.880 If you architected so far with humans and APIs in mind, you need to start rethinking about how you need to give deterministic guardrails and deterministic authorization controls to the agent. s121 00:12:33.880 --> 00:12:46.839 And OAuth is a good place to start, but you need something beyond OAuth to make sure that the agents have extremely fine-grained access controls, and agents are always acting by themselves on behalf of certain users. s122 00:12:46.839 --> 00:12:49.480 Thank you so much for your time.