WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=3wj6sgbi1YA NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/3wj6sgbi1YA.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:12.559 --> 00:00:13.599 Hello everybody. s3 00:00:13.599 --> 00:00:15.120 Uh my name is Pedro. s4 00:00:15.120 --> 00:00:18.000 I'm a software engineer at Airbet. s5 00:00:18.000 --> 00:00:23.840 And today we're going to be talking about how we built our agent MCP and our CLI. s6 00:00:23.840 --> 00:00:35.280 Um, and before we go into the nitty-gritty of all of that, um, I wanted to give a bit of context on what we do at Airbet and how all of this kind of fits into, s7 00:00:35.280 --> 00:00:36.079 uh, our product. s8 00:00:36.079 --> 00:00:38.160 So, Airbite's been around for a while. s9 00:00:38.160 --> 00:00:42.399 Um, we started as the open- source uh, standard for data movement. s10 00:00:42.399 --> 00:00:51.840 Um, connecting all of your SAS tools that you might be using like Zenesk, Stripe, uh, GitHub, etc. to um, your data analytics pipelines. s11 00:00:51.840 --> 00:01:00.000 And now we're using that same expertise and have evolved to being the data and action layer for your AI agents. s12 00:01:00.000 --> 00:01:13.840 Um, and we do that by offering a or agent product um that has what we call the context store which connects to all of those tools that that I mentioned before and sits in between your agents s13 00:01:13.840 --> 00:01:22.799 um to provide a search optimized index of all of your data um that is able to be more efficiently queried by your agents. s14 00:01:22.799 --> 00:01:36.799 So, you're able to gather information about all of these um records and and and contacts and stuff that you might have in these all all these tools and join them together in an easy way uh through our contact store. s15 00:01:36.799 --> 00:01:42.400 And so, our goal is that we want to make data available and actionable everywhere. s16 00:01:42.400 --> 00:01:47.759 And so, we have a couple different interfaces by which you can access this context store. s17 00:01:47.759 --> 00:01:54.799 Um, one is our SDK, which is more usable if you are uh building your own agents and your tools. s18 00:01:54.799 --> 00:02:00.399 Um, our UI, which is, um, just a quick way that you can use it through our web app. s19 00:02:00.399 --> 00:02:09.360 Uh, and then what we're going to be focusing on in this talk, which is our MCP and our CLI, which is um, a way to easily make that um, available s20 00:02:09.360 --> 00:02:14.239 within existing uh, agents like Chad GBT um, and and Claude. s21 00:02:14.239 --> 00:02:18.640 So what exactly are we exposing through these interfaces? s22 00:02:18.640 --> 00:02:20.160 Um three main things. s23 00:02:20.160 --> 00:02:28.000 One is the ability to manage all of the the core resources of our product to our organizations workspaces and connectors. s24 00:02:28.000 --> 00:02:32.239 Um and so all these interfaces should let you uh interact with that. s25 00:02:32.239 --> 00:02:43.360 Second um super important is be able to authenticate those thirdparty services that I mentioned about earlier um and do so in a way that's secure and doesn't expose your credentials s26 00:02:43.360 --> 00:02:45.519 uh to to the AI agents. s27 00:02:45.519 --> 00:02:47.519 And third um ask and act. s28 00:02:47.519 --> 00:02:56.319 So we want to expose the ability to search across the context store uh be able to read and write uh take action on all those systems. s29 00:02:56.319 --> 00:03:03.519 Um, and so we want all of our interfaces to be able to span across these uh three capabilities. s30 00:03:03.680 --> 00:03:15.360 Um, so the MCP and the CLI, the way the way that we've structured things is that they are thin unified interfaces into our platform, not reimplementations of the platform. s31 00:03:15.360 --> 00:03:25.440 So most of the surface layer of these uh interfaces are presentational where you take a command uh and you call the platform and format the results in some way. s32 00:03:25.440 --> 00:03:30.159 Uh and everything kind of sits on top of our shared platform and APIs. s33 00:03:30.159 --> 00:03:40.480 Um and because we want to we deploy a lot, you know, our our platform changes like over 10 times a day, we want to keep these things up to date as we go. s34 00:03:40.480 --> 00:03:49.440 And so a big part of this as well is open a open API uh specs as a source of truth for how to connect to to our APIs. s35 00:03:49.440 --> 00:03:55.519 And a lot of those interfaces um are built on top of uh those specs. s36 00:03:56.400 --> 00:03:58.959 Okay, so let's go into these interfaces. s37 00:03:58.959 --> 00:04:01.680 Let's start with let's start with the the MCP. s38 00:04:01.680 --> 00:04:06.480 And so MCPs are really great for users that might not be technical. s39 00:04:06.480 --> 00:04:14.319 Um, and in our case, it's great for those users that just want to plug their data straight into Chat DBT or Claude in an easy way. s40 00:04:14.319 --> 00:04:17.919 Um, and the cool thing is they're they're very nicely packaged. s41 00:04:17.919 --> 00:04:20.079 They're very uh easy to install. s42 00:04:20.079 --> 00:04:24.320 For example, here's our um listing on the Chad GBT app. s43 00:04:24.320 --> 00:04:27.919 uh listing the official uh store that they have. s44 00:04:27.919 --> 00:04:38.240 Um and we also have a application for the the anthropic one uh which we're still pending on if there's anybody from Enthropic out there that wants to approve us. s45 00:04:38.240 --> 00:04:46.560 Um but even if you're not on these marketplaces, um they are pretty trivial to install uh for a an end user. s46 00:04:46.560 --> 00:04:53.759 You just give it the hosted MCP URL and you're able to um connect pretty quickly. s47 00:04:53.919 --> 00:04:56.479 Okay, so let's look at uh an example. s48 00:04:56.479 --> 00:04:58.639 This is our MCP in action. s49 00:04:58.639 --> 00:05:02.240 Here I'm asking, you know, which connectors do I have available? s50 00:05:02.240 --> 00:05:07.280 Um, and it goes through, hey, I have four connectors set up in this workspace. s51 00:05:07.280 --> 00:05:13.840 And then I can ask it some questions about my data like show five paying customers with Zenes support tickets. s52 00:05:13.840 --> 00:05:22.800 Um, and so this right now is looking through all the connectors that we have, searching the context store for that information that I that I asked it about. s53 00:05:22.800 --> 00:05:30.960 Um, and then combining that information across Zenesk and Stripe, um, so that it can come up with an answer for me. s54 00:05:30.960 --> 00:05:31.120 Okay. s55 00:05:31.120 --> 00:05:31.840 And there we go. s56 00:05:31.840 --> 00:05:39.840 So it cross referenced Zenesk and Stripe and then came back with, you know, five of those people that um, have open support tickets. s57 00:05:39.840 --> 00:05:40.160 Cool. s58 00:05:40.160 --> 00:05:49.440 So let's go into kind of some lessons learned while building this uh MCP and things that you might want to keep in mind if you're doing uh something similar. s59 00:05:49.440 --> 00:05:55.120 So one thing that we decided to do um after trial and error is to keep the tool count small. s60 00:05:55.120 --> 00:06:05.280 So um we for the execution part of the connector where you're pulling that data, we only expose or really deal with two main tools. s61 00:06:05.280 --> 00:06:14.400 one is to describe the connector to get the schema of how do you query that information and two is the execute that's able to read write and delete records. s62 00:06:14.400 --> 00:06:26.080 Um the opposite end of the spectrum here would be since we connect to so many different um connectors and each of these has different entities different actions you can take. s63 00:06:26.080 --> 00:06:29.919 We could have exposed, you know, an MCP tool for each one of these. s64 00:06:29.919 --> 00:06:39.039 Um, but instead what we found is really keeping it narrow, using progressive discovery of all of the capabilities um that the connector has available. s65 00:06:39.039 --> 00:06:52.400 Um, and really limiting the amount of context that you shove into the those tool descriptions um really helps with the the performance of the of the MCP server. s66 00:06:52.400 --> 00:06:54.000 Two, Olaf is the way. s67 00:06:54.000 --> 00:06:59.199 So this this refers to how you authenticate the MCP server itself. s68 00:06:59.199 --> 00:07:02.560 Um so this is an example for like anthropic. s69 00:07:02.560 --> 00:07:06.800 You click a button uh to connect and that spins up our ooth flow. s70 00:07:06.800 --> 00:07:14.960 So you can sign in to air bite um accept the the privileges and then you're all set up with all the connectors that you've uh linked up. s71 00:07:14.960 --> 00:07:31.360 Um, and technically you can use other authentication mechanisms through MCP, but you're really going to have a hard time um if you do like beer off or or anything like that because the spec is really heavy on ooth and all of the clients, the the main ones at least s72 00:07:31.360 --> 00:07:38.000 um are really leaning towards the the OAF as a better UX for uh installing that connector. s73 00:07:38.000 --> 00:07:50.960 So that's an example for uh the the enthropic um custom connector um registration where they have no no way of you to put like a bearer token or anything. s74 00:07:50.960 --> 00:07:52.960 It's just all defaulting to to OOTH. s75 00:07:52.960 --> 00:07:55.599 So that's is is very important. s76 00:07:55.599 --> 00:08:01.919 And two, users actually expect um to have long lived sessions uh through these MCP servers. s77 00:08:01.919 --> 00:08:14.879 Um unlike a web app where you might be okay with having to sign in again um a couple times, uh users expect to use MCPs more like a mobile app where they don't really have to think about authenticating. s78 00:08:14.879 --> 00:08:17.120 It's just available uh whenever they want to use it. s79 00:08:17.120 --> 00:08:33.360 And so that has its own little um you know uh things to keep in mind as you're implementing ooth is keeping offline long sessions um that are stored separately from the web app and have its own set of uh of life cycles. s80 00:08:33.440 --> 00:08:33.839 Cool. s81 00:08:33.839 --> 00:08:35.919 Elicitation though is not the way. s82 00:08:35.919 --> 00:08:46.399 So I'll get into what elicitation is but a big part of um the MCP server that I mentioned is you need to be able to connect your tools. s83 00:08:46.399 --> 00:08:49.200 you need to be able to give credentials in a way that's safe. s84 00:08:49.200 --> 00:08:54.959 Um, and to do that, uh, basically when you ask, you know, how do I connect to to HubSpot? s85 00:08:54.959 --> 00:09:07.440 Um, we generate a link that you go to which you can pops up this widget and you're able to select, you know, which uh entities you want to connect and redirect to to HubSpot s86 00:09:07.440 --> 00:09:09.920 to go through the that OOTH flow. s87 00:09:09.920 --> 00:09:22.800 Um, now technically MCP has something in the spec that allows you to surface those kinds of URLs um and and process those inputs um called elicitation. s88 00:09:22.800 --> 00:09:25.680 It's a way that you can request information from the user. s89 00:09:25.680 --> 00:09:37.360 For the longest time um it was only form mode elicitation which is not um applicable for sensitive um sensitive uh data like credentials for an API. s90 00:09:37.360 --> 00:09:44.160 But more recently um this URL mode elicitation became a thing um that is very similar to what we're doing. s91 00:09:44.160 --> 00:09:48.000 But the problem is uh support across the MCP clients. s92 00:09:48.000 --> 00:09:58.480 So that's a big theme uh when working with MCP is that even though there is a spec features that is are supported across the clients kind of vary quite a bit. s93 00:09:58.480 --> 00:10:04.959 Um so this is an example from cloud desktop where you know it just is telling me hey I don't support URL mode elicitation. s94 00:10:04.959 --> 00:10:05.760 I can't do that. s95 00:10:05.760 --> 00:10:12.720 And so we had to go back to doing our own thing um and not using the spec for that for that kind of thing. s96 00:10:12.720 --> 00:10:15.600 Okay, let's talk about the CLI a bit. s97 00:10:15.600 --> 00:10:27.680 So if MCPS are easy for these non-technical users, um CLI are kind of the more technical counterpart might call it the kind of more powerful in some ways that we'll get into. s98 00:10:27.680 --> 00:10:36.880 And this is the the same example we saw earlier, but this time with the the MCP or sorry, with the CLI where we're asking, you know, which connectors do I have available? s99 00:10:36.880 --> 00:10:41.040 It's this time loading a skill that tells it about the CLI. s100 00:10:41.040 --> 00:10:44.640 Um, and then, you know, using the CLI itself to query things. s101 00:10:44.640 --> 00:10:46.079 We got back the connectors. s102 00:10:46.079 --> 00:10:50.160 Then we asked, you know, show five paying customers that have Zenes support tickets. s103 00:10:50.160 --> 00:10:51.760 Same as last time. s104 00:10:51.760 --> 00:11:05.120 um it's able to this time use the CLI to query that query the context store aggregate that data and then eventually it should come back with a result now s105 00:11:05.120 --> 00:11:18.240 okay um so there's those same five um five users so um on the design side for CLI I think one important thing to keep in mind is to design those CLIs with agents in mind s106 00:11:18.240 --> 00:11:26.160 um some of the things that might be helpful for users when you're making a CLI are going to be like blockers for agents. s107 00:11:26.160 --> 00:11:28.959 Um, and we'll get into that a little bit more. s108 00:11:28.959 --> 00:11:36.320 Um, the first design decision here that we made was take JSON in and return JSON out. s109 00:11:36.320 --> 00:11:54.320 So uh for humans it's very nice to specify um a bunch of flags uh to work with CLIs but when it comes to these more complicated um inputs that uh you have to build to to make these queries s110 00:11:54.320 --> 00:11:58.480 and agents are really good at working with JSON. s111 00:11:58.480 --> 00:12:08.399 um they're able to construct this uh in a much more um direct and and less of a guessy way um than using those flags. s112 00:12:08.399 --> 00:12:26.800 And so on the output, JSON also makes it easy to deal with longer responses um be able to leverage one of the key capabilities of CLIs, which is hey, you have these this Unix system with pipes that you can redirect the output to different um programs. s113 00:12:26.800 --> 00:12:39.360 So you can use jq for example to select certain certain things from the output and trim that and so JSON really unlocks a lot of that um capability. s114 00:12:39.360 --> 00:12:42.320 Two consistency is very much key here. s115 00:12:42.320 --> 00:12:49.040 So it should be dead obvious for agents to know how to do something across all of the resources that you're exposing. s116 00:12:49.040 --> 00:13:00.800 If you have structured your CLI in a way that it's it's structured where there's a noun and then a verb for this example you know connectors list connectors create s117 00:13:00.800 --> 00:13:08.320 make sure that when you're implementing a workspace um CLI uh commands they also follow that same format. s118 00:13:08.320 --> 00:13:11.600 So workspace list is another one that that we have. s119 00:13:11.600 --> 00:13:23.600 And so it's very um critical that agents know the right way to do things and it's easy for them to know the right way to do things without having to read a bunch of documentation. s120 00:13:23.600 --> 00:13:33.839 And another interesting fact here is, you know, if you see agents making the same mistake over and over, um calling the wrong command a bunch of times, consider making that your contract. s121 00:13:33.839 --> 00:13:39.519 Make it dead simple for agents to stumble across the the right way to to use things. s122 00:13:39.519 --> 00:13:50.079 um because that usually means that something is not quite as obvious as it should be in your AP in your CLI contract if agents keep making the same kind of mistakes. s123 00:13:50.079 --> 00:13:54.320 Three, skills are skills ship with uh the CLI tool. s124 00:13:54.320 --> 00:14:05.040 So skills are a must uh when you're doing CLIs because how else if you have a CLI installed is an agent going to know that that CLI even exists in the first place. s125 00:14:05.040 --> 00:14:09.600 So it helps a lot with discovery of uh your tools that are available. s126 00:14:09.600 --> 00:14:13.920 Um and they also help with uh using that tool effectively. s127 00:14:13.920 --> 00:14:17.519 So uh we have a skill that gets surfaced. s128 00:14:17.519 --> 00:14:22.560 It has a a description that is able to come up whenever you ask anything about connectors. s129 00:14:22.560 --> 00:14:25.680 It knows to bring up that CLI and invoke it. s130 00:14:25.680 --> 00:14:30.560 Um and then the way that you structure the skill is also something to um think through. s131 00:14:30.560 --> 00:14:35.440 you know, you don't want to put too much into the skill uh that you're going to bog down the context. s132 00:14:35.440 --> 00:14:46.480 Um, skills have this uh ability to be separated out into references um where you can have a main skill that points to other files in a in a skill directory s133 00:14:46.480 --> 00:14:49.680 um that can expand information about the the different commands. s134 00:14:49.680 --> 00:15:00.480 And that really uh helps cut down the the amount of uh of context that you're loading in and um leverages the progressive discovery um ability of uh of these agents. s135 00:15:00.480 --> 00:15:01.120 Cool. s136 00:15:01.120 --> 00:15:05.839 And then another interesting fact uh for off side, off was interesting on the MCP side. s137 00:15:05.839 --> 00:15:15.680 It is interesting in other ways for CLIs because we can't assume that a human is going to be present to able to interact with those prompts. s138 00:15:15.680 --> 00:15:24.480 Um when CLIs are being invoked by agents, it can't sit there and wait for you to type um a prompt into the CLI. s139 00:15:24.480 --> 00:15:38.399 you have to expose things either through flags um or in the case of credentials environment variables and config files um so that things can be set up out of band and you don't have to wait for that uh whenever you're invoking the CLI s140 00:15:38.399 --> 00:15:49.839 um and I would say aside from off um that's a general thing a lot of CLIs that are built for humans you type a command and it'll wait there ask you for prompts s141 00:15:49.839 --> 00:15:55.519 uh and that is not um you know able to be used properly ly by these agents. s142 00:15:55.519 --> 00:15:57.920 Um they'll just kind of get get stuck there. s143 00:15:57.920 --> 00:15:59.839 Um cool. s144 00:15:59.839 --> 00:16:04.880 So CLI and MCPs, which one is better? s145 00:16:04.880 --> 00:16:06.959 Well, they both have kind of their own trade-offs. s146 00:16:06.959 --> 00:16:12.560 You know, with the MCP, the spec is not uniformly implemented across the board. s147 00:16:12.560 --> 00:16:17.759 Um there's a bunch of limits that these providers decide to implement aside from the spec as well. s148 00:16:17.759 --> 00:16:24.560 For example, like Claude has a 2 kilobyte limit on server and tool call uh descriptions. s149 00:16:24.560 --> 00:16:26.720 There's a limit of how long the tools can run. s150 00:16:26.720 --> 00:16:31.120 So it's harder to do more longunning processes through MCP. s151 00:16:31.120 --> 00:16:36.720 Um and uh you know there's there's other kind of kind of limits that they've introduced there. s152 00:16:36.720 --> 00:16:44.320 And then on the CLI, the cool things about MCP like you know you can have a hosted MCP server that just is always updated. s153 00:16:44.320 --> 00:16:46.000 That's not quite the case with the CLI. s154 00:16:46.000 --> 00:16:47.759 you have to think more about versioning. s155 00:16:47.759 --> 00:16:51.440 It's harder to keep packages up to date uh with all these new features. s156 00:16:51.440 --> 00:17:03.680 Um and there's no while the limits can be well limiting on the MCP side, not having any of those on the CLI across the tools uh across the the calls of the CLI s157 00:17:03.680 --> 00:17:11.199 can make things go wrong and continually go through loops that um you know make the agent spin. s158 00:17:11.199 --> 00:17:13.760 And so really I think both have their place. s159 00:17:13.760 --> 00:17:27.600 I think MCPs are great for those non-technical users, be able to quickly prototype things, build one-off reports, um, and even leverage that MCP ecosystem to have things uh, talk to each other. s160 00:17:27.600 --> 00:17:42.559 And then the CLI is better for those more technical users, more longer running tasks, maybe outputs that are super long and you need to leverage those um features that Unix and and the pipes give you where you can redirect output to a file. s161 00:17:42.559 --> 00:17:45.520 You can do a tail, do a head on that output. s162 00:17:45.520 --> 00:17:51.200 Um that's where really the the CLI um starts to to shine. s163 00:17:51.200 --> 00:17:54.400 All right, that is basically all that I had. s164 00:17:54.400 --> 00:18:00.640 Um, we have a booth over there at Airbite um that you should check out and come work with us. s165 00:18:00.640 --> 00:18:02.640 Thanks.