WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=lbaXnx0KLA8 NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/lbaXnx0KLA8.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:13.520 --> 00:00:15.760 Uh hey everyone, I'm Dustin Macholic. s3 00:00:15.760 --> 00:00:18.240 Uh I work at Indeed. s4 00:00:18.240 --> 00:00:20.760 Uh we're the number one job site in the world. s5 00:00:20.760 --> 00:00:22.720 And I have to apologize for my voice. s6 00:00:22.720 --> 00:00:25.920 I'm recovering from a cold that I had last week. s7 00:00:25.920 --> 00:00:31.760 Um Yeah, so at Indeed, we build uh job search. s8 00:00:31.760 --> 00:00:36.640 And uh I also work on a team that does AI platform. s9 00:00:36.640 --> 00:00:41.920 And I do like AI guardrails and and gateways and compliance stuff. s10 00:00:41.920 --> 00:00:47.960 Uh occasionally uh my team gets cool projects to work on because we have relationships with the vendors. s11 00:00:47.960 --> 00:00:49.880 Uh MCP apps is one of those. s12 00:00:49.880 --> 00:00:52.440 And um MCP connectors. s13 00:00:52.440 --> 00:00:55.800 So, this is a little bit of like practical MCP apps. s14 00:00:55.800 --> 00:00:59.280 They gave a great introduction uh to MCP apps. s15 00:00:59.280 --> 00:01:08.080 This is a little bit of a lessons from the trenches, uh which is a little bit of like what did we learn in building MCP and MCP apps for Claude s16 00:01:08.080 --> 00:01:17.840 chat chat GPT and our own internal uh uh career scout, which is our uh agent that we have for job seekers. s17 00:01:18.200 --> 00:01:21.560 So, this is the this is the chat-based interface. s18 00:01:21.560 --> 00:01:24.000 So, a lot of my examples are going to be job search. s19 00:01:24.000 --> 00:01:27.360 This will I have a lot of screenshots of job searches. s20 00:01:27.360 --> 00:01:34.320 Um So, this is this is a job search, which is basically uh you know, I'm looking for barista in Austin. s21 00:01:34.320 --> 00:01:36.680 And this is a text-based response. s22 00:01:36.680 --> 00:01:38.680 And this works pretty well. s23 00:01:38.680 --> 00:01:41.760 Uh as we kind of discussed, like there's no branding here. s24 00:01:41.760 --> 00:01:43.480 There's no Indeed branding. s25 00:01:43.480 --> 00:01:47.720 Uh you know, Claude decided to say these are some jobs in Austin. s26 00:01:47.720 --> 00:01:50.240 These are some jobs in some suburbs of Austin. s27 00:01:50.240 --> 00:01:50.760 That's cool. s28 00:01:50.760 --> 00:01:52.360 That's probably good for the user. s29 00:01:52.360 --> 00:02:01.760 There may be some limitations uh of like what we can do for branding or how we can uh you know, how we can control things. s30 00:02:01.760 --> 00:02:14.120 And you'd be actually be surprised uh unless you've tried to do this yourself that um it's really hard to get Claude or ChatGPT to link to things cuz they don't want you to leave their environment. s31 00:02:14.120 --> 00:02:25.320 It makes complete sense, but uh if you get back, here's five jobs that are somewhere on the internet uh without any links, that's a terrible user experience. s32 00:02:25.320 --> 00:02:34.160 Uh so it's it took us a ridiculous number of hours and evals to make sure that like Claude would consistently link to things. s33 00:02:34.160 --> 00:02:39.920 So with MCP apps and with uh apps SDK, uh we could kind of control that. s34 00:02:39.920 --> 00:02:44.200 Uh we can decide uh you know, that we've got an apply button. s35 00:02:44.200 --> 00:02:48.680 We can decide what stuff is important that we want to highlight at the top. s36 00:02:48.680 --> 00:03:00.560 We can provide a link to view details um so that when you click it, you get uh a a pop-up, a a modal that has all the job details so you don't have to leave s37 00:03:00.560 --> 00:03:02.160 the environment. s38 00:03:02.160 --> 00:03:08.800 So it's it's a win-win uh for for both for both uh for both companies. s39 00:03:09.320 --> 00:03:19.880 So uh MCP apps is really good, but if you're thinking about hey, I want to build an MCP app or I want to take my website, I want to put it in ChatGPT or Claude, s40 00:03:19.880 --> 00:03:25.959 uh it's not quite just as easy as like dragging and dropping into a chat interface. s41 00:03:25.959 --> 00:03:32.600 Uh you really want to think about how you are representing the data, how you're making it available to the user. s42 00:03:32.600 --> 00:03:45.040 So if you do a very naive thing, uh which is you still call your existing APIs uh for loading data, um then it basically becomes a black box to the model, s43 00:03:45.040 --> 00:03:45.640 right? s44 00:03:45.640 --> 00:03:47.880 So you say, "Hey, I want to do something." s45 00:03:47.880 --> 00:03:50.120 The model says, "Cool, I'll call a tool." s46 00:03:50.120 --> 00:03:57.840 The tool says, "Okay, I'm going to show some stuff, but the model has no idea what what you're displaying." s47 00:03:57.840 --> 00:04:04.880 So, there's all these like follow-up questions, like tell me about the first result, or please rank this list of companies. s48 00:04:04.880 --> 00:04:07.959 The model has no idea what data is being displayed. s49 00:04:07.959 --> 00:04:19.120 So, the very first rule uh for me uh building MCP apps, anything that you show to the user also needs to be provided as data to the model. s50 00:04:19.120 --> 00:04:29.320 I think this makes sense, but uh I've definitely seen some MCP apps where they just, you know, use it to inject some HTML on the page and then call some APIs, s51 00:04:29.320 --> 00:04:32.880 uh and that's that just makes a big black box for the model. s52 00:04:32.880 --> 00:04:38.400 So, this is really easy to do uh using MCP spec and the MCP app spec. s53 00:04:38.400 --> 00:04:44.120 Uh this structured content, this is what you would already be returning if you were doing just text-based MCP. s54 00:04:44.120 --> 00:04:49.360 Um and then this resource URI, that's points to where the HTML is. s55 00:04:49.360 --> 00:04:52.600 You need to return both of these, and you need to keep them in sync, right? s56 00:04:52.600 --> 00:05:00.000 If you add something new to the API, you make sure you add something add that same data uh back. s57 00:05:00.440 --> 00:05:15.960 So, kind of the next step um that uh that once you do that, uh you'll find is that now you've provided data to the model, and you provided this black box that it has no idea about, s58 00:05:15.960 --> 00:05:23.040 it's still going to try and describe the it's still going to try and take the output and describe it as it normally would. s59 00:05:23.040 --> 00:05:24.720 So, you end up with like, "Here's your display." s60 00:05:24.720 --> 00:05:29.240 and then here's the model doing basically the same thing that it would normally do. s61 00:05:29.240 --> 00:05:36.919 Um so, what you need to do is you need to update your description in order to tell it that you're going to be displaying stuff in your MCP app. s62 00:05:36.919 --> 00:05:50.440 Uh that way you get this like nice, you know, here's a list of you know, here's a little summary of things, and you know, the results are are are showed above the links, rather than it trying to like do a whole text-based display. s63 00:05:50.440 --> 00:05:57.840 Um you'll end up with this a little bit of a battle between like what gets displayed in UI and what gets displayed by the model. s64 00:05:57.840 --> 00:06:01.440 You can try and steer that with descriptions. s65 00:06:01.440 --> 00:06:13.080 So, even something as simple as results were automatically displayed to the user as UI components at the top of your description, your tool description, uh that covers that covers s66 00:06:13.080 --> 00:06:14.840 quite a bit of the cases. s67 00:06:14.840 --> 00:06:24.760 Um So, that's one of the next things that you're going to want to do uh once you're providing both data and API access. s68 00:06:24.760 --> 00:06:28.000 Uh the next thing is there's these interactable pieces, right? s69 00:06:28.000 --> 00:06:33.640 There's the apply button, there's a view details button, which pops up a big job description. s70 00:06:33.640 --> 00:06:41.360 Um this is the same case where, you know, as you interact with those, the model's not going to know necessarily what you're looking at. s71 00:06:41.360 --> 00:06:47.560 So, you click view details, you get a big modal that's here's everything about the job. s72 00:06:47.840 --> 00:06:51.680 There's once again a whole bunch of questions that the user could ask. s73 00:06:51.680 --> 00:06:55.840 Uh write a cover letter for this job, summarize this job description. s74 00:06:55.840 --> 00:07:07.600 It has no idea because you've loaded that data in either in via API or you loaded it in dynamically, uh you know, hit you return 10 jobs and user clicks on one, s75 00:07:07.600 --> 00:07:11.120 the model has no idea which one you clicked on. s76 00:07:11.120 --> 00:07:20.240 Uh so, any information about user interactions, you also need to provide to the model. s77 00:07:20.560 --> 00:07:25.440 And uh once again, MCP abstract has a pretty easy way to handle it. s78 00:07:25.440 --> 00:07:36.000 There's this update model context um method, which lets you pass in a string uh that is So, for MCP apps, there's a single string. s79 00:07:36.000 --> 00:07:46.360 Uh so, if you like want to track multiple events over time, you kind of have to append uh multiple things to the string, but this is this is this is an example from s80 00:07:46.360 --> 00:07:59.360 the MCP apps uh documentation where basically, you know, this is a shopping cart application and they add the total cost and all the items that are on the shopping cart so the user can ask for s81 00:07:59.360 --> 00:08:03.395 uh you know, "Tell me information about the items that are in my shopping cart." s82 00:08:03.395 --> 00:08:05.395 [snorts] s83 00:08:05.880 --> 00:08:13.040 So, these two things give you an app that like the model could kind of see what's going on. s84 00:08:13.040 --> 00:08:26.960 Um but it doesn't necessarily make a really good MCP app yet uh because it gives you it gives you some UI that looks like what you want, but the thing that I usually do, s85 00:08:26.960 --> 00:08:33.599 I don't give I don't give the I don't I don't give Claude uh my easy problems to solve. s86 00:08:33.599 --> 00:08:36.200 I give Claude my really hard problems to solve, right? s87 00:08:36.200 --> 00:08:40.479 Like if I just wanted to do one search, I would go to the web and do one search. s88 00:08:40.479 --> 00:08:42.479 I want to do a whole bunch of searches. s89 00:08:42.479 --> 00:08:51.560 Uh this is this is out of date cuz there's no Sonic 5 here yet, but uh this is screenshot from 2 days ago. s90 00:08:51.560 --> 00:08:57.800 Uh but yeah, so that So, like this job search is "Hey, I'm looking for this I'm looking for this title. s91 00:08:57.800 --> 00:09:02.440 I'm willing to relocate, so I want to search across a whole bunch of different cities. s92 00:09:02.440 --> 00:09:07.520 Um you know, I'm looking for the highest paying option, so I want you to just cherry-pick a few out of there. s93 00:09:07.520 --> 00:09:11.720 There's some industries that I absolutely don't want to work in." s94 00:09:11.720 --> 00:09:14.920 Text-based MCP does really well. s95 00:09:14.920 --> 00:09:17.200 Like we all see this, right? s96 00:09:17.200 --> 00:09:21.680 Claude will do 10 different searches, 15 different searches. s97 00:09:21.680 --> 00:09:30.511 It'll filter, it'll pull out all the individual pieces, uh and then it gives you a nice table at the bottom, uh which is which is super nice. s98 00:09:30.511 --> 00:09:30.600 [snorts] s99 00:09:30.600 --> 00:09:40.920 But with the MCP app that we were just discussing, you know, we we said, "Hey, uh you know, my results are going to be displayed in this UI. s100 00:09:40.920 --> 00:09:48.560 Uh We know Claude Claude will call it once and then it'll be like, "Oh, I guess the results are already displayed. s101 00:09:48.560 --> 00:09:50.400 I'm not going to like do a deep dive, right?" s102 00:09:50.400 --> 00:10:03.120 Like it doesn't It's you as a user are not going to want 10 different carousels and Claude will also notice that like it's already been displaying some stuff and it won't call s103 00:10:03.120 --> 00:10:05.400 to show 10 different carousels. s104 00:10:05.400 --> 00:10:15.560 Um and so what you really want to do, and this is rule three, this like supersedes all the other rules, uh which is basically you want to separate your data processing s105 00:10:15.560 --> 00:10:18.120 from your UI rendering. s106 00:10:18.120 --> 00:10:24.877 And this particular wording I stole uh from OpenAI uh s107 00:10:24.877 --> 00:10:24.880 [snorts] s108 00:10:24.880 --> 00:10:29.320 OpenAI uh Apps SDK documentation. s109 00:10:29.320 --> 00:10:32.040 There's a couple places where they say this. s110 00:10:32.040 --> 00:10:38.600 Um but basically they want to You want to separate your data processing from your UI rendering. s111 00:10:38.600 --> 00:10:47.040 So the job search that we had we're just going to have that be a standard text-based MCP application. s112 00:10:47.040 --> 00:10:50.200 Claude can call that as many times as it wants. s113 00:10:50.200 --> 00:11:01.200 And then we have a render tool that either you can pass you can have the model basically pass all the data that it wants to render in or you can do a reference s114 00:11:01.200 --> 00:11:02.200 uh to it. s115 00:11:02.200 --> 00:11:07.000 So in our particular case uh you know, we had search jobs. s116 00:11:07.000 --> 00:11:14.520 Now we have a search jobs that doesn't return any UI and we have a render jobs widget that takes a list of IDs. s117 00:11:14.520 --> 00:11:20.480 Um and so that list of IDs uh can be, you know, Claude can do a search. s118 00:11:20.480 --> 00:11:24.320 It can get a hundred different jobs that it cares about. s119 00:11:24.320 --> 00:11:25.440 It can filter those. s120 00:11:25.440 --> 00:11:30.680 It can find five that it cares about and it can show those five to the user. s121 00:11:30.680 --> 00:11:40.520 Um now as you make these like render uh as you make these render calls, you need to update the descriptions to say like where did they get the data, s122 00:11:40.520 --> 00:11:42.920 what format the data should be. s123 00:11:42.920 --> 00:11:53.360 But it's a fairly easy fairly easy mechanism to update your tool description to say you need to always call one of these three tools first in order to get s124 00:11:53.360 --> 00:11:56.320 the data that you're going to be using for rendering. s125 00:11:56.320 --> 00:12:07.720 So, search jobs, this is a pretty good um It's a pretty good example of of how we where we want to split data from rendering. s126 00:12:07.720 --> 00:12:16.240 I think there's a ton of other like in most industries, you can kind of come up with like, "Hey, where do I want to you know, where do I want to split?" s127 00:12:16.240 --> 00:12:24.960 Like I want to be I want the model to be able to explore this data and then I want it to turn around and choose to be able to render it. s128 00:12:24.960 --> 00:12:25.240 Right? s129 00:12:25.240 --> 00:12:31.160 Like there's there's examples of you know e-commerce, right? s130 00:12:31.160 --> 00:12:34.520 Like a bunch of e-commerce options. s131 00:12:34.520 --> 00:12:41.600 If you've got a map, you know, maybe you want to come up with like five different addresses and then you pass in addresses. s132 00:12:41.600 --> 00:12:45.520 The other thing that you can do is you can let the model be a lot more creative. s133 00:12:45.520 --> 00:12:54.218 Like one of the things that we've seen in some of this text-based stuff is that you know, the model will say like, "This is a s134 00:12:54.218 --> 00:12:54.240 [clears throat] s135 00:12:54.240 --> 00:12:56.440 This is a reason why I picked this one." s136 00:12:56.440 --> 00:12:59.320 Or "This is a reason why this is really good." s137 00:12:59.320 --> 00:13:09.720 And so, you know, potentially we can add something to the render jobs widget where you say give us an ID and a reason why you think that this is a good fit. s138 00:13:09.720 --> 00:13:15.880 Or give us an ID and highlight a section of the job description that is really good. s139 00:13:15.880 --> 00:13:28.120 Um, so you can get really creative with your render tools to be able to have to be able to give some extra character that the model can inject into those s140 00:13:28.120 --> 00:13:33.920 so that so that you've got a much better experience for the user. s141 00:13:34.120 --> 00:13:34.174 So s142 00:13:34.174 --> 00:13:34.240 [snorts] s143 00:13:34.240 --> 00:13:49.320 the key takeaways right when building MCP apps, you want to focus on the data before you focus on the UI which sounds sounds opposite of you know how we were thinking about it of like hey there's all these there's these MCP apps like s144 00:13:49.320 --> 00:13:52.840 it's how I put UI into chat GPT. s145 00:13:52.840 --> 00:14:05.440 I think if you want to really have a good MCP apps experience, you need to you need to look at and see what data do I want to what data do I want to give to the model, what data do I want it to be able to do s146 00:14:05.440 --> 00:14:15.160 and then rendering is a side effect of of that or it's a result of the the model exploring the data. s147 00:14:15.160 --> 00:14:19.360 And then I think small composable tools, right? s148 00:14:19.360 --> 00:14:27.360 So uh basically, you know maybe there's like two or three different ways you can search for jobs. s149 00:14:27.360 --> 00:14:36.520 We could build two or three different search tools and then there's one render tool or maybe there's like maybe there's two render tools there's one that's render a list of jobs, s150 00:14:36.520 --> 00:14:39.600 one that's, you know, highlight one particular job. s151 00:14:39.600 --> 00:14:42.800 So you could build a bunch of much smaller tools. s152 00:14:42.800 --> 00:14:54.360 The descriptions can be fairly simple so you don't overload the model, but it gives the model flexibility about how it wants to explore the data and how it wants to render the tools. s153 00:14:54.600 --> 00:14:58.280 So that's basically basically my talk. s154 00:14:58.280 --> 00:15:01.839 I'm a few minutes few minutes fast. s155 00:15:01.839 --> 00:15:08.960 But I don't have a booth that I'm going to hang out in, but I'll I'll hang out in the hall if anyone has any questions. s156 00:15:08.960 --> 00:15:15.960 And I am either my last name or my first initial last name on most social media platforms. s157 00:15:15.960 --> 00:15:18.200 Thank you. s158 00:15:32.075 --> 00:15:33.600 [music] s159 00:15:33.600 --> 00:15:35.600 Mhm.