MCP Apps: Give the Model Data, Give the User a UI — Dustin Mihalik, Indeed
https://www.youtube.com/watch?v=lbaXnx0KLA8
AI Engineer · 15 min · 159 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:13] Uh hey everyone, I'm Dustin Macholic.
[00:15] Uh I work at Indeed.
[00:18] Uh we're the number one job site in the world.
[00:20] And I have to apologize for my voice.
[00:22] I'm recovering from a cold that I had last week.
[00:25] Um Yeah, so at Indeed, we build uh job search.
[00:31] And uh I also work on a team that does AI platform.
[00:36] And I do like AI guardrails and and gateways and compliance stuff.
[00:41] Uh occasionally uh my team gets cool projects to work on because we have relationships with the vendors.
[00:47] Uh MCP apps is one of those.
[00:49] And um MCP connectors.
[00:52] So, this is a little bit of like practical MCP apps.
[00:55] They gave a great introduction uh to MCP apps.
[00:59] This is a little bit of a lessons from the trenches, uh which is a little bit of like what did we learn in building MCP and MCP apps for Claude
[01:08] chat chat GPT and our own internal uh uh career scout, which is our uh agent that we have for job seekers.
[01:18] So, this is the this is the chat-based interface.
[01:21] So, a lot of my examples are going to be job search.
[01:24] This will I have a lot of screenshots of job searches.
[01:27] Um So, this is this is a job search, which is basically uh you know, I'm looking for barista in Austin.
[01:34] And this is a text-based response.
[01:36] And this works pretty well.
[01:38] Uh as we kind of discussed, like there's no branding here.
[01:41] There's no Indeed branding.
[01:43] Uh you know, Claude decided to say these are some jobs in Austin.
[01:47] These are some jobs in some suburbs of Austin.
[01:50] That's cool.
[01:50] That's probably good for the user.
[01:52] There may be some limitations uh of like what we can do for branding or how we can uh you know, how we can control things.
[02:01] And you'd be actually be surprised uh unless you've tried to do this yourself that um it's really hard to get Claude or ChatGPT to link to things cuz they don't want you to leave their environment.
[02:14] It makes complete sense, but uh if you get back, here's five jobs that are somewhere on the internet uh without any links, that's a terrible user experience.
[02:25] Uh so it's it took us a ridiculous number of hours and evals to make sure that like Claude would consistently link to things.
[02:34] So with MCP apps and with uh apps SDK, uh we could kind of control that.
[02:39] Uh we can decide uh you know, that we've got an apply button.
[02:44] We can decide what stuff is important that we want to highlight at the top.
[02:48] We can provide a link to view details um so that when you click it, you get uh a a pop-up, a a modal that has all the job details so you don't have to leave
[03:00] the environment.
[03:02] So it's it's a win-win uh for for both for both uh for both companies.
[03:09] So uh MCP apps is really good, but if you're thinking about hey, I want to build an MCP app or I want to take my website, I want to put it in ChatGPT or Claude,
[03:19] uh it's not quite just as easy as like dragging and dropping into a chat interface.
[03:25] Uh you really want to think about how you are representing the data, how you're making it available to the user.
[03:32] So if you do a very naive thing, uh which is you still call your existing APIs uh for loading data, um then it basically becomes a black box to the model,
[03:45] right?
[03:45] So you say, "Hey, I want to do something."
[03:47] The model says, "Cool, I'll call a tool."
[03:50] The tool says, "Okay, I'm going to show some stuff, but the model has no idea what what you're displaying."
[03:57] So, there's all these like follow-up questions, like tell me about the first result, or please rank this list of companies.
[04:04] The model has no idea what data is being displayed.
[04:07] So, the very first rule uh for me uh building MCP apps, anything that you show to the user also needs to be provided as data to the model.
[04:19] I think this makes sense, but uh I've definitely seen some MCP apps where they just, you know, use it to inject some HTML on the page and then call some APIs,
[04:29] uh and that's that just makes a big black box for the model.
[04:32] So, this is really easy to do uh using MCP spec and the MCP app spec.
[04:38] Uh this structured content, this is what you would already be returning if you were doing just text-based MCP.
[04:44] Um and then this resource URI, that's points to where the HTML is.
[04:49] You need to return both of these, and you need to keep them in sync, right?
[04:52] If you add something new to the API, you make sure you add something add that same data uh back.
[05:00] So, kind of the next step um that uh that once you do that, uh you'll find is that now you've provided data to the model, and you provided this black box that it has no idea about,
[05:15] it's still going to try and describe the it's still going to try and take the output and describe it as it normally would.
[05:23] So, you end up with like, "Here's your display."
[05:24] and then here's the model doing basically the same thing that it would normally do.
[05:29] Um so, what you need to do is you need to update your description in order to tell it that you're going to be displaying stuff in your MCP app.
[05:36] Uh that way you get this like nice, you know, here's a list of you know, here's a little summary of things, and you know, the results are are are showed above the links, rather than it trying to like do a whole text-based display.
[05:50] Um you'll end up with this a little bit of a battle between like what gets displayed in UI and what gets displayed by the model.
[05:57] You can try and steer that with descriptions.
[06:01] So, even something as simple as results were automatically displayed to the user as UI components at the top of your description, your tool description, uh that covers that covers
[06:13] quite a bit of the cases.
[06:14] Um So, that's one of the next things that you're going to want to do uh once you're providing both data and API access.
[06:24] Uh the next thing is there's these interactable pieces, right?
[06:28] There's the apply button, there's a view details button, which pops up a big job description.
[06:33] Um this is the same case where, you know, as you interact with those, the model's not going to know necessarily what you're looking at.
[06:41] So, you click view details, you get a big modal that's here's everything about the job.
[06:47] There's once again a whole bunch of questions that the user could ask.
[06:51] Uh write a cover letter for this job, summarize this job description.
[06:55] It has no idea because you've loaded that data in either in via API or you loaded it in dynamically, uh you know, hit you return 10 jobs and user clicks on one,
[07:07] the model has no idea which one you clicked on.
[07:11] Uh so, any information about user interactions, you also need to provide to the model.
[07:20] And uh once again, MCP abstract has a pretty easy way to handle it.
[07:25] There's this update model context um method, which lets you pass in a string uh that is So, for MCP apps, there's a single string.
[07:36] Uh so, if you like want to track multiple events over time, you kind of have to append uh multiple things to the string, but this is this is this is an example from
[07:46] the MCP apps uh documentation where basically, you know, this is a shopping cart application and they add the total cost and all the items that are on the shopping cart so the user can ask for
[07:59] uh you know, "Tell me information about the items that are in my shopping cart."
[08:03] [snorts]
[08:05] So, these two things give you an app that like the model could kind of see what's going on.
[08:13] Um but it doesn't necessarily make a really good MCP app yet uh because it gives you it gives you some UI that looks like what you want, but the thing that I usually do,
[08:26] I don't give I don't give the I don't I don't give Claude uh my easy problems to solve.
[08:33] I give Claude my really hard problems to solve, right?
[08:36] Like if I just wanted to do one search, I would go to the web and do one search.
[08:40] I want to do a whole bunch of searches.
[08:42] Uh this is this is out of date cuz there's no Sonic 5 here yet, but uh this is screenshot from 2 days ago.
[08:51] Uh but yeah, so that So, like this job search is "Hey, I'm looking for this I'm looking for this title.
[08:57] I'm willing to relocate, so I want to search across a whole bunch of different cities.
[09:02] Um you know, I'm looking for the highest paying option, so I want you to just cherry-pick a few out of there.
[09:07] There's some industries that I absolutely don't want to work in."
[09:11] Text-based MCP does really well.
[09:14] Like we all see this, right?
[09:17] Claude will do 10 different searches, 15 different searches.
[09:21] It'll filter, it'll pull out all the individual pieces, uh and then it gives you a nice table at the bottom, uh which is which is super nice.
[09:30] [snorts]
[09:30] But with the MCP app that we were just discussing, you know, we we said, "Hey, uh you know, my results are going to be displayed in this UI.
[09:40] Uh We know Claude Claude will call it once and then it'll be like, "Oh, I guess the results are already displayed.
[09:48] I'm not going to like do a deep dive, right?"
[09:50] Like it doesn't It's you as a user are not going to want 10 different carousels and Claude will also notice that like it's already been displaying some stuff and it won't call
[10:03] to show 10 different carousels.
[10:05] Um and so what you really want to do, and this is rule three, this like supersedes all the other rules, uh which is basically you want to separate your data processing
[10:15] from your UI rendering.
[10:18] And this particular wording I stole uh from OpenAI uh
[10:24] [snorts]
[10:24] OpenAI uh Apps SDK documentation.
[10:29] There's a couple places where they say this.
[10:32] Um but basically they want to You want to separate your data processing from your UI rendering.
[10:38] So the job search that we had we're just going to have that be a standard text-based MCP application.
[10:47] Claude can call that as many times as it wants.
[10:50] And then we have a render tool that either you can pass you can have the model basically pass all the data that it wants to render in or you can do a reference
[11:01] uh to it.
[11:02] So in our particular case uh you know, we had search jobs.
[11:07] Now we have a search jobs that doesn't return any UI and we have a render jobs widget that takes a list of IDs.
[11:14] Um and so that list of IDs uh can be, you know, Claude can do a search.
[11:20] It can get a hundred different jobs that it cares about.
[11:24] It can filter those.
[11:25] It can find five that it cares about and it can show those five to the user.
[11:30] Um now as you make these like render uh as you make these render calls, you need to update the descriptions to say like where did they get the data,
[11:40] what format the data should be.
[11:42] But it's a fairly easy fairly easy mechanism to update your tool description to say you need to always call one of these three tools first in order to get
[11:53] the data that you're going to be using for rendering.
[11:56] So, search jobs, this is a pretty good um It's a pretty good example of of how we where we want to split data from rendering.
[12:07] I think there's a ton of other like in most industries, you can kind of come up with like, "Hey, where do I want to you know, where do I want to split?"
[12:16] Like I want to be I want the model to be able to explore this data and then I want it to turn around and choose to be able to render it.
[12:24] Right?
[12:25] Like there's there's examples of you know e-commerce, right?
[12:31] Like a bunch of e-commerce options.
[12:34] If you've got a map, you know, maybe you want to come up with like five different addresses and then you pass in addresses.
[12:41] The other thing that you can do is you can let the model be a lot more creative.
[12:45] Like one of the things that we've seen in some of this text-based stuff is that you know, the model will say like, "This is a
[12:54] [clears throat]
[12:54] This is a reason why I picked this one."
[12:56] Or "This is a reason why this is really good."
[12:59] And so, you know, potentially we can add something to the render jobs widget where you say give us an ID and a reason why you think that this is a good fit.
[13:09] Or give us an ID and highlight a section of the job description that is really good.
[13:15] Um, so you can get really creative with your render tools to be able to have to be able to give some extra character that the model can inject into those
[13:28] so that so that you've got a much better experience for the user.
[13:34] So
[13:34] [snorts]
[13:34] the key takeaways right when building MCP apps, you want to focus on the data before you focus on the UI which sounds sounds opposite of you know how we were thinking about it of like hey there's all these there's these MCP apps like
[13:49] it's how I put UI into chat GPT.
[13:52] I think if you want to really have a good MCP apps experience, you need to you need to look at and see what data do I want to what data do I want to give to the model, what data do I want it to be able to do
[14:05] and then rendering is a side effect of of that or it's a result of the the model exploring the data.
[14:15] And then I think small composable tools, right?
[14:19] So uh basically, you know maybe there's like two or three different ways you can search for jobs.
[14:27] We could build two or three different search tools and then there's one render tool or maybe there's like maybe there's two render tools there's one that's render a list of jobs,
[14:36] one that's, you know, highlight one particular job.
[14:39] So you could build a bunch of much smaller tools.
[14:42] The descriptions can be fairly simple so you don't overload the model, but it gives the model flexibility about how it wants to explore the data and how it wants to render the tools.
[14:54] So that's basically basically my talk.
[14:58] I'm a few minutes few minutes fast.
[15:01] But I don't have a booth that I'm going to hang out in, but I'll I'll hang out in the hall if anyone has any questions.
[15:08] And I am either my last name or my first initial last name on most social media platforms.
[15:15] Thank you.
[15:32] [music]
[15:33] Mhm.