MCP Apps: Give the Model Data, Give the User a UI — Dustin Mihalik, Indeed

AI Engineer · 15 min · 159 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:13Uh hey everyone, I'm Dustin Macholic.
00:15Uh I work at Indeed.
00:18Uh we're the number one job site in the world.
00:20And I have to apologize for my voice.
00:22I'm recovering from a cold that I had last week.
00:25Um Yeah, so at Indeed, we build uh job search.
00:31And uh I also work on a team that does AI platform.
00:36And I do like AI guardrails and and gateways and compliance stuff.
00:41Uh occasionally uh my team gets cool projects to work on because we have relationships with the vendors.
00:47Uh MCP apps is one of those.
00:49And um MCP connectors.
00:52So, this is a little bit of like practical MCP apps.
00:55They gave a great introduction uh to MCP apps.
00:59This is a little bit of a lessons from the trenches, uh which is a little bit of like what did we learn in building MCP and MCP apps for Claude
01:08chat chat GPT and our own internal uh uh career scout, which is our uh agent that we have for job seekers.
01:18So, this is the this is the chat-based interface.
01:21So, a lot of my examples are going to be job search.
01:24This will I have a lot of screenshots of job searches.
01:27Um So, this is this is a job search, which is basically uh you know, I'm looking for barista in Austin.
01:34And this is a text-based response.
01:36And this works pretty well.
01:38Uh as we kind of discussed, like there's no branding here.
01:41There's no Indeed branding.
01:43Uh you know, Claude decided to say these are some jobs in Austin.
01:47These are some jobs in some suburbs of Austin.
01:50That's cool.
01:50That's probably good for the user.
01:52There may be some limitations uh of like what we can do for branding or how we can uh you know, how we can control things.
02:01And you'd be actually be surprised uh unless you've tried to do this yourself that um it's really hard to get Claude or ChatGPT to link to things cuz they don't want you to leave their environment.
02:14It makes complete sense, but uh if you get back, here's five jobs that are somewhere on the internet uh without any links, that's a terrible user experience.
02:25Uh so it's it took us a ridiculous number of hours and evals to make sure that like Claude would consistently link to things.
02:34So with MCP apps and with uh apps SDK, uh we could kind of control that.
02:39Uh we can decide uh you know, that we've got an apply button.
02:44We can decide what stuff is important that we want to highlight at the top.
02:48We can provide a link to view details um so that when you click it, you get uh a a pop-up, a a modal that has all the job details so you don't have to leave
03:00the environment.
03:02So it's it's a win-win uh for for both for both uh for both companies.
03:09So uh MCP apps is really good, but if you're thinking about hey, I want to build an MCP app or I want to take my website, I want to put it in ChatGPT or Claude,
03:19uh it's not quite just as easy as like dragging and dropping into a chat interface.
03:25Uh you really want to think about how you are representing the data, how you're making it available to the user.
03:32So if you do a very naive thing, uh which is you still call your existing APIs uh for loading data, um then it basically becomes a black box to the model,
03:45right?
03:45So you say, "Hey, I want to do something."
03:47The model says, "Cool, I'll call a tool."
03:50The tool says, "Okay, I'm going to show some stuff, but the model has no idea what what you're displaying."
03:57So, there's all these like follow-up questions, like tell me about the first result, or please rank this list of companies.
04:04The model has no idea what data is being displayed.
04:07So, the very first rule uh for me uh building MCP apps, anything that you show to the user also needs to be provided as data to the model.
04:19I think this makes sense, but uh I've definitely seen some MCP apps where they just, you know, use it to inject some HTML on the page and then call some APIs,
04:29uh and that's that just makes a big black box for the model.
04:32So, this is really easy to do uh using MCP spec and the MCP app spec.
04:38Uh this structured content, this is what you would already be returning if you were doing just text-based MCP.
04:44Um and then this resource URI, that's points to where the HTML is.
04:49You need to return both of these, and you need to keep them in sync, right?
04:52If you add something new to the API, you make sure you add something add that same data uh back.
05:00So, kind of the next step um that uh that once you do that, uh you'll find is that now you've provided data to the model, and you provided this black box that it has no idea about,
05:15it's still going to try and describe the it's still going to try and take the output and describe it as it normally would.
05:23So, you end up with like, "Here's your display."
05:24and then here's the model doing basically the same thing that it would normally do.
05:29Um so, what you need to do is you need to update your description in order to tell it that you're going to be displaying stuff in your MCP app.
05:36Uh that way you get this like nice, you know, here's a list of you know, here's a little summary of things, and you know, the results are are are showed above the links, rather than it trying to like do a whole text-based display.
05:50Um you'll end up with this a little bit of a battle between like what gets displayed in UI and what gets displayed by the model.
05:57You can try and steer that with descriptions.
06:01So, even something as simple as results were automatically displayed to the user as UI components at the top of your description, your tool description, uh that covers that covers
06:13quite a bit of the cases.
06:14Um So, that's one of the next things that you're going to want to do uh once you're providing both data and API access.
06:24Uh the next thing is there's these interactable pieces, right?
06:28There's the apply button, there's a view details button, which pops up a big job description.
06:33Um this is the same case where, you know, as you interact with those, the model's not going to know necessarily what you're looking at.
06:41So, you click view details, you get a big modal that's here's everything about the job.
06:47There's once again a whole bunch of questions that the user could ask.
06:51Uh write a cover letter for this job, summarize this job description.
06:55It has no idea because you've loaded that data in either in via API or you loaded it in dynamically, uh you know, hit you return 10 jobs and user clicks on one,
07:07the model has no idea which one you clicked on.
07:11Uh so, any information about user interactions, you also need to provide to the model.
07:20And uh once again, MCP abstract has a pretty easy way to handle it.
07:25There's this update model context um method, which lets you pass in a string uh that is So, for MCP apps, there's a single string.
07:36Uh so, if you like want to track multiple events over time, you kind of have to append uh multiple things to the string, but this is this is this is an example from
07:46the MCP apps uh documentation where basically, you know, this is a shopping cart application and they add the total cost and all the items that are on the shopping cart so the user can ask for
07:59uh you know, "Tell me information about the items that are in my shopping cart."
08:03[snorts]
08:05So, these two things give you an app that like the model could kind of see what's going on.
08:13Um but it doesn't necessarily make a really good MCP app yet uh because it gives you it gives you some UI that looks like what you want, but the thing that I usually do,
08:26I don't give I don't give the I don't I don't give Claude uh my easy problems to solve.
08:33I give Claude my really hard problems to solve, right?
08:36Like if I just wanted to do one search, I would go to the web and do one search.
08:40I want to do a whole bunch of searches.
08:42Uh this is this is out of date cuz there's no Sonic 5 here yet, but uh this is screenshot from 2 days ago.
08:51Uh but yeah, so that So, like this job search is "Hey, I'm looking for this I'm looking for this title.
08:57I'm willing to relocate, so I want to search across a whole bunch of different cities.
09:02Um you know, I'm looking for the highest paying option, so I want you to just cherry-pick a few out of there.
09:07There's some industries that I absolutely don't want to work in."
09:11Text-based MCP does really well.
09:14Like we all see this, right?
09:17Claude will do 10 different searches, 15 different searches.
09:21It'll filter, it'll pull out all the individual pieces, uh and then it gives you a nice table at the bottom, uh which is which is super nice.
09:30[snorts]
09:30But with the MCP app that we were just discussing, you know, we we said, "Hey, uh you know, my results are going to be displayed in this UI.
09:40Uh We know Claude Claude will call it once and then it'll be like, "Oh, I guess the results are already displayed.
09:48I'm not going to like do a deep dive, right?"
09:50Like it doesn't It's you as a user are not going to want 10 different carousels and Claude will also notice that like it's already been displaying some stuff and it won't call
10:03to show 10 different carousels.
10:05Um and so what you really want to do, and this is rule three, this like supersedes all the other rules, uh which is basically you want to separate your data processing
10:15from your UI rendering.
10:18And this particular wording I stole uh from OpenAI uh
10:24[snorts]
10:24OpenAI uh Apps SDK documentation.
10:29There's a couple places where they say this.
10:32Um but basically they want to You want to separate your data processing from your UI rendering.
10:38So the job search that we had we're just going to have that be a standard text-based MCP application.
10:47Claude can call that as many times as it wants.
10:50And then we have a render tool that either you can pass you can have the model basically pass all the data that it wants to render in or you can do a reference
11:01uh to it.
11:02So in our particular case uh you know, we had search jobs.
11:07Now we have a search jobs that doesn't return any UI and we have a render jobs widget that takes a list of IDs.
11:14Um and so that list of IDs uh can be, you know, Claude can do a search.
11:20It can get a hundred different jobs that it cares about.
11:24It can filter those.
11:25It can find five that it cares about and it can show those five to the user.
11:30Um now as you make these like render uh as you make these render calls, you need to update the descriptions to say like where did they get the data,
11:40what format the data should be.
11:42But it's a fairly easy fairly easy mechanism to update your tool description to say you need to always call one of these three tools first in order to get
11:53the data that you're going to be using for rendering.
11:56So, search jobs, this is a pretty good um It's a pretty good example of of how we where we want to split data from rendering.
12:07I think there's a ton of other like in most industries, you can kind of come up with like, "Hey, where do I want to you know, where do I want to split?"
12:16Like I want to be I want the model to be able to explore this data and then I want it to turn around and choose to be able to render it.
12:24Right?
12:25Like there's there's examples of you know e-commerce, right?
12:31Like a bunch of e-commerce options.
12:34If you've got a map, you know, maybe you want to come up with like five different addresses and then you pass in addresses.
12:41The other thing that you can do is you can let the model be a lot more creative.
12:45Like one of the things that we've seen in some of this text-based stuff is that you know, the model will say like, "This is a
12:54[clears throat]
12:54This is a reason why I picked this one."
12:56Or "This is a reason why this is really good."
12:59And so, you know, potentially we can add something to the render jobs widget where you say give us an ID and a reason why you think that this is a good fit.
13:09Or give us an ID and highlight a section of the job description that is really good.
13:15Um, so you can get really creative with your render tools to be able to have to be able to give some extra character that the model can inject into those
13:28so that so that you've got a much better experience for the user.
13:34So
13:34[snorts]
13:34the key takeaways right when building MCP apps, you want to focus on the data before you focus on the UI which sounds sounds opposite of you know how we were thinking about it of like hey there's all these there's these MCP apps like
13:49it's how I put UI into chat GPT.
13:52I think if you want to really have a good MCP apps experience, you need to you need to look at and see what data do I want to what data do I want to give to the model, what data do I want it to be able to do
14:05and then rendering is a side effect of of that or it's a result of the the model exploring the data.
14:15And then I think small composable tools, right?
14:19So uh basically, you know maybe there's like two or three different ways you can search for jobs.
14:27We could build two or three different search tools and then there's one render tool or maybe there's like maybe there's two render tools there's one that's render a list of jobs,
14:36one that's, you know, highlight one particular job.
14:39So you could build a bunch of much smaller tools.
14:42The descriptions can be fairly simple so you don't overload the model, but it gives the model flexibility about how it wants to explore the data and how it wants to render the tools.
14:54So that's basically basically my talk.
14:58I'm a few minutes few minutes fast.
15:01But I don't have a booth that I'm going to hang out in, but I'll I'll hang out in the hall if anyone has any questions.
15:08And I am either my last name or my first initial last name on most social media platforms.
15:15Thank you.
15:32[music]
15:33Mhm.