MCP Apps: Extending the Frontier — Ido Salomon & Liad Yosef

AI Engineer · 18 min · 234 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

  1. 00:01[music]
  2. 00:12Hi.
  3. 00:13So, hi everyone.
  4. 00:15We built this talk yesterday, so it might be out of date.
  5. 00:18I'm Ido Sadan, I am the creator of MCPY and co-creator and maintainer of MCP apps in the MCP steering committee.
  6. 00:26I also created Adam Craft if you were in the talk yesterday.
  7. 00:30I'm the Adi.
  8. 00:30I work with Ido on MCPY.
  9. 00:32I'm also the co-creator and maintainer of the MCP apps spec and recently co-founded Aura, which is a research lab for the agentic web.
  10. 00:39And we're going to talk a little bit more about it later.
  11. 00:45So, MCP apps are all around us.
  12. 00:47You might not even realize it, but all the fancy apps you have today in ChatGPT and VS Code and Slack are actually all based on MCP and the MCP app spec.
  13. 01:01And if we take a step back and we ask, why do we need MCP apps?
  14. 01:05What's the idea behind MCPY or MCP apps?
  15. 01:07So, when we work with chats, when we chat client clients, we used to text because that's the natural interface, but text is really the worst way to convey a lot of information, right?
  16. 01:18Because we don't want walls of text.
  17. 01:20And actually, this is the main blocker from companies to build an MCP server.
  18. 01:25They don't want to be reduced to a textual database.
  19. 01:27They don't want to lose their brand identity in the process.
  20. 01:30They don't want their data that they work so hard on um building the UX for to look something like this.
  21. 01:37So, instead of this, what if the apps could just send their UI to the chat, right?
  22. 01:46What if every service and every brand could just send their user interface to the chat?
  23. 01:51So, instead of us looking at something like this, we could just have the apps send their own identity, their own UI chunks into the chat, and then we take a look and we see, "Okay, yeah, I know this is Shopify in the middle.
  24. 02:03I know this is Hugging Face.
  25. 02:05I know this is Monday."
  26. 02:06And what if we don't want to do it only as a visualization?
  27. 02:10We also want to do it interactive.
  28. 02:12So, we want the users to be able to actually interact with Hugging Face, for example.
  29. 02:18And for Hugging Face to actually do something with it.
  30. 02:22So, we don't have to imagine the future as we said with MCPUI, which I created in May last year, and took that, which is essentially like an open protocol for
  31. 02:35interactive applications over MCP.
  32. 02:37So, it's not only how you transmit UI, but also how that UI, that application connect connect communicates with the host.
  33. 02:46And just a few months ago, we partnered with Anthropic and OpenAI to create the official extension to MCP, which we call MCP apps based on MCPUI, MCP SDK, and other
  34. 03:00solutions in the field.
  35. 03:01Their launch was pretty cool with Claude and VS Code supporting it to begin with, but now obviously also OpenAI and others have adopted it.
  36. 03:13Yeah, and there are a lot of early adopters to MCPUI.
  37. 03:1711 Labs, Shopify, Postman.
  38. 03:19Those were one of the first first companies to support it back like a year ago.
  39. 03:23They were the one believing in this spec, in this vision.
  40. 03:27And Goose also supported it.
  41. 03:30And it's a it's a funny anecdote because today Block released their agentic commerce solution that is based on MCP apps.
  42. 03:36So, a year ago Goose was the first client to support MCPUI, and now it is part of Block's product uh product.
  43. 03:45And today we have a lot more clients that are supporting MCPUI.
  44. 03:49We We Cursor, and we have co-pilot and GitHub ChatGPT support MCP apps.
  45. 03:57ChatGPT apps that you know are actually based on MCP apps and open eye actually recommend using MCP apps as the protocol to build ChatGPT apps.
  46. 04:08Postman and a lot more and obviously Cloud supports MCP apps.
  47. 04:12But we also have a lot large community around it, right?
  48. 04:15So people start to to build plugins to MCP apps and um integrations to different agents and also courses on how to build MCP apps.
  49. 04:24This is by integration for MCP apps.
  50. 04:26So we have a lot community around it.
  51. 04:28Um There's a repo X app which is the repo for MCP apps where everyone can just come and propose PRs and ideas of how to how to extend this spec and we have a work group in the MCP committee
  52. 04:41and we're convening every 3 weeks.
  53. 04:43We have a tri-weekly meeting on the future of the protocol and how to make the spec not just serve the bigger apps but also the community.
  54. 04:51So it's an open working group with Anthropic, Open AI and all the partners in in the MCP apps protocol.
  55. 04:58Okay, so let's look at a few of the core concepts of MCP apps.
  56. 05:03The first and most obvious one is how do we even transmit UI over MCP?
  57. 05:08So if we look at this example of Cloud like you know agent times like a few months ago and I would ask something best case scenario it would reach out to my MCP server
  58. 05:20and it would get back a textual response which is obviously suboptimal.
  59. 05:26So let's say I do want to get some something better.
  60. 05:31So now I can use existing MCP primitives like a resource and now return HTML.
  61. 05:38And I can take that HTML and since Cloud supports MCP apps it can turn it into an interactive application of the best soundtrack in the world.
  62. 05:50And what if you wanted to be really interactive, right?
  63. 05:53This is nice because it shows the best soundtrack in the world.
  64. 05:55What if I want to favorite one of the songs there?
  65. 05:58I want interaction.
  66. 06:00I want communication between the app and the host.
  67. 06:04So, when the user clicks on the favorite button, MCP apps actually standardizes this flow.
  68. 06:09So, instead of the app sending a message to to the backend, to Spotify's backend, it's actually sending a message to the host saying, "Hey, user clicked a button.
  69. 06:19Do something with it.
  70. 06:20I recommend you to call a tool in Spotify's MCP server."
  71. 06:23And the host decides what to do.
  72. 06:25The host keeps this control of the flow.
  73. 06:27In this case, the host can decide to actually call the favorite favorite tool.
  74. 06:31And MCP apps standardizes this flow.
  75. 06:35Okay, so seeing is believing.
  76. 06:36So, let's see an example from Claude.
  77. 06:42Yeah.
  78. 06:43Uh so, let's say that I'm a product manager to understand the status of my funnel.
  79. 06:48So, I would go to Claude and I would ask what's the status?
  80. 06:51In the again, old world of a few months ago, uh I would get back the textual response.
  81. 06:56Let's say that it's PostHog.
  82. 06:58So, it reached out to the PostHog server, got back the textual response.
  83. 07:01It's factually correct, but it's useless.
  84. 07:03I mean, how do I even take that and understand quickly what's going on?
  85. 07:08I would have to read, which I don't want to do.
  86. 07:10Uh and it's pretty challenging.
  87. 07:12Uh but luckily, because both PostHog server and Claude as a host support MCP apps, I can just say, "Show me."
  88. 07:21And now, instead of getting that block of text, I can actually get something useful, uh which is this interactive um widget that you would get, you know, on the PostHog
  89. 07:32uh uh server.
  90. 07:33And when you have that, you can at a glance see what's going on.
  91. 07:36And as you can see, it's branded PostHog.
  92. 07:39So, you're actually getting the PostHog experience within ChatGPT or Claude, etc. Uh but it doesn't really end there.
  93. 07:46As we said, MCP apps is also like an interactive photo call.
  94. 07:50So, not only can I see and and interact with it, I can also do stuff like ask him to explain what a funnel is.
  95. 07:58I might not even know that.
  96. 07:59So, again, instead of getting that huge wall of text explaining what a funnel is, I can just get this generative UI answer from Claude, which uses MCP apps.
  97. 08:09It streams like the HTML inside, and now I can get this nice interactive experience of learning.
  98. 08:15And not only is it visually nice and helps me understand, but it's also fully interactive.
  99. 08:22And when we say interactive, it actually means that clicking it would help me communicate with the host.
  100. 08:29So, let's say that I want to understand like a particular step in the funnel.
  101. 08:33Uh I just go and I click on it, and since it's an MCP app, it can send a prompt back to the uh model and say, "Okay, explain this specific step to me."
  102. 08:43And I can advance the flow.
  103. 08:46Uh so, this is a like an example of of how that uh looks.
  104. 08:51So, how does it actually work?
  105. 08:52If you look at the architecture of it, uh so, we started by prompting.
  106. 08:57So, we type something in.
  107. 08:58Uh we asked for the funnel information.
  108. 09:01A tool call went out.
  109. 09:02Since our server supports MCP apps, that tool call is actually linked to a resource.
  110. 09:08And if you look at the uh code here, then, you know, it's it's a it's just a resource with a uh some prefix.
  111. 09:15Uh we take that.
  112. 09:16It's pretty simple code.
  113. 09:17I could just add the but it's still the the resource with the HTML, and you're done.
  114. 09:21Uh that resource is then um consumed by the host.
  115. 09:27In practice, it's usually consumed beforehand, like it's preloaded.
  116. 09:30Uh but imagine that it's just consumed in real time.
  117. 09:33That same HTML then passed to the host that also supports MCP apps.
  118. 09:37MCP apps basically if you look at the MCP UI SDK, just a React component or a web component that just accepts that resource plus a callback which is how we implement that communication protocol as I said earlier.
  119. 09:51And renders it in a sandbox.
  120. 09:54So, like we said, not only is it presentational, I can click.
  121. 09:57So, what happens when I click?
  122. 09:59So, we click on it, it sends back through that callback the event all the way up.
  123. 10:04The model takes that event and then it can send out a tool call or call a resource or anything else that's completing the agentic flow.
  124. 10:14And this architecture actually brings a new philosophy or a new vision to the web.
  125. 10:18So, instead of us thinking of the web as tabs or services that we need to consume using a browser, we're now consuming it using our own personal assistants, right?
  126. 10:30What does it mean?
  127. 10:31It means that if I want to accomplish a task, for example, plan a um anniversary.
  128. 10:36So, up until now I had to open 20 tabs in the browser and I had to try to convey my intent to each of those services.
  129. 10:43And by saying conveying my intent, it means that I have to interact with the dashboards or the UIs of those companies.
  130. 10:49So, just to plan an anniversary, I need to convey my intent to Google Calendar and Amazon and Booking and Booking again and Amazon again and all and I don't need 99%
  131. 11:00of the UI that is shown there because this UI doesn't know me.
  132. 11:03It doesn't have the context on me.
  133. 11:05What if we could just take these UIs and just break them into atoms?
  134. 11:10And those atoms can be composed by my own personal assistant, right?
  135. 11:14Because I don't need the the UI.
  136. 11:16I need those atoms.
  137. 11:17So, if we can take these atoms and have my cloud or ChatGPT or OpenCloud just use them using MCP UI, we can have this flow.
  138. 11:27So, my proactive assistant can say, "Yeah, I know.
  139. 11:30I see that you have an anniversary coming and instead of just showing me data from Google Calendar, it can display a Google Calendar chunk.
  140. 11:37Now, for me it's good because I know Google Calendar, I trust Google.
  141. 11:40For Google it's good because it maintains their brand and identity and for the host it's good because they don't need to develop these capability themselves.
  142. 11:48And it goes even deeper because if I'm interacting with Amazon, instead of Amazon being reduced to just a list of items or or text, I can see Amazon.
  143. 11:57I can I can know that this is this is Amazon and I can complete my entire flow without even leaving my assistant.
  144. 12:05And this is the agentic web.
  145. 12:07This is how we're going to consume the web because my assistant will have the context on me.
  146. 12:12It It will know to pull the the map from booking.com.
  147. 12:15I don't need to know that, right?
  148. 12:17So, this is going to be the shift that we're going to see very soon where websites are going to shift into small chunks of UIs inside inside personal assistants.
  149. 12:25Um
  150. 12:26[snorts]
  151. 12:26and with that come new interaction mindset because um if I click on something in the Shopify's MCP app, then Shopify doesn't control my journey anymore.
  152. 12:35The host does.
  153. 12:37Um and no application will control the user journey anymore.
  154. 12:39So, Amazon won't be able to know to see my flow.
  155. 12:43It everything will go through the chat for auditability.
  156. 12:46Um and MCP apps actually standardizes it by defining this three level of control over the user journey.
  157. 12:53So, an app can notify the chat that something happened or an app can actually ask the chat to run a prompt and and releasing all responsibility to the chat.
  158. 13:02So, MCP apps actually standardizes it and this is the new software flow, the new flow of interaction that we're going to see between applications, the chats, and the users.
  159. 13:13Um in 2026, we had we had an amazing year of standardizing MCP UI and 2026 is going to be the year where it's going to be a global standard for UI.
  160. 13:23Yeah.
  161. 13:23But, it's still evolving.
  162. 13:25There's a lot of stuff going on.
  163. 13:27Even in these past few months, these are some of the things that are already in or already contributed or proposed uh, by the community.
  164. 13:34Uh, so you still have a lot of time and a lot of room to influence how this future will look like.
  165. 13:39Uh, so you can go to X apps.
  166. 13:41Uh, that's the official SDK and spec is also hosted there.
  167. 13:45It's under the official model context protocol uh, repository.
  168. 13:48There will be a QR code later.
  169. 13:49Uh, so you don't have to uh, to uh, photograph it.
  170. 13:53Uh, and also um, the the cool thing about using X apps in particular is that because it's maintained by us directly, uh, all changes to the spec are immediately
  171. 14:04uh, reflected in the SDK.
  172. 14:06So, if you use that SDK, then you automatically get all the new stuff out of the back.
  173. 14:10Uh, these are some of the issues that we have.
  174. 14:13So, please feel free to come and uh, contribute.
  175. 14:16So, what's next?
  176. 14:17Um, there's a bunch of stuff coming up.
  177. 14:20Uh, the first thing uh, that we get a lot of uh, uh, of asked for is kind of reusable views.
  178. 14:27So, if you have uh, um, companies like Autodesk that have really heavy apps like they have in, you know, the entire 3D render there.
  179. 14:36They don't want to keep re-rendering that over and over again because it just it takes time, it's inefficient.
  180. 14:42Uh, that is the way that we had to do it uh, for the MVP.
  181. 14:46But we are working on thinking of maybe we can pass some identifier from the server uh, in a way that would help the model actually keep updating the same view.
  182. 14:56Uh, the other way to do this is
  183. 14:59Um, app tools, which is something uh, if you've heard of web MCP, which is Google standard of how agents will interact in with web views.
  184. 15:07So, in MCP us, we actually standardize it into app tools.
  185. 15:10So, up until now we saw the flow where users does something in the app and the app talks to the host.
  186. 15:17But what if the host or the chat wants to speak to the app?
  187. 15:20If the user writes something, uh, fill out this form for me and the chat will fill out the form for the user.
  188. 15:25So, MCP apps actually standardizes this this flow which we call view tools.
  189. 15:29That's actually that's in the spec right now.
  190. 15:31It's also it's going to be released very soon.
  191. 15:34And we're working on this generative UI spectrum where you have predefined UI.
  192. 15:38That's MCP apps.
  193. 15:39That's like the black box iframe that renders all trace UI in that example.
  194. 15:45But you also have other things on this spectrum like declarative UI like JSON render or A2UI.
  195. 15:51These specs that say yeah, the the app just returns an instructions on how to build the UI, but the chat will actually build the UI.
  196. 15:58And you have fully generative UI on the other end of the spectrum.
  197. 16:01And if you know cloud apps, yeah, MCP apps is agnostic to the way the UI is generated.
  198. 16:06And if you know cloud apps imagine feature where you can just ask cloud to generate a UI for you.
  199. 16:12That's actually based on MCP apps.
  200. 16:14So, this is an MCP app behind the scenes, but it supports generative UI.
  201. 16:18So, we're working on interoperability with those other standards.
  202. 16:22And actually just a few days ago we released a guide on how to do A2UI versus a generative UI standard and MCP apps which is the standard.
  203. 16:33How to do interoperability.
  204. 16:35How can a server can write A2UI and ship it to Gemini, but also wrap it as an MCP app to ship to ChatGPT and vice versa.
  205. 16:42An MCP app is supported everywhere.
  206. 16:44So, it can run everywhere.
  207. 16:46If you build it once, it runs in Libra Chat which is an open source MCP app supported in ChatGPT.
  208. 16:52That's the same app that you're seeing the same code base that runs in in both which is pretty cool.
  209. 16:58Yeah.
  210. 17:00Yeah.
  211. 17:01So, this isn't just a technology or a cool feature.
  212. 17:04This is an entirely new way to distribute applications.
  213. 17:08So, if you look just a few months back then someone said that ChatGPT in particular has 800 million weekly users which is 10% of the entire world population.
  214. 17:20That's insane.
  215. 17:21So, if you think about the web in general, it took around 13 years to get to that number of users.
  216. 17:27So, if you look at that and you think that in the last few months we actually had a growth of over 1 billion.
  217. 17:34Just that we have like 170 times the total addressable market of the Apple App Store when it launched.
  218. 17:42So, MCP apps are everywhere.
  219. 17:43So, actually to list them, it is called cloud, open AI, etc. It's already there.
  220. 17:49So, how do you get started?
  221. 17:51You can clone those you can go to the X apps.
  222. 17:55As a host also go to X apps or the MCP website.
  223. 17:59Please visit the official repo.
  224. 18:01[laughter]
  225. 18:01The X apps repo to get involved.
  226. 18:04And yeah.
  227. 18:05So, embrace the new web.
  228. 18:06It's awesome.
  229. 18:08With MCP apps you can write once and run it everywhere.
  230. 18:11And the future is looking bright.
  231. 18:13Not quite Travis, but with MCP and MCP apps we're close.
  232. 18:17And come talk to us afterwards.
  233. 18:18Yeah, thank you.
  234. 18:36[music]