Generative UI... in Python? — Jeremiah Lowin, Prefect
https://www.youtube.com/watch?v=Krzs8GeiWTc
AI Engineer · 17 min · 210 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:12] Good.
[00:13] Um, thank you all for for coming out.
[00:15] Um, I'm going to talk today about one of the weirdest pieces of software I've ever written.
[00:19] It's sort of on the edge of a whole lot of stuff I've been putting putting forward into the world.
[00:23] Um, so join me if you will.
[00:25] We're going to try and have the most reasoned approach to to a very strange thing that agents and MCP and other things have enabled.
[00:32] And so to begin, I want to talk about MCP apps.
[00:35] I don't know if any of you were able to join any of the other talks earlier today.
[00:39] Maybe the one that um Edo and Lead just gave maybe an hour ago.
[00:44] Just a show of hands.
[00:45] MCP apps familiarity.
[00:49] Okay, this is probably the best crowd I've ever given this talk to actually.
[00:52] So that's that's fantastic.
[00:54] Um, for those that didn't put their hands up, MCP apps is an extension of the MCP protocol that was introduced I think in January of this year.
[01:01] And the idea is this is a typical request response cycle for an MCP um tool.
[01:07] The user makes a request to the agent.
[01:10] The agent in turn decides to use an MCP tool that's hosted on an MCP server.
[01:15] A tool result comes back into the agent's context and the agent chooses to form some response and send it out to the user.
[01:23] And so fundamentally MCP servers are these fantastic ways of adding uh functions and business logic to your agents but never a direct connection between a user and the MCP server.
[01:33] It always goes through the brain of the agent and more importantly through the context window of the agent.
[01:37] So MCP apps are an extension of this which allow us actually to bypass the agent and instead what happens is the following.
[01:44] The user requests something from the agent.
[01:46] The agent uses a tool, but instead of that tool request going back to the agent, it is sent out to the user and it's sent out as HTML, CSS,
[01:54] JavaScript.
[01:55] It's a full UI and it can be whatever you want it to be.
[01:58] And so the idea is you have this way to make to basically put the internet into your agent, so to speak.
[02:03] You can ship any custom branded useful UI that you want.
[02:07] You can let the user have any interactive experience that they want.
[02:11] And then the user, as you can see in the diagram, the user now can interact with the application.
[02:15] they can use the tools, they can send information back into a backend host on that app and really get a full experience.
[02:21] You can imagine booking a table at a restaurant or um changing your seat on a plane or interacting with a schedule for AI engineer.
[02:29] There's a lot of things that you can do as a user now where the agent facilitated it, but you are going to interact as a human.
[02:35] And there's an extension coming now.
[02:36] This is going to come out in um in the July MCP release where the agent can actually interact with the app as well.
[02:43] And this will tee up some really interesting use cases we're not going to talk about today, but you could hypothetically play a game of chess against the agent now in a visual app where you make a move and then the agent interacts with the app as well.
[02:54] And so I think that's going to open up a whole new world of possibilities.
[02:59] Now um some of you may know a framework that that I'm the author of and my company maintains called FastMPP.
[03:05] Um fast MCP is one of the most popular ways of building MCP servers.
[03:08] And so whenever new cool things come to the world of MCP, the first thing I wonder is how can I deliver this to our users?
[03:16] And one of the most important things I have to share with you about our user base is that they're mostly Python engineers.
[03:23] And so and so this is a little bit of a problem when we want to deliver frontends and UIs because how are we actually going to do that?
[03:31] And this is the point in the talk where I reveal that I don't remember what the next slide exactly is.
[03:34] So we're going to take a peek at it.
[03:35] Nope, we're going to come back.
[03:36] Um, we we have we have a challenge now.
[03:40] Uh, how are we going to have have Python engineers build UIs that are best practice, that are interactive, that are beautiful, that are useful without pretending that we're going to do something silly, something that's been tried, and jam all of the front end, all of the ecosystem, everything into Python in some sort of like weird compromised
[04:00] haphazard Frankenstein of a system.
[04:02] And so I really struggled with this.
[04:04] I need I really need I feel an obligation to find a way to deliver this, but I I can't I can't pretend we're going to ship React and Python.
[04:12] It's not going to work.
[04:13] And so we thought pretty hard about who are our users in the fastmcp ecosystem.
[04:17] Who are these Python developers who tend to be in enterprises?
[04:20] What are they doing and what do they need these UIs for?
[04:22] What do they need these MCP apps for?
[04:24] Um what they don't need is consumer-grade custom UIs that are fully branded.
[04:30] That that's not what these folks are doing.
[04:32] what they are uh primarily charged with is sharing information throughout their throughout their organization for collecting information throughout their organization and so it changed the nature of what we expect them to do within MCP apps framework and that constraint became really useful so fundamentally
[04:48] we expect that they're going to do things like build tables they're going to collect information through forms and they're going to want to share charts and so fundamentally with this constraint
[04:59] we can introduce a piece of software that we open sourced a few months ago and has been surprisingly popular among this crowd called prefab and it's a scoped UI
[05:08] building framework for the purpose of delivering UIs through an agent for the set of purposes that I mentioned a moment ago.
[05:16] So this is a hello world card.
[05:18] You might see this in any front-end framework, literally anyone.
[05:22] It'll have something that looks like this and it's on their website and you you type your name in and it it updates live.
[05:27] But of course, the weird thing about this one is that the code that generated it is entirely written in Python.
[05:33] And so I hope that you're feeling what I feel when I look at this, which is a really weird combination of like, yes, that's cool, and this really freaks me out.
[05:43] The yes, that's cool, comes from the fact that I think there's something about this code, even if you can't see it up close, I can make it a little bigger.
[05:49] There's something about this that like kind of makes sense.
[05:52] You can see the structure of the of the of the UI in the code, but there's also something about it that's obviously alien and a little bit a little bit odd.
[06:00] And we come to this conclusion when you when you feel that when you look at it, which is that when you compose a front end in Python, it's actually starts to feel good as long as we scope the challenge right.
[06:10] We are not trying to build a front end from scratch.
[06:13] We are trying to compose a front end from a bunch of worldclass well-designed components.
[06:18] And that's how we keep the guardrails and that's how we keep the user in mind.
[06:22] The user here is not trying to do something arbitrary.
[06:24] They're trying to take a well structured front end and put it in front of um whomever they're delivering it to.
[06:29] And so here's a little quick tour of that DSL.
[06:32] Um primarily we're using context managers.
[06:34] For those of you who do know fastmcp, you know that arguably you could reduce fastmc down and say the core innovation of fastmcp is that we used a python decorator to build an entire MCP server.
[06:44] So if you want to take the same reductive approach to prefab, you could say we use a context manager to build an entire UI.
[06:49] And by nesting components with as context managers as you see here, we are building up the exact same structure in the UI.
[06:58] It feels very natural when you read it.
[07:00] You can see how things are structured.
[07:02] Um each element of the UI, each component, which is a beautiful shaden component when it's rendered, as you can see here, is a class that you instantiate.
[07:10] You can parameterize it.
[07:11] you can pass it stuff like CSS classes and and make it look however you want.
[07:16] And then the last thing which we're not going to have enough time to really explore today is these reactive variables.
[07:20] Um I'll show you a demo of those in a moment, but essentially we have a full way to um create client side interactivity and bind data between components that allows you to build these really rich experiences
[07:30] again without having to go fully into the JavaScript world and leave an ecosystem that my user base at least is extremely comfortable with.
[07:39] Um, and this is the pipeline that prefab is essentially exposing.
[07:42] We use a Python DSL that I just shared with you.
[07:44] We use that to build a declarative representation of a UI that then gets serialized into a JSON protocol.
[07:52] And that JSON protocol is ultimately rendered by a React app which is hosted as the actual MCP app.
[07:58] And so this is going to open up a whole lot of possibilities for us that again I'm going to show you in just a second.
[08:02] But the key to this whole thing is the JSON in the middle.
[08:05] The Python is actually an accident that I discovered after the fact because it was a weird idiosyncratic thing that I wanted.
[08:09] The point of this was can we create a serializable representation of a UI and that's the JSON protocol again.
[08:17] And because it's serializable, I can generate it from an agent.
[08:20] I can send it to an agent.
[08:21] I can generate it as a human and ask an agent to modify.
[08:24] There's all this cool stuff that happens because of that intermediate representation in JSON.
[08:28] And then when the when the Python DSL just fell out of this and was really beautiful and easy to use, I kind of felt like we had something So
[08:35] uh we have these docs and sort of to prove the point this was another constraint we took on this is I don't even know what this is this is a doc these are the docs for the data table component in prefab I think there's
[08:45] 130 or 140 components that we ship that you can compose into an arbitrary form the docs for prefab are 100% rendered in prefab so the data table that's here in the basic usage it is live rendered in prefab the Python code you can see it sneaking in at the bottom of the screen that Python code is being
[09:03] rendered live by the renderer to generate that.
[09:05] If you want, you can take any example in the prefab docs, you can click a link, pop them into the playground, and you can edit the code live, the Python code live, and you will see the UI update.
[09:15] And again, this is super weird.
[09:16] If you're feeling a little uncomfortable about this, that is that is okay.
[09:20] It makes a lot more sense when we constrain the problem.
[09:22] And remember that we're composing a UI rather than building it.
[09:25] So, I want to bring this back to the thing I opened with now, which is MCP servers and more specifically MCP apps.
[09:31] You are welcome to use prefab for any kind of front-end problem you have.
[09:35] My team has started using it for small interactive data apps and things to explore.
[09:39] They've been building presentations with it.
[09:40] We ship a a dark mode theme that honestly looks kind of like the one I'm showing you right now to make slides and presentations.
[09:47] You can do a lot of stuff with it.
[09:48] But the reason we built it, the use case that it is satisfying is for MCP servers.
[09:52] And so um I want to give you a quick tour of three ways that you can use it, three increasingly sophisticated ways that you can use it in your MCP server.
[10:00] The first is to build an interactive tool.
[10:02] As I showed you at the beginning of the talk, typically an MCP tool is something your agent calls and the agent gets the result and you don't get to interact with it at all.
[10:10] So what's the easiest way that we can advance that that interactive functionality?
[10:15] I'm going to show you here.
[10:17] This is a fastm tool.
[10:18] It's been decorated with a tool decorator as you can see and it's just a Python function that returns some information.
[10:23] Bearing in mind this information will go to the agent, not the user.
[10:26] If we want to turn this into a fully interactive tool with prefab, we're going to make one change.
[10:32] Instead of returning a Python dictionary at the end, which will go to the agent, we're going to return a prefab component.
[10:38] In this case, it's the data table.
[10:39] These are the this is what I just showed you the docs for a moment ago.
[10:42] And when we return this prefab component, FastMPP will automatically detect that.
[10:47] It will automatically infer that you in fact want to return an MCP app.
[10:50] and it will spin up all the machinery to get the HTML, the JavaScript, the CSS, the render, everything in place so that your user will see a data table.
[10:58] Here's what this looks like in practice.
[10:59] This is using the goose client, which is an excellent one.
[11:02] Um, I asked a server that had the function I just uh showed you, show me the team directory.
[11:07] And what pops up uh this would be better as a GIF.
[11:09] I apologize, but what pops up is a fully interactive data table component.
[11:12] It supports um searching and filtering and sorting and pagionation and all this stuff.
[11:17] And all it is is what I showed you a moment ago.
[11:20] Just return the data table class and all this will be taken care of.
[11:24] We can go a step further.
[11:24] What if in addition to the data table, we want to show a pie chart right next to the data table that breaks down this team directory.
[11:30] As you might imagine, very, very, very similar code.
[11:33] Instead of the data table alone, we're now going to import a grid and a pie chart.
[11:37] And if you look at the bottom, you'll see that we compose both the pie chart and the data table into a grid very naturally with a context manager.
[11:45] And this is the result.
[11:46] we now get a pie chart next to our data table.
[11:49] So this follows a principle that we really try to hold in a lot of our software at Prefect, which is one line of code, one big noticeable change.
[11:55] We try to keep that complexity incremental.
[11:57] And so this satisfies a lot of things that I think are really important about frameworks and DSLs.
[12:02] Um, this is very quickly because we won't have time to go into it.
[12:05] This is just an example I threw together and recorded of fully client side interactivity where all of these controls are linked.
[12:11] Uh, stuff's updating, text is updating, values are updating.
[12:15] No JavaScript was written.
[12:16] This is just a couple of classes composed that all have the same attribute assigned.
[12:21] So they all work together automatically.
[12:23] Um, oh, and I did throw in a quick code example of what that looks like.
[12:27] We have a class called RX, which as you may guess stands for reactive.
[12:30] If you use these reactive variables, you can just reference them anywhere in your code.
[12:34] You can format them.
[12:35] You can you can make them the name of something and it will automatically compile into the correct uh JavaScript implementation.
[12:42] The second thing that we can do is a fastmcp app.
[12:44] So if an interactive tool is sort of a oneshot here's a user interface and you can interact with it in the client a fastmcp app is a full application
[12:53] with a backend and in this case the MCP server is going to be the back end.
[12:57] We don't have time to go through a full worked example in this session but here's what the code looks like just to give you a sense of the ergonomics.
[13:03] We're going to write a class which is our fastmcp app and then we're going to decorate at least two uh functions with app.ui UI.
[13:11] That's the entry point that's going to return the prefab components that form the base UI of that application.
[13:17] And then at least one, I guess this is optional, so zero or more um app.tools.
[13:22] And these are essentially backend methods that you can now reference in the UI.
[13:27] So you could have a button that takes data that the user has entered into a form and sends it to a database using a decorated tool like this.
[13:35] One thing that we use and we ship as a built-in component now in fastmcp is an upload component.
[13:42] So as you can see because only the agent has access to an MCP server you can't simply upload a file to an MCP server.
[13:50] It has to go through the brain of the agent.
[13:52] And so what ends up happening is a lot of people create basically an upload tool on their MCP server forget that the agent has to actually call it.
[13:59] And what you end up doing is the world's most expensive copy paste operation.
[14:02] You give the agent a megabyte of text.
[14:05] the agent retypes it character by character into the MCP and now yes in fact you have uploaded it but it's extremely extremely inefficient.
[14:13] So this is a really good use case for an MCP app where you ask the agent to bring up the app interface you drag a file into it and now the file bypasses the agent and goes right into the server.
[14:24] And we've made that a oneliner like this along with a handful of other um useful tools.
[14:29] And is this a gift?
[14:30] this is not a GIF or if it is it's not rendering but it would look like this in your client in any client that supports MCP apps if you ask the agent I need to upload something it can now show you this and you can upload safely
[14:41] and most importantly cheaply um oh I do have a gift mind know your own slides is a good lesson from this talk so here's the agent is now interacting with a file that I just uploaded that I dragged and dropped um I'll make these slides available later if you'd like to see this or of course it's a
[14:56] oneliner you could try it in your servers uh this afternoon.
[15:00] The last thing that I want to talk about which is sort of enabled by this architecture is a fully generative UI.
[15:06] Um, we're just going to skip and let this play while I talk.
[15:09] So, this is a very simple demo where I asked Claude, "Hey, just I'm giving a talk on this.
[15:14] Just start streaming the most interesting UI you can come up with."
[15:17] And so, it just went.
[15:19] And what it's doing here is we exposed a tool that accepts the JSON uh the protocol serialization of a UI that prefab is based on.
[15:28] And so now as the agent is streaming that information over the wire, we are in real time rendering whatever we've got, healing that JSON and rendering it.
[15:36] And so this was a really cool demo and it was really effective and people like this because now you don't even have to define the UI yourself.
[15:42] All you have to do is use the skill we already ship, share it with your agent so it knows how to write a UI and off it goes.
[15:48] It can make you whatever you want.
[15:50] There are some clients that have built-in versions of this.
[15:52] if they have a built-in version, you may prefer to use it by all means, but this may be a way for you to build your own custom uh approach or limited set of components that are useful to you.
[16:00] Now, a really interesting thing happened when we spun this up.
[16:04] So, as I mentioned, originally the plan was for the agent to send JSON over the wire and have it be rendered into this full React application.
[16:12] What we ended up discovering is that the Python representation of a UI is about 70% smaller than the JSON representation.
[16:21] So we don't do this anymore.
[16:23] When I recorded this demo was streaming JSON.
[16:25] What we now do is we actually stream the Python over the wire.
[16:28] It's executed in a sandbox.
[16:30] It's turned into JSON on the server and then that's rendered.
[16:34] And so this has a dramatic dramatic token efficiency, cost, and latency uh benefit.
[16:39] So it would work exactly the same as when I recorded this demo, but this is just one of those things that we've learned on the fly.
[16:45] And it's really fascinating that the Python representation is just that much more compact and ergonomic than the full um JSON one.
[16:52] So um that's prefab.
[16:54] If you'd like to check it out, if you're curious, if you want to see the weirdest thing I've ever built um along with however other many people, you can see the docs at prefab.pref.io.
[17:02] You can see the full library uh which is on our GitHub here.
[17:06] And this is already fully baked into FastmcP.
[17:08] So, if you're using a recent version of FastMPP, you should be able to install this optional addition, uh, import the components, return them, and start playing with these MCP apps.
[17:17] Thank you all for coming.