Building the Engine While Flying the Plane: Launching the Figma MCP Server — Jesse Lumarie, Figma

AI Engineer · 16 min · 182 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12All right, get started.
00:14Hi, I'm Jesse and I've been a software engineer at Figma for about three years and I'm going to talk to you about how we built Figma's first MCP server in about 3 months.
00:23Um, and in case you don't know what it is, the Figma MCP server is a way for you to send context between production uh code and design and vice versa.
00:31AI tools don't need to build a dedicated integration.
00:34They can just use our Figma MCP and they kind of get started.
00:39So, taking you back to November 2024, long long time ago, Anthropic released the MCP server spec and uh everyone in the world of AI was sort of starting to use it and experiment with it.
00:50Um but outside of anthropic none of the other AI agents or labs were really using it.
00:55So OpenAI cursor VS Code they didn't support it yet.
00:58Um once we got access to the feature in cursor however uh we were able to kind of ideate and understand what it was capable of and we got something a little bit closer to an actual product
01:08and that's where my story begins.
01:10Um I was working on growth initiatives at the time and I saw our internal demo and I really wanted to use it.
01:15I thought it'd be great for non-designers uh to kind of be able to use Figma.
01:19Um, I started creating actually a Figma plug-in based MCP server and I started doing it one day a week.
01:24It was kind of my 20% project that we didn't we didn't really have 20% projects, but I really wanted to work on it, so I did.
01:30Um, and so I got staffed with some other folks um on the team.
01:33We were the MC Peeps.
01:34That's a peep.
01:35It's a candy if you don't know.
01:36Um, they're delicious.
01:37And I I just want to be very clear um I'm going to say I a lot and we there was a big team behind this so it's not just me and they're all fantastic.
01:45Um a few weeks later after we started getting our initial architecture sorted a new version of the spec dropped uh deprecating the support type that we were going to use which was server events
01:56and while the MCP spec was sort of chugging along clients were adding features and support in different paces.
02:02Uh cloud had early support cloud desktop but cloud code you was wasn't really supported uh with all the complete set of features.
02:09OpenAI and VS Code didn't h have support until that spec update and then it wasn't even then uh VS Code didn't go out of uh didn't get to G until July.
02:18It didn't mean that all the features were implemented either.
02:20There are lots of different pieces um and in many cases only tools were supported.
02:26VS Code was truly like the golden client.
02:29Uh they eventually supported kind of all pieces of the spec, but it was it was hard to kind of understand what you were building towards because clients supported so many different things.
02:37[snorts]
02:37But even though we didn't know exactly what the MCP server spec would be supported, we we knew that it would be extremely powerful and a great product uh for us to to utilize.
02:45And so we started building um and so taking a a more highle re recap about a year ago uh we launched our local MCP server.
02:54And what local means it was heavily designed for developer use cases.
02:58You kind of had to know what you were doing a little bit and we targeted developers because they were the first to adopt AI workflows.
03:05they would use a single plop prompt like help me implement this and a developer could pull everything that they would normally get from Figma's dev mode into their coding agent.
03:14This include things like component data, spacing, variables um and from there we just kept adding more and more re read tools like for fig jam for make um etc. But they all shared this kind of uh mutual goal to make Figma context available for develop develop
03:29developers wherever they are.
03:32So, Figma, if you don't know about Figma, um Figma is a canvas.
03:38Um and it's represented as a scene graph in C++.
03:41It's a graph of connected nodes, not unlike the HTML DOM.
03:44And we had a number of different ways we could represent the scene graph.
03:47We had this internal representation which was kind of akin to JSX or XML, effectively converting the scene graph into JSS tags and XML tags and passing those to the agent.
03:57It was abstract and sparse, but it didn't have super rigorous fidelity.
04:01Another option that we had internally was um D2R which is our like way of saying a react tailwind representation.
04:10Uh and the reason we had this is Figma has a sites product and so we already had a way of basically converting the scene graph into HTML.
04:18If you actually copy the output of the Figma MCP today and you paste into like a simple MCP or simple HTTP server, it should be pixel perfect.
04:26Um and if it's not, file a bug.
04:28Uh but um we had a hunch that this representation would be the best one because lots of the models were sort of rldled on this React Tailwind type of code.
04:38Um and we we had a suspicion that it would work really well.
04:41The last one that we kind of considered was just a plain image.
04:43But back in early 2025, 2025 agents weren't great at converting images directly to HTML or CSS or sort of other languages.
04:51And so we kind of use that as an additional piece of context, not as the sole one.
04:55And to give you what what this kind of looks like in practice, on the left here we have a a Figma frame and on the right we have the
05:02React Tailwind code.
05:04Um you can also see at the very top the image can crew meetup uh link.
05:09Uh we also basically abstract out the images within the code or sorry within the scene graph and put them at the top level.
05:16Our first attempt was just passing B 64 data into the code and that was just a terrible idea.
05:21It it just blew up the context window and was bad all around.
05:24um don't do that.
05:25Um we'd also pass an image of the current node to the agent as well.
05:30While the image by itself did not do a good job of converting to uh code, having the code context plus the image actually had better agentic output.
05:40So what do I mean by better?
05:42How did we know what was better?
05:43We tried to do evals.
05:45Um and so we did some sort of very simple eval to start with a mix of quantitative and qualitative data.
05:51Uh, from a quantitative standpoint, we we looked at did it use variables?
05:55Uh, did it use the theming we expected?
05:57Did it use the right spot?
05:58And from the qualitative side of things, does it look good?
06:01Did it make good decisions with incomplete information?
06:04And we spent like two hours grading an eval into an Excel spreadsheet.
06:09And we said, we're never we're never doing that again.
06:10It was awful.
06:11Don't do eval by hand if you can help it.
06:13Um, we had a bunch of toy repos that we kind of created or kind of had folks create for us.
06:19Um and we eventually ended up coding up a web app to sort of help us with the eval which made things a lot easier at least from like a um process perspective.
06:28One interesting thing is Figma um there are Figma files which we are converting there's a lot of open source code out there but there's not a lot of uh open- source code that also has fig files attached and so we had to either create our own or sort of find different ways to make automated systems.
06:44Um, and now we have a um, eval that sort of runs like hundreds of times a week.
06:50Engineers can kick this off and sort of grade against prompt changes um, with LLM judges.
06:55So, we kind of remove the human from the loop where we don't need it.
06:58Um, but having an agent translate a pixel uh, perfect version of code isn't enough.
07:04Um, I mentioned that the React Tailwind version of our output was pixel perfect.
07:09That's really only half the story.
07:10An enterprise doesn't care if it's pixel perfect if it's not using its like battle tested accessible and internationalized components.
07:17At Figma, we already had this concept of code connect which allows you to link design components to components in your codebase.
07:23We needed a way to use this with our MCP server so that an a an agent used the correct components.
07:28For example, this beautiful button here.
07:31Uh this would be a perfect representation if you were to throw that into an HTML server.
07:36But you kind of see two problems.
07:38um if you had a primary button in your codebase, you wouldn't be referencing it.
07:42And that's not ideal if it has accessibil accessibility properties or internationalization properties.
07:47And then second, um you'd eat up the context window.
07:50We we use Oh, this happened last time.
07:53Uh we use React Tailwind to um basically convert things over.
07:58Um but we want to make sure we do it in the sparest way possible.
08:04All right, just going to keep vamping a bit here.
08:07And so picture the same thing on screen.
08:09Uh but now we have like all this React Tailwind code.
08:12It's going to then be converted into sending over uh basically a sparse representation of it via code connect.
08:18Um and by connecting the user's code to the design, we're able to pass back effectively what is a pointer which allows the agent to use the code component leading to our higher fidelity
08:29uh implementation.
08:30So effectively you go from like this big old thing of uh react tailin to the small react component that just says use button component.
08:40Um all right it's going to let you restart it.
08:44Okay cool pause that.
08:55All right I can start talking a little bit about the next bit.
08:57Oh, wait.
08:57You can see this cool thing.
08:59You can ask yourself how I described it well, but yeah, it's basically like a React component that you're able to then bring into your code.
09:05Um, once we felt good about the serialization syntax, we started to look at what an MCB server can be.
09:11And the MCP spec had a lot of great pieces in it, but some features weren't quite fleshed out within clients, and other features we really wish existed.
09:19Many clients only implemented a subset of the spec, and many features were very experimental.
09:24This is the client compatibility matrix from March 2025.
09:28Today, for example, we expose a host of resources to an agent so that it can figure out um how to use our server as well as different help articles within Figma.
09:37Um whereas before we would send that information down with like an air for example, and the agent would have to call uh wasting inference and sort of reasoning to sort of figure out what is actually going wrong.
09:47Um one small part of the spec uh that was missing was server instructions.
09:51I shouldn't say missing from the spec.
09:52it was in the spec, but no clients implemented it.
09:55Um, and it wasn't really highlighted in the docs until Anthropic added a nice blog post uh to sort of talk about it and then some clients started adding it.
10:02And therefore, we would add uh additional instructions into each tool call.
10:07Basically instructing the LLM how to use our server um even though server descriptions weren't necessarily written out yet.
10:15Some other features that we really really wanted were elicitation and sampling.
10:19Um, elicitation, if you haven't heard of it, is a way for you to ask the user a question, take that input, and pass it back to your server.
10:27So, here we have it's VS Code, uh, and basically just asking, you know, what's my name?
10:31And you're able to take that input and pass it back, uh, to the server.
10:35Um, which is interesting on its own, but we thought in combination with sampling, which is unfortunately deprecated, but it's fine because you're able to work around it.
10:44Sampling is a way of having a server query the client's LLM from from our server and in kind of the canonical case was for small queries.
10:54We thought it'd be really useful to combine elicitation and sampling into a single workflow.
10:59We talked about how code connect improves users workflows quite a bit and kind of makes outputs a lot better.
11:05What we wanted to do was ask a user can we map out your code code base for code connections so that our MC MCP server can link them so that the output would be better and reduce the amount of context we send.
11:16Unfortunately though most of the clients didn't implement these features and didn't allow you to properly query the the agent in the context of the codebase.
11:23So for sampling even when VS code supported it you could only really query it as a general agent not specific to the codebase.
11:30But we were able to kind of hack around it using tools.
11:33When you got the context of a particular component or sorry of a particular design in Figma, if we noticed it was a component and that it wasn't code connected,
11:41we'd send down a prompt to ask the user if they'd want to map the unlink component component.
11:46Kind of mimicking elicitation.
11:49If the user said yes, we'd send down another prompt to have the agent scan the code for potential matches, mimicking sampling.
11:56We then service them in a specified format or ask the agent to do so and then have them send it back in bulk to make a bunch of code connections.
12:03Um the screenshot on the right is the MCP inspector and if you haven't used it and you're developing an MCP server, you're doing yourself a disservice.
12:10It's a really great tool and it's open source and great.
12:13Um but the magic in our case was combining these two features because we could ask the user for for permission.
12:21we can have the agent give us those suggestions and we can map them and in the end the users got a better experience.
12:25Um that's pretty great.
12:27Um the last sort of little thing that we did was we wanted to make our output the best it could be and we didn't necessarily know when we were starting you know we had our eval but we didn't know if the react tailwind code would be successful for other types of code bases.
12:40Um and outside of the elicitation and sampling which didn't really work as we wanted there was no way of getting that information from the user.
12:45So we added some optional query arguments to our tool calls for ones like get design context where they would send back what sort of language what sort of framework the user might be using.
12:54This is imperfect uh agents lie but it was at least a signal for us to understand like oh this type of user this felt user may not have had a good experience.
13:03Perhaps it's because our translation layer wasn't working as well.
13:06We have found that that works pretty well but this was kind of our way of verifying that.
13:10Um while we were working towards our first beta, we know we wanted four things.
13:15We wanted to launch quickly.
13:17Um we wanted to have the highest possible bar for our security.
13:21We wanted to respect file permissions.
13:23And we wanted to respect our pricing impact packaging so we didn't have abuse vectors.
13:27And so after the spec changed and introduced OOTH in March 2025, we had to decide whether to keep our MCP server local or sort of switch to the new remote server using streamable HTTP and kind of like work on all the off problems.
13:40Um we punted so until HMR there wasn't this offspec to to build from and we could easily relay off from our web app to our desktop app.
13:49So for folks who don't know the Figma desktop app is Electron and so the front end of it is a web app and we basically just run figma.com in that and then we have an
13:58um IPC uh bridge between the two and that sends it to our node process that allows us to talk to the user's file system.
14:05Um we then sort of expose a server events server in node and that way clients could talk directly locally.
14:12The local story was also really great with enterprises because they kind of like the idea of our data not being sent anywhere.
14:18Um, this architecture was our fastest path to getting something into the hands of users to understand product market fit and what kind of tools and use cases folks had.
14:28Um, we launched the MCP server internally and the reception was extremely honest.
14:34Um, we
14:35[laughter]
14:36uh but we we worked out a lot of the kinks and we started to get some really positive feedback in the community um from from a bunch of nice folks.
14:43Um, but launching this part was just the beginning.
14:46Um we had a lot of improvements that we wanted to make and we immediately started working on the remote server as soon as we launched.
14:52Um clients were on different timelines and we were still trying to figure out you know where we're going but we knew we wanted to get the remote server out so that's what we worked on.
15:00Um in September we launched the remote server.
15:03We ged both servers in October 2025.
15:07Um, and then we started adding read and write capabilities and kind of all these things combined ended up making for Figma one of the fastest growing products that they've ever had, which was not something we expected when we started working on this.
15:19Um, so late last year, I started working on a slightly different thing.
15:24Um, we started to see some research that designers really wanted to shift to writing production code in certain cases and we didn't really have a dedicated product for this.
15:31So I started hacking with a bunch of MCP MCP folks at an offsite and this eventually became something called make in your local codebase uh which is kind of Figma's agent solution for for working on on git GitHub and and local code bases.
15:45Um and the reason I bring this up is only slightly self- serving but it relates to this this next slide.
15:49Um if there's one thing you want to take away from this talk it's that we're so early like this has not been a long time.
15:57The MC MCP spec is only two years old and we're still figuring out the best way to do things.
16:02And then second, um, Figma's done a great job of letting engineers build and figure out, uh, what's next and letting them run with it run with it.
16:10I wasn't staffed on MCP.
16:12I wasn't staffed on our make product, but I ended up helping them be built um, just because I was kind of given the leeway to do so and learning a ton along the way.
16:21Um, that's all for my talk.
16:24I'll be around today and tomorrow, but feel free to reach and happy to talk about MCP, Figma and all that.
16:28Thanks so much for your time.