How to Get Your Org to Adopt Coding Agents (Without Shipping Garbage) — Eyal Blum, Figma
https://www.youtube.com/watch?v=5Bn0xro2ol8
AI Engineer · 17 min · 148 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:01] [music]
[00:12] Good afternoon.
[00:13] My name is Alon Blum.
[00:15] I am a software engineer at Figma.
[00:17] And in my talk today, we're going to talk about how we've adopted or are adopting agent into our workflow at Figma while maintaining high quality for our code base.
[00:31] So, as you may know, Figma is the browser-based editor where design and engineering and now AI agent collaborate together to ship code.
[00:43] Uh this Figma has pivoted very strongly from being a traditional tool to an AI-first tool.
[00:52] But in this talk, I'm not going to talk about our product.
[00:54] I'm going to talk more about our internal organization and how our engineering org has been adopting AI agents.
[01:04] Um what we we found internally is both organizations, companies, and individual there's kind of a three-act process of AI adoption.
[01:16] You start with picking up something, whether it was a lot of the people in this room who have been doing using our AI pal and have been using AI for a while and they picked up something and got some simple things to work very well.
[01:30] 10x faster.
[01:32] Then you start applying those same practices to bigger problems and AI fails pretty badly at that, gives you bad stuff, lots of bugs, and the trust that you build breaks down.
[01:45] And then from that point, you start building the real skill, which is learning how to use AI correctly and put the right guardrails and the right prompting and the right context and all the stuff that we've been talking
[01:58] all day about here in all the talks in order to actually build a real scale.
[02:03] And one thing that is happening internally as we we adopted whether teams or individuals the adoption is uneven.
[02:13] We have teams that are very AI forward and have already transformed their entire workflows and then we have teams that are still experimenting in the earlier act and or have lost confidence and they all need to work together in order to ship
[02:28] our product.
[02:29] Um So, they need to coexist in the organization and we need to find a way to support them and while bringing on everybody along for the journey and getting everybody to the third act of the story.
[02:44] Aside from that main friction point, we have also noticed other friction points that happened as we adopt AI.
[02:53] One thing that we've heard a lot from developers and managers have have been noticing is that reduced developer agency causes um engineers to lose some of their job satisfaction.
[03:04] So, if a lot of people used to take a lot of pride and enjoyment in writing code and getting into the flow and a lot of people feel like that's been lost or they're losing a lot of that
[03:14] element and getting into more of a prompt cycle where they just wait on output from AI and then speak to the AI that like not as much fun as they used to have and they're getting burned out.
[03:25] Um we've noticed another interesting thing.
[03:27] It's actually our best engineer, the one that hold all their contacts in their brain.
[03:31] Um they end up getting out of the burden.
[03:34] What ends up happening is they they know where all the pitfalls are.
[03:37] They are like holding together with with like their mental duct tape all the places that agents are not working well and they're preventing all the really bad stuff from coming in or all they they have all the
[03:50] institutional contact that have never written down in their head and they get so much burden and and become bottlenecks and gets really frustrated.
[03:58] So, they actually end up being slowest to adopt because they see all the problem uh first hand.
[04:04] That's another big big issue that we've seen.
[04:08] Um and this one I'm sure everybody can resonate or in Sorry, I'm sure everybody here will resonate.
[04:15] Um that all of a sudden all the design docs and all the Slack messages you know, this the emails have gotten three or four times as long and we've gotten two or three times as many emails
[04:26] and they say basically as much as they did before.
[04:29] So, communication has gotten quite inefficient and some of the markers of like what is high quality and important things versus not so much high quality um has become challenging to navigate.
[04:42] Um so, I'm going to spend uh the next few minutes talking about some of the lessons that we've learned and how we've been trying to apply this.
[04:50] This is a journey.
[04:51] We have not come out through the other end, but we've seen some really interesting progress along a lot of these lines.
[05:00] Um I think this uh a lot of the speakers here have touched upon this, but investing in verification is probably the highest value thing we can do in our code base.
[05:10] Um anytime that we can lift a left shift anything in our workflow from a human needing to do it to an agent being able to verify it.
[05:20] So, for example, when uh Playwright and MCP came out, instead of having humans navigate the code, now the agent can explore the code.
[05:30] That was a big win unlock for productivity in a lot of our team.
[05:34] That's really That's always a a big win for us.
[05:38] The other thing is um it's even better if when you find something that the agent has found to be useful, take the time to take that and encode into a deterministic
[05:50] flow.
[05:51] A deterministic flow that can be easily repeated is saved on tokens, save on time for the and then it also you also know that you're using the the LLM
[06:00] when it needs to reason, but when you have something that is already known and basically can be encoded into a test, spending that time always always pays dividends.
[06:11] Um And another tip, if you tell your scale your agent to write the code that you're writing um like at the red to green to red to green at the TDD
[06:23] style, it almost always gives you better results because you set a goal, then you tell the agent to strive toward that goal, it will almost always give you better results than writing the code and then writing the test afterward because then it will fit the test to the code rather than fit the code to pass the verification
[06:39] criteria.
[06:42] Um this is the testing pyramid that uh can't the classic testing pyramid from the previous uh just when you think about the testing themselves, which you had the end-to-end test and the integration test and the unit test.
[06:55] This is very similar.
[06:57] Move as much as you can down to the deterministic analysis where that's linting, the compiler, um the unit test themselves.
[07:05] Whatever that can come be covered easily, you can have engine agent do reviews on it based on on criteria, so um architectural standards that that have been easily encoded into the code base, you can move into the agent.
[07:20] And then only at the very top you need to have some sort of human review, which is usually around the functionality and this is the right thing to build.
[07:28] That like only leave the human to do what the humans need to actually be involved in.
[07:34] Um Another really important thing is the planning versus prompting.
[07:39] This is really tied into the giving agency back to developers and finding a replacement to the craft of writing code.
[07:48] Um spending a lot of time writing the plan and then sending enough to the agent basically as a as an implementation that can be done automatically is something that we find to
[08:01] really kind of reintroduce the joy of of building back into the process.
[08:07] And so it's not uncommon to spend a week writing a very detailed plan, making all the decisions, flushing it out, iterating, sending it out to teammates to review.
[08:18] And then only when it's ready and you've flushed out all the decision, you can send it to the agent.
[08:22] The agent will um send it back to you when it's implemented.
[08:26] And that that has been really successful also in accelerating and also really restoring some of the joy into the development process.
[08:38] Uh so what makes a good plan?
[08:40] Um really important to start with a why at the top.
[08:44] It really helps preventing agent drift.
[08:45] If you have like a bold big section of kind of like it when you write a design doc, you want to have the executive summary.
[08:51] Put that in there for the agent.
[08:52] Otherwise, they'll start drifting over time and make sure that the agent don't go back and change that because they feel like it.
[08:59] Uh so we start with a why.
[09:01] Make sure that the plan can be broken down into small parts that can each be verified independently.
[09:08] And my personal way of knowing what is a good size would I want to review that the PR that will correspond to that part?
[09:16] If it's going to be too big for me to want to review in one sitting, it's kind of like the test is I'm going to get need to get a cup of coffee before I read this.
[09:23] That means it's too big and I'm going to want to have it broken down into pieces.
[09:27] And then I make sure that each part can be validated independently cuz what I don't want to have is have five stages and then the first one is written but not validated, and then everything else is is built on top of all the assumptions.
[09:41] So, having kind of a validation gate or an exception criteria for each phase really helps and make the plan uh resilient to drift.
[09:51] And all and and there's all kind of technique on how to manage the contacts and doing a a software factory on top of that.
[09:59] But once you have the plan, you can use whatever loop uh you want or whatever workflow you want in order to implement it.
[10:07] Uh this is a screenshot that I randomly picked up a plan, but this is what I usually look for.
[10:12] The executive summary at the top, the phases break it down, and then each one of them I would go into lots of details so that I can just fit it into a sub agent,
[10:20] and the sub agent can independently work on that and not have to worry about it.
[10:25] Um that's that's it.
[10:26] There are other workflows that would work or other structure to the plan.
[10:29] I find that part of the things that great about uh AI workflows is that everybody can set up the thing that works best for them.
[10:38] Oh-oh.
[10:41] No, thank you.
[10:43] Everybody can very easily set up the workflow that work exactly for them for them.
[10:47] So, there's diminishing return in trying to centralize everybody on one thing, but as long as it works for their flow and other people can iterate with them, I find that it generally works very well.
[10:59] And this is just an example kind of a brag of like this is uh could be a result from a plan.
[11:04] Um there are probably 20 PRs here.
[11:07] Some of them would be maybe 10 lines, and some of them would be 100 lines.
[11:11] There's probably nothing bigger than that, and that allows us to This is in the pre-AI world, this plan probably worked in that for a week.
[11:20] I aligned with the other with three other teams for another week on that, and then I just sent it to an agent to implement overnight, and it came back.
[11:27] This is probably from two plans, not one, but it's it's basically six weeks of of coding work just It's um only took 1 week, so that's where I got the 5x
[11:39] speed up.
[11:40] If I include the review cycle at the end that we always have to remember.
[11:45] Um moving on from planning back to the issue that we had with the skeptics and the people who are burdened with the most work, make sure that you bring them in and take their feedback really seriously.
[11:59] They're skeptic because they're seeing the the way you are lacking validation, where your tools fail.
[12:05] So, and their feedback is basically the road map of how to improve your agent interacting with the code base.
[12:12] So, just make sure to bring them in rather than trying to um figure out how to make them use the AI.
[12:19] Just let's have them be in charge of the road map to make AI safe your organization, and they will come along once they see that that the improvement that they're making actually making their life better.
[12:33] Um and as you can see, they'll not be shy about telling you what you need to fix.
[12:37] This is Latin hour sitting with a bunch of people, and this is the result of brainstorms.
[12:45] Um another thing that's been really helpful with my team specifically, and we're working to adopt it in the broader organization as well, is to make sure that you have an attention-aware communication.
[12:58] In the age of AI, human attention is a scarce resource.
[13:01] I think I've heard it for multiple talks, and a lot of people have have come to the same conclusion.
[13:06] You can't get more human attention.
[13:08] So, where you spend your time and what you're reading is really becomes really important.
[13:13] Um so, since it's such a scarce resource, marking what was generated by AI versus what was written by human is really helpful to know how much time you need to spend reading this, and how much slop can you expect in this part of the communication?
[13:29] Um and that can building a new culture around that self-communication that really helps.
[13:36] Um So, for example, um the team that team that I work with, we've decided we always every PR description will start with something like that, something that I wrote by hand.
[13:47] It could be very short that I describe what this is in code and what this is doing.
[13:51] And then the AI description is going to come after that, which is I will probably read it.
[13:55] I will probably edit it to remove uh some wrong things, but they didn't write every line here, so they should be more suspicious and they should pay more attention to what I wrote in the top and they should override it.
[14:06] Things like that in Slack, in email, it's like leaning into the fact that everybody knows that you're using AI to to craft your communication, but just let them share about it, tell them what they should read and what
[14:19] you they should pay less attention to.
[14:22] And I remember early on, maybe like earlier in this year, I tried to I had some senior engineers in our org that had kind of were very much AI skeptic and I tried to reach out to them to see what was the problem, what was going on.
[14:39] I said, "I tried to run an analysis on some of the PR comments that you've run."
[14:43] And obviously I used the AI to do that.
[14:46] And then I didn't distinguish very clearly what I wrote versus what they what AI generated.
[14:53] And they got very upset.
[14:55] They're like, "Why is sending I did not expect somebody um that I respect this much to send me something that's clearly this sloppy."
[15:02] And then like I I took immediately like I apologize.
[15:06] I realize I should have marked it clearly and marked my intention like this is what I wrote.
[15:10] This is what the AI wrote and I need your feedback on that because I don't have the context to know if it is sloppy or not and that's what I'm asking you for, so
[15:17] lesson like that and change the culture is just as important as some the engineering challenges that we've been facing.
[15:28] Um another thing that's really helpful around the adoption is um as you progress through adoption, there's a lot of very fancy tools and a lot of very fancy workflow that we've
[15:40] we've been implementing, but one of the really effective thing is just letting people use the AI where they're at.
[15:47] So, uh it help it really helps normalize uh the use of AI for everyday tasks and it helps reduce the friction.
[15:55] And really one of the most powerful thing is being able to tag an agent in the Slack message with somebody and they can you just do this for me?
[16:03] And have the agents to close the loop in the thread.
[16:06] Um that that's kind of thing is really powerful.
[16:09] And then you can go on top of that and have all this thing automated and do all kind of fancy things, but if you have a new conversation with somebody who's not fully bought in and then you can tag it in a non
[16:21] like non-passive-aggressive way.
[16:23] You can tag it and say, "Let's try to see if the agent can get it this time."
[16:27] And they close the loop and if it's a good experience, that really helps people try it out on their own in other cases.
[16:36] And our journey continues.
[16:37] We're still learning even though we're shipping AI externally, our AI adoption um we're experimenting with with so many things all the time.
[16:47] Our automation story is not uh fully there yet.
[16:51] We're still trying trying to figure out when we should use how we can use cloud agent effectively given all the dependencies we have for some of our bell system.
[17:00] And so we are continuing to learn.
[17:01] It's a culture shift, it's an engineering shift and I don't know about you, but for I've been I've been working in the valley for the last 15 years and this is the biggest change by orders of magnitude of everything that I've seen
[17:14] in term culture and technology.
[17:17] So, um we're all here together and we're all figuring it out and that's that's what I wanted to talk to you today.
[17:23] Thank you.
[17:24] [applause]
[17:40] [music]