How to Get Your Org to Adopt Coding Agents (Without Shipping Garbage) — Eyal Blum, Figma

AI Engineer · 17 min · 148 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Good afternoon.
00:13My name is Alon Blum.
00:15I am a software engineer at Figma.
00:17And in my talk today, we're going to talk about how we've adopted or are adopting agent into our workflow at Figma while maintaining high quality for our code base.
00:31So, as you may know, Figma is the browser-based editor where design and engineering and now AI agent collaborate together to ship code.
00:43Uh this Figma has pivoted very strongly from being a traditional tool to an AI-first tool.
00:52But in this talk, I'm not going to talk about our product.
00:54I'm going to talk more about our internal organization and how our engineering org has been adopting AI agents.
01:04Um what we we found internally is both organizations, companies, and individual there's kind of a three-act process of AI adoption.
01:16You start with picking up something, whether it was a lot of the people in this room who have been doing using our AI pal and have been using AI for a while and they picked up something and got some simple things to work very well.
01:3010x faster.
01:32Then you start applying those same practices to bigger problems and AI fails pretty badly at that, gives you bad stuff, lots of bugs, and the trust that you build breaks down.
01:45And then from that point, you start building the real skill, which is learning how to use AI correctly and put the right guardrails and the right prompting and the right context and all the stuff that we've been talking
01:58all day about here in all the talks in order to actually build a real scale.
02:03And one thing that is happening internally as we we adopted whether teams or individuals the adoption is uneven.
02:13We have teams that are very AI forward and have already transformed their entire workflows and then we have teams that are still experimenting in the earlier act and or have lost confidence and they all need to work together in order to ship
02:28our product.
02:29Um So, they need to coexist in the organization and we need to find a way to support them and while bringing on everybody along for the journey and getting everybody to the third act of the story.
02:44Aside from that main friction point, we have also noticed other friction points that happened as we adopt AI.
02:53One thing that we've heard a lot from developers and managers have have been noticing is that reduced developer agency causes um engineers to lose some of their job satisfaction.
03:04So, if a lot of people used to take a lot of pride and enjoyment in writing code and getting into the flow and a lot of people feel like that's been lost or they're losing a lot of that
03:14element and getting into more of a prompt cycle where they just wait on output from AI and then speak to the AI that like not as much fun as they used to have and they're getting burned out.
03:25Um we've noticed another interesting thing.
03:27It's actually our best engineer, the one that hold all their contacts in their brain.
03:31Um they end up getting out of the burden.
03:34What ends up happening is they they know where all the pitfalls are.
03:37They are like holding together with with like their mental duct tape all the places that agents are not working well and they're preventing all the really bad stuff from coming in or all they they have all the
03:50institutional contact that have never written down in their head and they get so much burden and and become bottlenecks and gets really frustrated.
03:58So, they actually end up being slowest to adopt because they see all the problem uh first hand.
04:04That's another big big issue that we've seen.
04:08Um and this one I'm sure everybody can resonate or in Sorry, I'm sure everybody here will resonate.
04:15Um that all of a sudden all the design docs and all the Slack messages you know, this the emails have gotten three or four times as long and we've gotten two or three times as many emails
04:26and they say basically as much as they did before.
04:29So, communication has gotten quite inefficient and some of the markers of like what is high quality and important things versus not so much high quality um has become challenging to navigate.
04:42Um so, I'm going to spend uh the next few minutes talking about some of the lessons that we've learned and how we've been trying to apply this.
04:50This is a journey.
04:51We have not come out through the other end, but we've seen some really interesting progress along a lot of these lines.
05:00Um I think this uh a lot of the speakers here have touched upon this, but investing in verification is probably the highest value thing we can do in our code base.
05:10Um anytime that we can lift a left shift anything in our workflow from a human needing to do it to an agent being able to verify it.
05:20So, for example, when uh Playwright and MCP came out, instead of having humans navigate the code, now the agent can explore the code.
05:30That was a big win unlock for productivity in a lot of our team.
05:34That's really That's always a a big win for us.
05:38The other thing is um it's even better if when you find something that the agent has found to be useful, take the time to take that and encode into a deterministic
05:50flow.
05:51A deterministic flow that can be easily repeated is saved on tokens, save on time for the and then it also you also know that you're using the the LLM
06:00when it needs to reason, but when you have something that is already known and basically can be encoded into a test, spending that time always always pays dividends.
06:11Um And another tip, if you tell your scale your agent to write the code that you're writing um like at the red to green to red to green at the TDD
06:23style, it almost always gives you better results because you set a goal, then you tell the agent to strive toward that goal, it will almost always give you better results than writing the code and then writing the test afterward because then it will fit the test to the code rather than fit the code to pass the verification
06:39criteria.
06:42Um this is the testing pyramid that uh can't the classic testing pyramid from the previous uh just when you think about the testing themselves, which you had the end-to-end test and the integration test and the unit test.
06:55This is very similar.
06:57Move as much as you can down to the deterministic analysis where that's linting, the compiler, um the unit test themselves.
07:05Whatever that can come be covered easily, you can have engine agent do reviews on it based on on criteria, so um architectural standards that that have been easily encoded into the code base, you can move into the agent.
07:20And then only at the very top you need to have some sort of human review, which is usually around the functionality and this is the right thing to build.
07:28That like only leave the human to do what the humans need to actually be involved in.
07:34Um Another really important thing is the planning versus prompting.
07:39This is really tied into the giving agency back to developers and finding a replacement to the craft of writing code.
07:48Um spending a lot of time writing the plan and then sending enough to the agent basically as a as an implementation that can be done automatically is something that we find to
08:01really kind of reintroduce the joy of of building back into the process.
08:07And so it's not uncommon to spend a week writing a very detailed plan, making all the decisions, flushing it out, iterating, sending it out to teammates to review.
08:18And then only when it's ready and you've flushed out all the decision, you can send it to the agent.
08:22The agent will um send it back to you when it's implemented.
08:26And that that has been really successful also in accelerating and also really restoring some of the joy into the development process.
08:38Uh so what makes a good plan?
08:40Um really important to start with a why at the top.
08:44It really helps preventing agent drift.
08:45If you have like a bold big section of kind of like it when you write a design doc, you want to have the executive summary.
08:51Put that in there for the agent.
08:52Otherwise, they'll start drifting over time and make sure that the agent don't go back and change that because they feel like it.
08:59Uh so we start with a why.
09:01Make sure that the plan can be broken down into small parts that can each be verified independently.
09:08And my personal way of knowing what is a good size would I want to review that the PR that will correspond to that part?
09:16If it's going to be too big for me to want to review in one sitting, it's kind of like the test is I'm going to get need to get a cup of coffee before I read this.
09:23That means it's too big and I'm going to want to have it broken down into pieces.
09:27And then I make sure that each part can be validated independently cuz what I don't want to have is have five stages and then the first one is written but not validated, and then everything else is is built on top of all the assumptions.
09:41So, having kind of a validation gate or an exception criteria for each phase really helps and make the plan uh resilient to drift.
09:51And all and and there's all kind of technique on how to manage the contacts and doing a a software factory on top of that.
09:59But once you have the plan, you can use whatever loop uh you want or whatever workflow you want in order to implement it.
10:07Uh this is a screenshot that I randomly picked up a plan, but this is what I usually look for.
10:12The executive summary at the top, the phases break it down, and then each one of them I would go into lots of details so that I can just fit it into a sub agent,
10:20and the sub agent can independently work on that and not have to worry about it.
10:25Um that's that's it.
10:26There are other workflows that would work or other structure to the plan.
10:29I find that part of the things that great about uh AI workflows is that everybody can set up the thing that works best for them.
10:38Oh-oh.
10:41No, thank you.
10:43Everybody can very easily set up the workflow that work exactly for them for them.
10:47So, there's diminishing return in trying to centralize everybody on one thing, but as long as it works for their flow and other people can iterate with them, I find that it generally works very well.
10:59And this is just an example kind of a brag of like this is uh could be a result from a plan.
11:04Um there are probably 20 PRs here.
11:07Some of them would be maybe 10 lines, and some of them would be 100 lines.
11:11There's probably nothing bigger than that, and that allows us to This is in the pre-AI world, this plan probably worked in that for a week.
11:20I aligned with the other with three other teams for another week on that, and then I just sent it to an agent to implement overnight, and it came back.
11:27This is probably from two plans, not one, but it's it's basically six weeks of of coding work just It's um only took 1 week, so that's where I got the 5x
11:39speed up.
11:40If I include the review cycle at the end that we always have to remember.
11:45Um moving on from planning back to the issue that we had with the skeptics and the people who are burdened with the most work, make sure that you bring them in and take their feedback really seriously.
11:59They're skeptic because they're seeing the the way you are lacking validation, where your tools fail.
12:05So, and their feedback is basically the road map of how to improve your agent interacting with the code base.
12:12So, just make sure to bring them in rather than trying to um figure out how to make them use the AI.
12:19Just let's have them be in charge of the road map to make AI safe your organization, and they will come along once they see that that the improvement that they're making actually making their life better.
12:33Um and as you can see, they'll not be shy about telling you what you need to fix.
12:37This is Latin hour sitting with a bunch of people, and this is the result of brainstorms.
12:45Um another thing that's been really helpful with my team specifically, and we're working to adopt it in the broader organization as well, is to make sure that you have an attention-aware communication.
12:58In the age of AI, human attention is a scarce resource.
13:01I think I've heard it for multiple talks, and a lot of people have have come to the same conclusion.
13:06You can't get more human attention.
13:08So, where you spend your time and what you're reading is really becomes really important.
13:13Um so, since it's such a scarce resource, marking what was generated by AI versus what was written by human is really helpful to know how much time you need to spend reading this, and how much slop can you expect in this part of the communication?
13:29Um and that can building a new culture around that self-communication that really helps.
13:36Um So, for example, um the team that team that I work with, we've decided we always every PR description will start with something like that, something that I wrote by hand.
13:47It could be very short that I describe what this is in code and what this is doing.
13:51And then the AI description is going to come after that, which is I will probably read it.
13:55I will probably edit it to remove uh some wrong things, but they didn't write every line here, so they should be more suspicious and they should pay more attention to what I wrote in the top and they should override it.
14:06Things like that in Slack, in email, it's like leaning into the fact that everybody knows that you're using AI to to craft your communication, but just let them share about it, tell them what they should read and what
14:19you they should pay less attention to.
14:22And I remember early on, maybe like earlier in this year, I tried to I had some senior engineers in our org that had kind of were very much AI skeptic and I tried to reach out to them to see what was the problem, what was going on.
14:39I said, "I tried to run an analysis on some of the PR comments that you've run."
14:43And obviously I used the AI to do that.
14:46And then I didn't distinguish very clearly what I wrote versus what they what AI generated.
14:53And they got very upset.
14:55They're like, "Why is sending I did not expect somebody um that I respect this much to send me something that's clearly this sloppy."
15:02And then like I I took immediately like I apologize.
15:06I realize I should have marked it clearly and marked my intention like this is what I wrote.
15:10This is what the AI wrote and I need your feedback on that because I don't have the context to know if it is sloppy or not and that's what I'm asking you for, so
15:17lesson like that and change the culture is just as important as some the engineering challenges that we've been facing.
15:28Um another thing that's really helpful around the adoption is um as you progress through adoption, there's a lot of very fancy tools and a lot of very fancy workflow that we've
15:40we've been implementing, but one of the really effective thing is just letting people use the AI where they're at.
15:47So, uh it help it really helps normalize uh the use of AI for everyday tasks and it helps reduce the friction.
15:55And really one of the most powerful thing is being able to tag an agent in the Slack message with somebody and they can you just do this for me?
16:03And have the agents to close the loop in the thread.
16:06Um that that's kind of thing is really powerful.
16:09And then you can go on top of that and have all this thing automated and do all kind of fancy things, but if you have a new conversation with somebody who's not fully bought in and then you can tag it in a non
16:21like non-passive-aggressive way.
16:23You can tag it and say, "Let's try to see if the agent can get it this time."
16:27And they close the loop and if it's a good experience, that really helps people try it out on their own in other cases.
16:36And our journey continues.
16:37We're still learning even though we're shipping AI externally, our AI adoption um we're experimenting with with so many things all the time.
16:47Our automation story is not uh fully there yet.
16:51We're still trying trying to figure out when we should use how we can use cloud agent effectively given all the dependencies we have for some of our bell system.
17:00And so we are continuing to learn.
17:01It's a culture shift, it's an engineering shift and I don't know about you, but for I've been I've been working in the valley for the last 15 years and this is the biggest change by orders of magnitude of everything that I've seen
17:14in term culture and technology.
17:17So, um we're all here together and we're all figuring it out and that's that's what I wanted to talk to you today.
17:23Thank you.
17:24[applause]
17:40[music]