WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=5Bn0xro2ol8

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/5Bn0xro2ol8.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:01.309 --> 00:00:03.309
[music]

s2
00:00:12.720 --> 00:00:13.480
Good afternoon.

s3
00:00:13.480 --> 00:00:15.400
My name is Alon Blum.

s4
00:00:15.400 --> 00:00:17.680
I am a software engineer at Figma.

s5
00:00:17.680 --> 00:00:31.840
And in my talk today, we're going to talk about how we've adopted or are adopting agent into our workflow at Figma while maintaining high quality for our code base.

s6
00:00:31.960 --> 00:00:43.920
So, as you may know, Figma is the browser-based editor where design and engineering and now AI agent collaborate together to ship code.

s7
00:00:43.920 --> 00:00:52.160
Uh this Figma has pivoted very strongly from being a traditional tool to an AI-first tool.

s8
00:00:52.160 --> 00:00:54.360
But in this talk, I'm not going to talk about our product.

s9
00:00:54.360 --> 00:01:03.280
I'm going to talk more about our internal organization and how our engineering org has been adopting AI agents.

s10
00:01:04.440 --> 00:01:16.560
Um what we we found internally is both organizations, companies, and individual there's kind of a three-act process of AI adoption.

s11
00:01:16.560 --> 00:01:30.240
You start with picking up something, whether it was a lot of the people in this room who have been doing using our AI pal and have been using AI for a while and they picked up something and got some simple things to work very well.

s12
00:01:30.240 --> 00:01:32.400
10x faster.

s13
00:01:32.400 --> 00:01:45.760
Then you start applying those same practices to bigger problems and AI fails pretty badly at that, gives you bad stuff, lots of bugs, and the trust that you build breaks down.

s14
00:01:45.760 --> 00:01:58.440
And then from that point, you start building the real skill, which is learning how to use AI correctly and put the right guardrails and the right prompting and the right context and all the stuff that we've been talking

s15
00:01:58.440 --> 00:02:03.680
all day about here in all the talks in order to actually build a real scale.

s16
00:02:03.800 --> 00:02:13.280
And one thing that is happening internally as we we adopted whether teams or individuals the adoption is uneven.

s17
00:02:13.280 --> 00:02:28.600
We have teams that are very AI forward and have already transformed their entire workflows and then we have teams that are still experimenting in the earlier act and or have lost confidence and they all need to work together in order to ship

s18
00:02:28.600 --> 00:02:29.920
our product.

s19
00:02:29.920 --> 00:02:43.920
Um So, they need to coexist in the organization and we need to find a way to support them and while bringing on everybody along for the journey and getting everybody to the third act of the story.

s20
00:02:44.520 --> 00:02:53.200
Aside from that main friction point, we have also noticed other friction points that happened as we adopt AI.

s21
00:02:53.959 --> 00:03:04.239
One thing that we've heard a lot from developers and managers have have been noticing is that reduced developer agency causes um engineers to lose some of their job satisfaction.

s22
00:03:04.239 --> 00:03:14.880
So, if a lot of people used to take a lot of pride and enjoyment in writing code and getting into the flow and a lot of people feel like that's been lost or they're losing a lot of that

s23
00:03:14.880 --> 00:03:25.120
element and getting into more of a prompt cycle where they just wait on output from AI and then speak to the AI that like not as much fun as they used to have and they're getting burned out.

s24
00:03:25.320 --> 00:03:27.720
Um we've noticed another interesting thing.

s25
00:03:27.720 --> 00:03:31.519
It's actually our best engineer, the one that hold all their contacts in their brain.

s26
00:03:31.519 --> 00:03:34.480
Um they end up getting out of the burden.

s27
00:03:34.480 --> 00:03:37.800
What ends up happening is they they know where all the pitfalls are.

s28
00:03:37.800 --> 00:03:50.760
They are like holding together with with like their mental duct tape all the places that agents are not working well and they're preventing all the really bad stuff from coming in or all they they have all the

s29
00:03:50.760 --> 00:03:58.800
institutional contact that have never written down in their head and they get so much burden and and become bottlenecks and gets really frustrated.

s30
00:03:58.800 --> 00:04:04.520
So, they actually end up being slowest to adopt because they see all the problem uh first hand.

s31
00:04:04.520 --> 00:04:08.000
That's another big big issue that we've seen.

s32
00:04:08.000 --> 00:04:15.800
Um and this one I'm sure everybody can resonate or in Sorry, I'm sure everybody here will resonate.

s33
00:04:15.800 --> 00:04:26.200
Um that all of a sudden all the design docs and all the Slack messages you know, this the emails have gotten three or four times as long and we've gotten two or three times as many emails

s34
00:04:26.200 --> 00:04:29.240
and they say basically as much as they did before.

s35
00:04:29.240 --> 00:04:41.800
So, communication has gotten quite inefficient and some of the markers of like what is high quality and important things versus not so much high quality um has become challenging to navigate.

s36
00:04:42.520 --> 00:04:50.760
Um so, I'm going to spend uh the next few minutes talking about some of the lessons that we've learned and how we've been trying to apply this.

s37
00:04:50.760 --> 00:04:51.480
This is a journey.

s38
00:04:51.480 --> 00:04:58.960
We have not come out through the other end, but we've seen some really interesting progress along a lot of these lines.

s39
00:05:00.200 --> 00:05:10.400
Um I think this uh a lot of the speakers here have touched upon this, but investing in verification is probably the highest value thing we can do in our code base.

s40
00:05:10.400 --> 00:05:20.760
Um anytime that we can lift a left shift anything in our workflow from a human needing to do it to an agent being able to verify it.

s41
00:05:20.760 --> 00:05:30.080
So, for example, when uh Playwright and MCP came out, instead of having humans navigate the code, now the agent can explore the code.

s42
00:05:30.080 --> 00:05:34.160
That was a big win unlock for productivity in a lot of our team.

s43
00:05:34.160 --> 00:05:38.320
That's really That's always a a big win for us.

s44
00:05:38.320 --> 00:05:50.200
The other thing is um it's even better if when you find something that the agent has found to be useful, take the time to take that and encode into a deterministic

s45
00:05:50.200 --> 00:05:51.440
flow.

s46
00:05:51.440 --> 00:06:00.480
A deterministic flow that can be easily repeated is saved on tokens, save on time for the and then it also you also know that you're using the the LLM

s47
00:06:00.480 --> 00:06:11.320
when it needs to reason, but when you have something that is already known and basically can be encoded into a test, spending that time always always pays dividends.

s48
00:06:11.320 --> 00:06:23.440
Um And another tip, if you tell your scale your agent to write the code that you're writing um like at the red to green to red to green at the TDD

s49
00:06:23.440 --> 00:06:39.560
style, it almost always gives you better results because you set a goal, then you tell the agent to strive toward that goal, it will almost always give you better results than writing the code and then writing the test afterward because then it will fit the test to the code rather than fit the code to pass the verification

s50
00:06:39.560 --> 00:06:41.560
criteria.

s51
00:06:42.480 --> 00:06:55.320
Um this is the testing pyramid that uh can't the classic testing pyramid from the previous uh just when you think about the testing themselves, which you had the end-to-end test and the integration test and the unit test.

s52
00:06:55.320 --> 00:06:57.280
This is very similar.

s53
00:06:57.280 --> 00:07:05.520
Move as much as you can down to the deterministic analysis where that's linting, the compiler, um the unit test themselves.

s54
00:07:05.520 --> 00:07:20.200
Whatever that can come be covered easily, you can have engine agent do reviews on it based on on criteria, so um architectural standards that that have been easily encoded into the code base, you can move into the agent.

s55
00:07:20.200 --> 00:07:28.240
And then only at the very top you need to have some sort of human review, which is usually around the functionality and this is the right thing to build.

s56
00:07:28.240 --> 00:07:34.040
That like only leave the human to do what the humans need to actually be involved in.

s57
00:07:34.840 --> 00:07:39.240
Um Another really important thing is the planning versus prompting.

s58
00:07:39.240 --> 00:07:48.880
This is really tied into the giving agency back to developers and finding a replacement to the craft of writing code.

s59
00:07:48.880 --> 00:08:01.080
Um spending a lot of time writing the plan and then sending enough to the agent basically as a as an implementation that can be done automatically is something that we find to

s60
00:08:01.080 --> 00:08:07.680
really kind of reintroduce the joy of of building back into the process.

s61
00:08:07.680 --> 00:08:18.000
And so it's not uncommon to spend a week writing a very detailed plan, making all the decisions, flushing it out, iterating, sending it out to teammates to review.

s62
00:08:18.000 --> 00:08:22.160
And then only when it's ready and you've flushed out all the decision, you can send it to the agent.

s63
00:08:22.160 --> 00:08:26.360
The agent will um send it back to you when it's implemented.

s64
00:08:26.360 --> 00:08:38.039
And that that has been really successful also in accelerating and also really restoring some of the joy into the development process.

s65
00:08:38.680 --> 00:08:40.719
Uh so what makes a good plan?

s66
00:08:40.719 --> 00:08:44.080
Um really important to start with a why at the top.

s67
00:08:44.080 --> 00:08:45.720
It really helps preventing agent drift.

s68
00:08:45.720 --> 00:08:51.600
If you have like a bold big section of kind of like it when you write a design doc, you want to have the executive summary.

s69
00:08:51.600 --> 00:08:52.920
Put that in there for the agent.

s70
00:08:52.920 --> 00:08:59.000
Otherwise, they'll start drifting over time and make sure that the agent don't go back and change that because they feel like it.

s71
00:08:59.000 --> 00:09:01.400
Uh so we start with a why.

s72
00:09:01.400 --> 00:09:08.800
Make sure that the plan can be broken down into small parts that can each be verified independently.

s73
00:09:08.800 --> 00:09:16.160
And my personal way of knowing what is a good size would I want to review that the PR that will correspond to that part?

s74
00:09:16.160 --> 00:09:23.160
If it's going to be too big for me to want to review in one sitting, it's kind of like the test is I'm going to get need to get a cup of coffee before I read this.

s75
00:09:23.160 --> 00:09:27.720
That means it's too big and I'm going to want to have it broken down into pieces.

s76
00:09:27.720 --> 00:09:41.760
And then I make sure that each part can be validated independently cuz what I don't want to have is have five stages and then the first one is written but not validated, and then everything else is is built on top of all the assumptions.

s77
00:09:41.760 --> 00:09:51.800
So, having kind of a validation gate or an exception criteria for each phase really helps and make the plan uh resilient to drift.

s78
00:09:51.800 --> 00:09:59.480
And all and and there's all kind of technique on how to manage the contacts and doing a a software factory on top of that.

s79
00:09:59.480 --> 00:10:07.600
But once you have the plan, you can use whatever loop uh you want or whatever workflow you want in order to implement it.

s80
00:10:07.600 --> 00:10:12.720
Uh this is a screenshot that I randomly picked up a plan, but this is what I usually look for.

s81
00:10:12.720 --> 00:10:20.680
The executive summary at the top, the phases break it down, and then each one of them I would go into lots of details so that I can just fit it into a sub agent,

s82
00:10:20.680 --> 00:10:25.040
and the sub agent can independently work on that and not have to worry about it.

s83
00:10:25.040 --> 00:10:26.640
Um that's that's it.

s84
00:10:26.640 --> 00:10:29.840
There are other workflows that would work or other structure to the plan.

s85
00:10:29.840 --> 00:10:38.040
I find that part of the things that great about uh AI workflows is that everybody can set up the thing that works best for them.

s86
00:10:38.040 --> 00:10:40.040
Oh-oh.

s87
00:10:41.160 --> 00:10:43.680
No, thank you.

s88
00:10:43.720 --> 00:10:47.720
Everybody can very easily set up the workflow that work exactly for them for them.

s89
00:10:47.720 --> 00:10:59.520
So, there's diminishing return in trying to centralize everybody on one thing, but as long as it works for their flow and other people can iterate with them, I find that it generally works very well.

s90
00:10:59.800 --> 00:11:04.960
And this is just an example kind of a brag of like this is uh could be a result from a plan.

s91
00:11:04.960 --> 00:11:07.880
Um there are probably 20 PRs here.

s92
00:11:07.880 --> 00:11:11.120
Some of them would be maybe 10 lines, and some of them would be 100 lines.

s93
00:11:11.120 --> 00:11:20.160
There's probably nothing bigger than that, and that allows us to This is in the pre-AI world, this plan probably worked in that for a week.

s94
00:11:20.160 --> 00:11:27.400
I aligned with the other with three other teams for another week on that, and then I just sent it to an agent to implement overnight, and it came back.

s95
00:11:27.400 --> 00:11:39.440
This is probably from two plans, not one, but it's it's basically six weeks of of coding work just It's um only took 1 week, so that's where I got the 5x

s96
00:11:39.440 --> 00:11:40.240
speed up.

s97
00:11:40.240 --> 00:11:44.720
If I include the review cycle at the end that we always have to remember.

s98
00:11:45.760 --> 00:11:59.840
Um moving on from planning back to the issue that we had with the skeptics and the people who are burdened with the most work, make sure that you bring them in and take their feedback really seriously.

s99
00:11:59.840 --> 00:12:05.680
They're skeptic because they're seeing the the way you are lacking validation, where your tools fail.

s100
00:12:05.680 --> 00:12:12.560
So, and their feedback is basically the road map of how to improve your agent interacting with the code base.

s101
00:12:12.560 --> 00:12:19.280
So, just make sure to bring them in rather than trying to um figure out how to make them use the AI.

s102
00:12:19.280 --> 00:12:32.960
Just let's have them be in charge of the road map to make AI safe your organization, and they will come along once they see that that the improvement that they're making actually making their life better.

s103
00:12:33.240 --> 00:12:37.280
Um and as you can see, they'll not be shy about telling you what you need to fix.

s104
00:12:37.280 --> 00:12:44.120
This is Latin hour sitting with a bunch of people, and this is the result of brainstorms.

s105
00:12:45.320 --> 00:12:58.720
Um another thing that's been really helpful with my team specifically, and we're working to adopt it in the broader organization as well, is to make sure that you have an attention-aware communication.

s106
00:12:58.720 --> 00:13:01.240
In the age of AI, human attention is a scarce resource.

s107
00:13:01.240 --> 00:13:06.320
I think I've heard it for multiple talks, and a lot of people have have come to the same conclusion.

s108
00:13:06.320 --> 00:13:08.160
You can't get more human attention.

s109
00:13:08.160 --> 00:13:13.360
So, where you spend your time and what you're reading is really becomes really important.

s110
00:13:13.360 --> 00:13:29.040
Um so, since it's such a scarce resource, marking what was generated by AI versus what was written by human is really helpful to know how much time you need to spend reading this, and how much slop can you expect in this part of the communication?

s111
00:13:29.040 --> 00:13:36.800
Um and that can building a new culture around that self-communication that really helps.

s112
00:13:36.800 --> 00:13:47.880
Um So, for example, um the team that team that I work with, we've decided we always every PR description will start with something like that, something that I wrote by hand.

s113
00:13:47.880 --> 00:13:51.800
It could be very short that I describe what this is in code and what this is doing.

s114
00:13:51.800 --> 00:13:55.920
And then the AI description is going to come after that, which is I will probably read it.

s115
00:13:55.920 --> 00:14:06.680
I will probably edit it to remove uh some wrong things, but they didn't write every line here, so they should be more suspicious and they should pay more attention to what I wrote in the top and they should override it.

s116
00:14:06.680 --> 00:14:19.760
Things like that in Slack, in email, it's like leaning into the fact that everybody knows that you're using AI to to craft your communication, but just let them share about it, tell them what they should read and what

s117
00:14:19.760 --> 00:14:22.520
you they should pay less attention to.

s118
00:14:22.520 --> 00:14:39.520
And I remember early on, maybe like earlier in this year, I tried to I had some senior engineers in our org that had kind of were very much AI skeptic and I tried to reach out to them to see what was the problem, what was going on.

s119
00:14:39.520 --> 00:14:43.240
I said, "I tried to run an analysis on some of the PR comments that you've run."

s120
00:14:43.240 --> 00:14:46.640
And obviously I used the AI to do that.

s121
00:14:46.640 --> 00:14:53.520
And then I didn't distinguish very clearly what I wrote versus what they what AI generated.

s122
00:14:53.520 --> 00:14:55.040
And they got very upset.

s123
00:14:55.040 --> 00:15:02.960
They're like, "Why is sending I did not expect somebody um that I respect this much to send me something that's clearly this sloppy."

s124
00:15:02.960 --> 00:15:06.360
And then like I I took immediately like I apologize.

s125
00:15:06.360 --> 00:15:10.640
I realize I should have marked it clearly and marked my intention like this is what I wrote.

s126
00:15:10.640 --> 00:15:17.640
This is what the AI wrote and I need your feedback on that because I don't have the context to know if it is sloppy or not and that's what I'm asking you for, so

s127
00:15:17.640 --> 00:15:26.280
lesson like that and change the culture is just as important as some the engineering challenges that we've been facing.

s128
00:15:28.160 --> 00:15:40.520
Um another thing that's really helpful around the adoption is um as you progress through adoption, there's a lot of very fancy tools and a lot of very fancy workflow that we've

s129
00:15:40.520 --> 00:15:47.320
we've been implementing, but one of the really effective thing is just letting people use the AI where they're at.

s130
00:15:47.320 --> 00:15:55.480
So, uh it help it really helps normalize uh the use of AI for everyday tasks and it helps reduce the friction.

s131
00:15:55.480 --> 00:16:03.440
And really one of the most powerful thing is being able to tag an agent in the Slack message with somebody and they can you just do this for me?

s132
00:16:03.440 --> 00:16:06.480
And have the agents to close the loop in the thread.

s133
00:16:06.480 --> 00:16:09.880
Um that that's kind of thing is really powerful.

s134
00:16:09.880 --> 00:16:21.320
And then you can go on top of that and have all this thing automated and do all kind of fancy things, but if you have a new conversation with somebody who's not fully bought in and then you can tag it in a non

s135
00:16:21.320 --> 00:16:23.560
like non-passive-aggressive way.

s136
00:16:23.560 --> 00:16:27.600
You can tag it and say, "Let's try to see if the agent can get it this time."

s137
00:16:27.600 --> 00:16:35.080
And they close the loop and if it's a good experience, that really helps people try it out on their own in other cases.

s138
00:16:36.240 --> 00:16:37.720
And our journey continues.

s139
00:16:37.720 --> 00:16:47.680
We're still learning even though we're shipping AI externally, our AI adoption um we're experimenting with with so many things all the time.

s140
00:16:47.680 --> 00:16:51.520
Our automation story is not uh fully there yet.

s141
00:16:51.520 --> 00:17:00.240
We're still trying trying to figure out when we should use how we can use cloud agent effectively given all the dependencies we have for some of our bell system.

s142
00:17:00.240 --> 00:17:01.920
And so we are continuing to learn.

s143
00:17:01.920 --> 00:17:14.600
It's a culture shift, it's an engineering shift and I don't know about you, but for I've been I've been working in the valley for the last 15 years and this is the biggest change by orders of magnitude of everything that I've seen

s144
00:17:14.600 --> 00:17:17.000
in term culture and technology.

s145
00:17:17.000 --> 00:17:23.040
So, um we're all here together and we're all figuring it out and that's that's what I wanted to talk to you today.

s146
00:17:23.040 --> 00:17:24.736
Thank you.

s147
00:17:24.736 --> 00:17:26.736
[applause]

s148
00:17:40.457 --> 00:17:42.457
[music]