Velocity Sickness: What Happens When Your Whole Team Gets 10x Faster — Matt Dailey, Ref.

AI Engineer · 20 min · 290 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:12Thank you so much for coming later in the week here.
00:14Uh I'm Matt.
00:17I'm the CEO and founder of Ref.
00:21The problem we work on at Ref is one you might be familiar with where individual engineers are going really fast with AI, but the team as a whole is not.
00:28And we're working to help close that gap.
00:32What I'm going to be talking about today is that is what happens when your whole team gets 10 times faster.
00:38All the things that don't go well.
00:40And my goal is to give you a blueprint for how to get get through those issues.
00:46Uh and have the whole team move faster together.
00:51The way I want to accomplish that is first talk about those issues, define some terms.
00:58Then we'll talk about how did we get here?
01:00Where are we at now?
01:01Um and we'll use that to triangulate on some solutions.
01:05We'll start high level and we'll work down to get more and more practical until we leave with a couple of things you can leave here and do immediately.
01:12Um how's that sound?
01:13I need thumbs up.
01:14All right, awesome.
01:15Great, that was great.
01:17Uh All right, so let's get into it.
01:22These are some of the problems you might be experiencing right now.
01:25Uh these are these are problems that affect individuals and are actually magnified at the team level once the entire team starts experiencing them.
01:34The first one, too many PRs to merge.
01:37This is the like classic first problem you hit when you start adopting AI as an engineer.
01:41You're like, "Great, I'm shipping stuff."
01:43And you push up those PRs and you're like, "There's no way I can merge all these."
01:47Uh and this is obviously magnified when the whole team does this.
01:51Merge conflicts, merge queue breaks down, things get bad.
01:55The second problem is that you're moving in many directions at once.
01:59This is also both individual and a team problem.
02:02Individually, this is where you have a bunch of agents doing different things.
02:05You're trying to remember who's doing what and your brain gets fried.
02:09At the organizational level, this is your engineers picking up things and running in a certain direction.
02:15Another engineer is running over here.
02:17Maybe some are bumping into each other and you're you're not moving cohesively with focus because you're just sprinting in all sorts of directions.
02:25The third problem you might be experiencing is declaring agent bankruptcy.
02:29This is a a common pattern I see engineers get into where you know, you're cranking.
02:34You have like your 12 terminals open.
02:35You're like at the end of the day you're like, "Yeah, I did a lot of work."
02:38You step away from your laptop.
02:40Uh spend time with your friends and family.
02:43Next morning you come back and it's like walking into just a room of strangers.
02:47Like, who are these people?
02:48What are they doing here?
02:49But like no problem.
02:51They're agents so you just like get rid of them and start over again.
02:55The problem though is you know, it feels like you're doing a lot of work but you're doing the same work and you're spending tokens twice.
03:03You're doing the same problems over over again.
03:07And if you think about that organizationally, that's your team not being efficient with both their time and their their token resources.
03:16Problem four though is actually the most important one.
03:19It's critical decisions being made by agents.
03:22Once you have agents doing a lot of your work, if if you as an engineer are letting an agent make a critical decision, you are seeding control of your code.
03:32You are no longer the owner of that code.
03:34The agent is.
03:35And if you imagine that at scale at your company, if the engineers across your team are you know, giving up ownership of the code, you no longer own the product.
03:46These are a bunch of problems you might be experiencing at different degrees at different days.
03:51Uh The way I like bundle it up is into this term velocity sickness.
03:56This is uh the stress caused by sudden output increases thanks to AI.
04:02Um it affects individuals or teams.
04:05Um and the result is output without impact.
04:09So, this is that feeling of like we're moving really fast.
04:11This should feel great.
04:13This should feel awesome, but for some reason it doesn't feel awesome.
04:16We're not having the like the things you expect to be happening are not happening.
04:20Um despite this feeling of being so productive.
04:26So, that's to define that term, but I want to tell uh a little story.
04:30Uh my talk is happening now.
04:37I want to tell a little story about somebody like to make this very real.
04:41Um This is a story about somebody uh who's actually not an engineer, but I think it parallels our engineering workflows a lot.
04:49I Part of my job is I get to go, find, and talk to people who are really pushing the forefront and try and learn from them.
04:54And I love that part of it.
04:56And so, this is somebody and I'm I'm talking through their identical workflows.
05:00Um and this is somebody who writes a newsletter, and their workflow is around how do I I have my ideas and then they're doing research and exploration and then uh like cohesion between those ideas
05:13and editorial to make sure it's in their voice.
05:14And they're they're walking me through their system for how they they manage this whole pipeline um and really scale out their their efficacy.
05:21And this is like very very much not slop.
05:23This is somebody who has is using agents to amplify their own voice uh in a way that's like very impressive.
05:29I'm like, "Wow, that's that is so cool.
05:30I'm I'm like I'm soaking it in."
05:33Uh And then they're like, "Yeah, I'm basically writing a book every week."
05:39And I'm like, "Oh, okay.
05:40Is like is your audience reading a book every week?"
05:44And they're like, "No, they're probably not."
05:46Right?
05:46They're not.
05:46They're they're This is a person who's writing a lot, but those pages that they're writing are going unread.
05:53Um I think that's part of this experience of velocity sickness is we're we're building things that are not mattering for the people we want them to matter for, the people we're trying to reach.
06:04Uh So, what we Instead of unread pages, um what we want is we want to write words that matter.
06:09We want to write words that connect with people.
06:12Um and the parallel to us as like software builders and product builders is is that we want to write we want to build products that connect with people.
06:20We want We want to build products that change the way people live and work and make their lives better.
06:25And we have more ability to do that than ever with AI.
06:29But we have this like feeling of velocity sickness where we're It feels like we should be doing that, but it's not quite landing.
06:35So, let's talk about how we got here.
06:38Um This is what the software engineering process used to look like before AI.
06:44We'd do some planning up front.
06:46We'd sit down and build.
06:47We'd implement.
06:47It'd be It'd be iterative.
06:48We'd be exploring.
06:48But largely we're sitting down building in isolation, implementing something.
06:52And at the end we sort of polish it up and ship it out the door.
06:55And this was great.
06:56We all knew how to do this.
06:57We had a lot of systems for this.
06:59Uh And it was good.
07:00And then AI came along.
07:01Um But at this time our our tools were built for this.
07:06Like all all our history of coding tools were built for this style of work.
07:11Um our IDE, our workhorse, um it was built for implementation and polish to be done by an individual, to be heads down building as a software engineer writing code.
07:22Um That's what That's a tool built for how we used to work.
07:27Um so, let's look at how how we work now.
07:30Our work looks a lot more like this.
07:32Where you do some planning up front.
07:34You start thinking about what am I trying to do here?
07:37You take this idea in your head and start to flesh it out.
07:40Um At some point an agent takes that idea and like implements it.
07:45This is like arguably should not even be on this slide cuz it's not our human work anymore.
07:48It's done by the agent.
07:50Um and then in the end we do some polishing where we take back that thing the agent has made for us and we like hold it in our hands and we say, is this
07:58is this what I wanted?
07:59Um This is a very different shape of work.
08:02Right?
08:03We're no longer doing this like heads down building.
08:07These are the two uh creative and collaborative parts of our work as engineers.
08:12This is where we like express our craft as an engineer.
08:15And the one that's most different is the the planning stage.
08:18So, I want to zoom in on that just a little bit.
08:21Uh that's the like exploratory, creative, collaborative part where we're we're thinking about like I have this vague idea.
08:28I have this complex system.
08:30I need to understand the contours of this system, apply this idea, and like really understand it.
08:36And I need to pull out what's relevant and express my taste as an engineer.
08:40As to like, where do I want this system to go?
08:42Um This is the kind of work that uh that's creative and and needs to be done together.
08:48I think the the shape of engineering teams and product teams is changing with AI.
08:52But we're always going to have this thing where we have a group of people responsible for managing a complex system and it's sort of deciding the future of that system.
09:00And that's this planning work.
09:03And so, we're we're starting to see some of the hints of like, okay, we had tools built for a certain type of work.
09:09Our work looks different now.
09:11Um Let's start to see think about that a little bit more.
09:14So, uh we used to have the IDE as our main tool.
09:17It was used for implementation.
09:19But now we're doing this different kind of work.
09:21So, at the very high level, we can start to think about solutions like, okay, we have a tool built for this type of work.
09:26We have a new type of work.
09:27Let's think about you know, what this new layer of our work is.
09:31This This is the decision layer.
09:32This is where we're thinking through what are the key decisions, doing that like craft of engineering, um and expressing our taste as engineers, ultimately a different thing than implementation.
09:43And it it's a different gear as an engineer.
09:46The skill now is what gear am I in?
09:49Am I using the appropriate tools for the gear that I'm what I'm trying to accomplish right now.
09:56So let's think about what a tool built for the decision layer would look like.
10:01It'd be a tool built for docs and not chat.
10:04And there's that's like a short sentence.
10:06There's a lot to unpack here though.
10:08So I'm going to spend a lot of time on this slide.
10:11The problem with chats is that they are the relic of building for implementation.
10:17So they're they're default isolated and ephemeral and and brain off.
10:23They're they're made to build things and get stuff done.
10:27And that's not really the same type of work we're doing at the decision layer.
10:30We're doing this creative exploratory work.
10:33So being in this isolated environment where I'm working with an agent, maybe I start with my vague idea and I'm exploring it and asking questions, but decisions are being made in that like that chat that are are not shared with my team
10:45that are going to disappear as as long I'm going to result in some code being output where those important decisions are not being made clear and shared with the team.
10:54And you're also in this mode where the agent saying Okay, this is what I want to do.
10:58Is is that okay?
10:59Let's go.
11:00Or or sometimes it'll say it'll ask you a question and it'll be like you know, this is the recommended option and then you're like, great.
11:07I don't even think about this.
11:08I'll just hit that one and we keep going.
11:12Docs start to solve this problem.
11:13So when you're if you center your work around working in docs they're meant to be bring forward the key decisions.
11:23This is this is what our work is now.
11:25Our work now is figure out what decisions matter and then make those decisions and then get out of the way while the agents fill in the rest.
11:35Working in Docs is the the classic way we would create alignment.
11:40If you were If you think back to being a manager or a lead on a team before AI, if your team was having struggling with alignment, you would not tell them like, "Let's go all work in Slack DMs.
11:50Let's like go direct message each other."
11:52You'd say, "Let's like bring forward key decisions, align on them, spend time on these, like find a way that we can really spend time getting our decisions right."
12:03So, there are two things you might be thinking looking at this that are not what I'm talking about here.
12:08The first one is plan mode.
12:10That's a great tool.
12:12The other one is like full-on factory spectrum and development.
12:15That's a great tool, too.
12:17I'm talking about something in the middle.
12:19Plan mode is great, but it's largely a a rich chat message where the agent is saying, "Hey, here's a like better visualization of what I'm trying to express to you."
12:30Yeah, that's great.
12:32But it's still in this isolated ephemeral environment.
12:35And what I'm suggesting is something more more durable, more shared, more long-lived that you and your team are spending time on.
12:42Similarly, the spectrum and development where we just define the behaviors, we operate at the like product level, is a little far away from the engineering reality.
12:52The engineering reality is that I need to understand my system and have a tool that like helps me understand that system and lay out those key decisions in a in a technical sense.
13:01The way I like to conceptualize this is as the portal to the software system, where you are like Tony Stark and you're like, "Show me what matters."
13:09And you're like, "I'm working on this.
13:10Pull out the bits that are relevant."
13:12And AI is amazing at finding things that are related to other things, helping you find what's what's relevant and lay them out on the table in front of you.
13:19Organize the pieces in the way you want to represent how you want the system to grow.
13:25But the big conceptual flip here is actually that we're pulling out the state.
13:30So, when you're living and working in a a long-lived session with an agent, there's this implicit context being built up like over that work, and that's great.
13:40Um But there's you're also doing actions and it's it's not shared.
13:44What you want is to separate the the agent as the action and the doc as the state.
13:51And so, you can spawn new agents that have the same context or starting from the same place that are able to collaborate and work on the same same piece of context and state.
14:01You're ultimately doing context engineering in this doc, so that every agent is largely stateless and starts from this place um the same place.
14:08And what that gives you is your team and yourself can look into this and understand what actually is in here.
14:15What are the decisions that are being made um and have a a clear understanding of what the key decisions are.
14:23So, this is a a different way of working where your your core atom of your work is a doc rather than a chat.
14:29Um What happens when you actually start to implement this?
14:33Well, the first thing we see happen actually is that people start to plan and then not implement their plan.
14:39Uh and this is actually like a really good sign.
14:43Because what that means is they're thinking through ideas.
14:45They're saying, "I have this idea.
14:46Let me explore it.
14:47Let me start to flesh it out.
14:48Like I have this vague thought.
14:50Don't just give me some code.
14:50Don't go off and like build it for me, but let me help help me understand the idea that I'm talking about.
14:56Um And then they have a bunch of these and then some of them are getting built and some of them aren't.
15:01So, that means they're prioritizing the ideas that after they've explored them are the ones that are worth building and and going in this to the next step with.
15:10Um One way I like to frame this is that you're you're shifting from code velocity to idea velocity.
15:16So, going back to our our problem statement like how are we dealing with velocity sickness?
15:23The velocity sickness is we're shipping too much code that's not going anywhere.
15:26The solution is to shift that velocity to ideas so that rather than you know, getting stuck in prototype gravity where we we build something and we're so excited to just ship that thing
15:39and we're going down like one path of the idea maze, we can now like more more effectively explore that whole maze and find that gold that's around the corner.
15:48Um and and really impact the people we're trying to help.
15:55So, let's go back.
15:57Here are those problems, those velocity sickness problems I talked about the beginning.
16:00Let's see how this starts to address those.
16:03First, too many PRs.
16:07We've moved the the review point earlier into the process.
16:10So, we're aligning on the key decisions up front.
16:13That means the code review is easier because the hardest part of any code review is, you know, what actually matters here.
16:18That's the first step is like, "Hmm, what do I care about here?"
16:22If you move that earlier, we've aligned on that, the code review becomes much simpler.
16:27Number two, moving in too many directions.
16:29Again, we're aligning early as a team and individually, I'm like understanding what I'm working on across many agents.
16:36If I'm working on a large thing, I've I've understood it initially.
16:40So, as a team, we're sharing these plans and we're aligning.
16:43This is what we're trying to build and it's easy before someone has spent even a day in AI going deep on some idea building a prototype, we can talk about it early and make sure we're aligned on where we actually taking the system.
16:54Um declaring agent bankruptcy is just not a thing because you've made your agent stateless.
16:58So, the result of their work is in the stock.
17:01If you need to rebuild your human context, you just read the doc.
17:05Now, you understand the state of this project and you can pick up from there.
17:10And the most important one, humans own the decisions.
17:13Like, that's what we're solving for.
17:15Humans need to own the decisions.
17:18That's how we retain ownership of our software and our products and make it like a true expression of what we're trying to create in the world.
17:29There are also some other benefits.
17:30So, when you have this state extracted and you're working from a shared context, you get more parallel agents.
17:37It's easier to work with parallel agents.
17:39Uh you get this durable decision log.
17:41So, something uh that's really powerful.
17:43I think a lot of people are thinking about how do we capture all the decisions going into these sessions?
17:49A really great solution to that is let's pull out all the decisions up front and agree to them and put them in a place that's durable so that we don't have to have like some LLM summarizing
17:58it and maybe picking the wrong things later on.
18:01We want to bring that forward so we can save it and refer to it later.
18:05Um and then my favorite one um is actually more collaboration.
18:10Like this process, we want we're doing more work that is creative, which means we should be collaborating more as engineers.
18:17I think the future of engineering is multiplayer.
18:21It's going to be multi multiplayer by default sooner than we think.
18:25Um and it's more important than ever because we're in this moment with AI where everything's moving faster than ever.
18:31There's a pressure like everyone's company feels existential, small or big.
18:35You need to deliver.
18:36And the way we get through that as humans is by working together.
18:39And we need tools that help us do that.
18:43So, here are three concrete things you can leave this room and do right now.
18:48The first one is to think of your work in terms of planning and polish.
18:52So, recognizing that there are two gears I'm working in.
18:55There's no longer this one focus of implementation.
18:57There's uh plan and then polish.
19:01Um and notice when you're in a single session and you're doing both of these in one session.
19:07Notice when you drift from the planning phase into the polish phase and is your tool serving you for what you're trying to do at that moment?
19:16Uh number two is to start to treat your plan as a portal to the software system.
19:20So, really treating it as this powerful, malleable tool to say like, what matters to to me for what I'm working on right now?
19:27Um and asking it to show that to you so that you can make the best decisions possible.
19:33And number three is share a plan.
19:35Like, don't just you know, write the plan, give it to your agent, and have them implement it.
19:40Give it to someone on your team.
19:41This is like, I feel like very unnatural for a lot of people.
19:44We always we we think we know what's going on.
19:47But, it's always valuable.
19:49You have smart teammates.
19:50They have great context in their heads.
19:52You should tap into that.
19:53They will give you good feedback.
19:55Uh it's a really valuable thing to do.
19:59That's my talk.
20:00I'm Matt.
20:00Um I'm the CEO of Ref.
20:02If you want to talk about any of this stuff, this has all my like connection information.
20:06Ref is a tool built for the decision layer.
20:08Uh we work with all of your existing implementation tools.
20:12Um if you want to see a demo, come you can come by our booth or just see me out there.
20:16Uh and I'd love to talk about this stuff, so come say hi.
20:19Thanks.