WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=_fHTqOs5wQA

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/_fHTqOs5wQA.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:13.120 --> 00:00:14.080
Okay.

s2
00:00:14.080 --> 00:00:14.880
Hey everybody.

s3
00:00:14.880 --> 00:00:17.440
We have like the line Stephen.

s4
00:00:17.440 --> 00:00:18.000
Hey.

s5
00:00:18.000 --> 00:00:24.640
Uh, we have a line that goes like all the way over here and a bottleneck right here that we're all engineers.

s6
00:00:24.640 --> 00:00:28.080
We should be able to figure out how to solve this problem.

s7
00:00:28.080 --> 00:00:28.800
No, just kidding.

s8
00:00:28.800 --> 00:00:30.720
I I don't want to step on anybody's toes.

s9
00:00:30.720 --> 00:00:35.520
Um, but if somebody wants to figure out what app he's using and help him get them through, that would be sweet.

s10
00:00:35.520 --> 00:00:39.280
My name is Kent Seed Dods and we're just going to get started because we have two hours.

s11
00:00:39.280 --> 00:00:41.840
I think we actually have plenty of time.

s12
00:00:41.840 --> 00:00:46.480
Um, but uh I I don't want to have you all just waiting around.

s13
00:00:46.480 --> 00:00:58.559
So, as people are filing in, be friendly and if you've got a seat next to you, like wave them over and let them in because I I am not certain that we're going to be able to fit everybody who is in line in this room.

s14
00:00:58.559 --> 00:01:04.479
And so, we do want to get every seat filled up so people um are able to make it in here.

s15
00:01:04.479 --> 00:01:15.200
Um I am just thrilled and honestly I don't want to say I'm shocked uh because that reveals some uh level of um uncertainty on my part but um so we won't say shocked but I'm thrilled

s16
00:01:15.200 --> 00:01:30.320
um that uh you all are here uh to talk about building the right thing um product engineering before we get into that to give people a little bit of time to come in uh I want by a raise of hands I want to understand who

s17
00:01:30.320 --> 00:01:36.960
uh I'm talking to so how many of you would consider yourself at your company, you are like individual contributor.

s18
00:01:36.960 --> 00:01:39.119
You're um maybe not writing the code anymore.

s19
00:01:39.119 --> 00:01:39.840
Who's doing that?

s20
00:01:39.840 --> 00:01:42.799
But you're like directing the agents to write code.

s21
00:01:42.799 --> 00:01:43.280
All right.

s22
00:01:43.280 --> 00:01:45.040
I kind of figured most of you would be that.

s23
00:01:45.040 --> 00:01:47.280
What if you were somebody who leads those people?

s24
00:01:47.280 --> 00:01:48.960
And you can be both.

s25
00:01:48.960 --> 00:01:49.600
Okay.

s26
00:01:49.600 --> 00:01:50.240
Awesome.

s27
00:01:50.240 --> 00:01:53.200
What if you were only somebody who leads those people?

s28
00:01:53.200 --> 00:01:53.680
Okay.

s29
00:01:53.680 --> 00:01:53.920
Okay.

s30
00:01:53.920 --> 00:01:55.040
We got a couple managers.

s31
00:01:55.040 --> 00:01:55.680
Uh all right.

s32
00:01:55.680 --> 00:01:57.759
Uh what about like product owners?

s33
00:01:57.759 --> 00:01:59.600
Anybody consider themselves product owner?

s34
00:01:59.600 --> 00:01:59.920
Okay.

s35
00:01:59.920 --> 00:02:00.719
Sweet.

s36
00:02:00.719 --> 00:02:02.479
Uh any CEOs in here?

s37
00:02:02.479 --> 00:02:03.759
Who's a CEO?

s38
00:02:03.759 --> 00:02:06.719
I was just going to be like, "Hey, that's cool for being a chief."

s39
00:02:06.719 --> 00:02:07.071
Um,

s40
00:02:07.071 --> 00:02:08.399
[laughter]

s41
00:02:08.399 --> 00:02:09.200
uh, all right.

s42
00:02:09.200 --> 00:02:09.599
Sweet.

s43
00:02:09.599 --> 00:02:11.840
And how many who has traveled the farthest?

s44
00:02:11.840 --> 00:02:15.280
I talked to a group of guys over here traveled from India.

s45
00:02:15.280 --> 00:02:17.120
Anybody travel farther from India?

s46
00:02:17.120 --> 00:02:18.319
Where did you go?

s47
00:02:18.319 --> 00:02:19.280
Melbourne.

s48
00:02:19.280 --> 00:02:20.400
Oh, Melbourne.

s49
00:02:20.400 --> 00:02:21.120
Nice.

s50
00:02:21.120 --> 00:02:23.680
That is one continent I have yet to set foot in.

s51
00:02:23.680 --> 00:02:24.959
Would like to one day?

s52
00:02:24.959 --> 00:02:26.080
Um, Melbourne is nice.

s53
00:02:26.080 --> 00:02:28.640
My sister actually lived uh in Melbourne for a while.

s54
00:02:28.640 --> 00:02:29.520
She liked it.

s55
00:02:29.520 --> 00:02:30.651
Big bugs.

s56
00:02:30.651 --> 00:02:31.840
[laughter]

s57
00:02:31.840 --> 00:02:35.280
Uh, anyone else come from really far away?

s58
00:02:35.280 --> 00:02:36.879
Uh, yeah, over here.

s59
00:02:36.879 --> 00:02:37.519
Germany.

s60
00:02:37.519 --> 00:02:38.239
Germany.

s61
00:02:38.239 --> 00:02:38.879
Nice.

s62
00:02:38.879 --> 00:02:45.599
Uh, I, uh, I was going to be in Germany next week, but I'm in a play with my kids and so I had to cancel.

s63
00:02:45.599 --> 00:02:47.040
Uh, Finding Neverland.

s64
00:02:47.040 --> 00:02:48.160
I'm not a main character.

s65
00:02:48.160 --> 00:02:50.720
That's for next play, maybe.

s66
00:02:50.720 --> 00:02:54.080
Uh, anybody else from far away?

s67
00:02:54.080 --> 00:02:54.319
Hey.

s68
00:02:54.319 --> 00:02:55.519
Yeah.

s69
00:02:55.519 --> 00:02:56.239
Brazil.

s70
00:02:56.239 --> 00:02:57.040
Oh, whoa, dude.

s71
00:02:57.040 --> 00:02:58.560
What do you Oh, no. The game just ended.

s72
00:02:58.560 --> 00:02:59.280
Who won?

s73
00:02:59.280 --> 00:03:00.239
Brazil.

s74
00:03:00.239 --> 00:03:01.040
Oh wow.

s75
00:03:01.040 --> 00:03:01.840
Okay.

s76
00:03:01.840 --> 00:03:02.640
Good for you guys.

s77
00:03:02.640 --> 00:03:04.239
That that was an interesting game.

s78
00:03:04.239 --> 00:03:05.360
1-1.

s79
00:03:05.360 --> 00:03:06.879
That was uh for a long time.

s80
00:03:06.879 --> 00:03:07.440
Very cool.

s81
00:03:07.440 --> 00:03:08.080
How exciting.

s82
00:03:08.080 --> 00:03:10.080
Congratulations, Brazil.

s83
00:03:10.080 --> 00:03:15.519
Uh okay, let's Dang, this stinks that um that everybody's not in.

s84
00:03:15.519 --> 00:03:20.560
But I'm It's so exciting that so many people are interested in learning what uh how to build the right thing.

s85
00:03:20.560 --> 00:03:22.720
But let's get right into it.

s86
00:03:22.720 --> 00:03:25.200
Here is um my clicker not working.

s87
00:03:25.200 --> 00:03:27.760
Let me plug it in.

s88
00:03:30.879 --> 00:03:32.879
Okay.

s89
00:03:33.840 --> 00:03:38.080
Um, oh, there's an on switch on this clicker.

s90
00:03:38.080 --> 00:03:39.599
There we go.

s91
00:03:39.599 --> 00:03:40.400
All right.

s92
00:03:40.400 --> 00:03:41.840
Here's my thesis.

s93
00:03:41.840 --> 00:03:45.920
This is the reason that um I am talking about this.

s94
00:03:45.920 --> 00:03:52.239
Well, actually to back it up here, I'll I'll just back up since we have a little bit more time to give you a little backstory on myself.

s95
00:03:52.239 --> 00:03:55.680
Um, I have been developing software for over a decade.

s96
00:03:55.680 --> 00:04:00.879
uh professionally I graduated from my uh university BYU in 2014.

s97
00:04:00.879 --> 00:04:03.680
So it's been 12 years since then.

s98
00:04:03.680 --> 00:04:09.519
Um and then I'd been developing software as it like you know part-time before that uh a little bit.

s99
00:04:09.519 --> 00:04:20.720
So yeah, about a half a decade uh I've been doing software development and um through all of that I very quickly got into teaching whether that was uh teaching on the side

s100
00:04:20.720 --> 00:04:33.680
uh or eventually in 2019 I went full-time teacher and that has always been for me about teaching experienced software engineers how to uh or rather accelerating the acquisition of experience for

s101
00:04:33.680 --> 00:04:35.440
uh experienced software engineers.

s102
00:04:35.440 --> 00:04:36.800
So you don't know how to write tests?

s103
00:04:36.800 --> 00:04:37.280
Well, great.

s104
00:04:37.280 --> 00:04:38.800
I'm going to teach you how to write tests.

s105
00:04:38.800 --> 00:04:39.680
You don't know React?

s106
00:04:39.680 --> 00:04:39.919
Great.

s107
00:04:39.919 --> 00:04:40.720
I'll teach you React.

s108
00:04:40.720 --> 00:04:43.440
You don't know full stack, I'll teach you full stack development.

s109
00:04:43.440 --> 00:04:49.199
And uh something happened this year that uh gave some of us a bit of a fright.

s110
00:04:49.199 --> 00:04:54.000
I know for myself, I had two existential crises this year.

s111
00:04:54.000 --> 00:04:56.320
How many people can relate to that?

s112
00:04:56.320 --> 00:04:56.800
All right.

s113
00:04:56.800 --> 00:04:57.199
Yeah.

s114
00:04:57.199 --> 00:05:07.039
We all get back from Christmas break or maybe on Christmas break you're using your agent and you realize, oh my goodness, this this thing is better at coding than I am.

s115
00:05:07.039 --> 00:05:11.440
or okay, maybe it's not quite, but it's gotten a lot better.

s116
00:05:11.440 --> 00:05:25.120
And so my existential crisis was not only am I a software engineer actually building software, but I'm teaching experienced engineers how to get experience in technology they don't have experience in yet, but now who needs to learn React?

s117
00:05:25.120 --> 00:05:31.520
Like that is just not a like if you're an experienced software engineer and you're like, "Oh, I've done some Vue before or whatever.

s118
00:05:31.520 --> 00:05:33.039
Oh, this project uses React."

s119
00:05:33.039 --> 00:05:34.720
You don't need a React course for that.

s120
00:05:34.720 --> 00:05:39.919
And I me just saying that means I lose money because you could go buy my React course.

s121
00:05:39.919 --> 00:05:45.840
But I'm telling you right now, you don't need that because you're an experienced software engineer and you can direct the agent to go build the right thing.

s122
00:05:45.840 --> 00:05:50.560
Like you don't need to know what used state is like who who cares about any of that anymore.

s123
00:05:50.560 --> 00:05:54.960
As long as you have the right building blocks, you can build really anything now.

s124
00:05:54.960 --> 00:06:00.320
So existential crisis for me, like who needs to learn implementation details anymore?

s125
00:06:00.320 --> 00:06:07.199
And so I came down to Yeah, I kind of maybe I should just pull up I have this other talk while everybody's coming in.

s126
00:06:07.199 --> 00:06:13.199
So you're going to get a little bit of that one um that is uh sort of related.

s127
00:06:13.199 --> 00:06:16.960
So uh we we'll we'll just do the first part of this one.

s128
00:06:16.960 --> 00:06:19.360
So um this is this is the work.

s129
00:06:19.360 --> 00:06:21.360
It's standing up on these two stands.

s130
00:06:21.360 --> 00:06:25.520
If you don't get the work all the way across, then it's going to fall over our tools.

s131
00:06:25.520 --> 00:06:28.319
We your IDE, your CI, whatever.

s132
00:06:28.319 --> 00:06:30.000
and then here's the work that you do.

s133
00:06:30.000 --> 00:06:38.880
And over time, um, our agents have just been eating into the work that we do, which is wonderful because it means that we have less work that we have to do.

s134
00:06:38.880 --> 00:06:41.840
We can accomplish more to, uh, get more done.

s135
00:06:41.840 --> 00:06:42.960
It's awesome.

s136
00:06:42.960 --> 00:06:47.600
Um, but, uh, yeah, it makes you ask yourself, what do you actually do?

s137
00:06:47.600 --> 00:06:54.960
And here's just total overwhelm of what's prompt engineering and reasoning and and loop engineering and flow engineering.

s138
00:06:54.960 --> 00:06:56.080
Like, what?

s139
00:06:56.080 --> 00:06:58.800
all of these different things that we're supposed to learn, right?

s140
00:06:58.800 --> 00:07:06.560
As an educator, I look at that and I think, yes, I could make bank on just every other week I do another course.

s141
00:07:06.560 --> 00:07:14.720
You come and I'll teach you what the how we do things this week and then I'll see you in two weeks because it's going to be different and I'm going to make so much money off of this.

s142
00:07:14.720 --> 00:07:22.240
Um, as much as I love making money, uh, and I do have six kids I have to take care of, um, I yeah, I'm not into that.

s143
00:07:22.240 --> 00:07:33.199
I would much rather teach you durable skills that you can uh learn and um and use long term so you don't have to keep coming back to your uh

s144
00:07:33.199 --> 00:07:34.575
dealer or whatever.

s145
00:07:34.575 --> 00:07:35.440
[laughter]

s146
00:07:35.440 --> 00:07:38.800
So we don't know what the future's going to look like.

s147
00:07:38.800 --> 00:07:40.080
It could look like that.

s148
00:07:40.080 --> 00:07:41.039
It could look like this.

s149
00:07:41.039 --> 00:07:43.120
We really just have no idea.

s150
00:07:43.120 --> 00:07:55.199
Um but uh the the fact is that there is a possible maybe we can argue this but there is a possible future where the work is actually completely done by AGI

s151
00:07:55.199 --> 00:08:02.160
right like that is like if you think that that is impossible and I'd love to have a conversation with you about predicting the future.

s152
00:08:02.160 --> 00:08:12.160
Um but I think that this is definitely possible but it's not very useful to plan for this because who knows what the world is going to look like if we eventually get to that point.

s153
00:08:12.160 --> 00:08:15.440
uh well like what is uh humanity at that point.

s154
00:08:15.440 --> 00:08:18.800
So let's say let's take it a step back from that.

s155
00:08:18.800 --> 00:08:21.759
So it's doing almost everything but not everything.

s156
00:08:21.759 --> 00:08:22.000
Okay.

s157
00:08:22.000 --> 00:08:23.440
So it's going to fall over.

s158
00:08:23.440 --> 00:08:25.120
It can't fill in that last little gap.

s159
00:08:25.120 --> 00:08:28.879
So there will be a piece that is us filling in that last little slice.

s160
00:08:28.879 --> 00:08:32.159
So my as I'm thinking about okay what do I do as an educator?

s161
00:08:32.159 --> 00:08:36.080
My interest is that okay let's fast forward time all the way to the future.

s162
00:08:36.080 --> 00:08:38.399
AGI is here and is doing everything.

s163
00:08:38.399 --> 00:08:40.399
Take one step back from that.

s164
00:08:40.399 --> 00:08:58.399
what is that slice that humans are still doing as software engineers that is still valuable and that is where I came to judgment being able to tell uh and and not even just judgment but product engineering in general being able to tell what to build what's the right thing that we should be working on

s165
00:08:58.399 --> 00:09:06.240
um and the metaphor that I like for this and gosh okay the metaphor that I like for this and I'll I'll actually bring this up in the talk uh the official

s166
00:09:06.240 --> 00:09:18.560
workshop later uh is uh an archer with arrows So you've you've got your arrows and you've gotten really good at like your product manager tells you hit that target and you're like okay great and you aim

s167
00:09:18.560 --> 00:09:30.480
you take account the wind everything and you hit the target you're like this dude who is aiming to hit the target through that ring which is just and he does it I just like wow

s168
00:09:30.480 --> 00:09:40.800
can you imagine being that good some of you are that good as software engineers but see here's the problem the problem is that we have gotten our arrows has changed

s169
00:09:40.800 --> 00:09:48.640
and now you don't have to be that good to hit the target because the and like I realize I'm doing some handwaving.

s170
00:09:48.640 --> 00:09:52.959
Yes, we all know that agents make mistakes, yada yada, but like look at the trajectory.

s171
00:09:52.959 --> 00:09:55.200
They're like homing devices now.

s172
00:09:55.200 --> 00:10:03.440
And so now not only can you hit any of those targets with ease, but you can actually hit way more targets than you ever possibly could have imagined before.

s173
00:10:03.440 --> 00:10:16.240
And the the trick is the fact that because of uh well the actually this has always been the case t all the targets are not created equal and so

s174
00:10:16.240 --> 00:10:20.720
knowing which one of those targets is the valuable thing to hit that's the differentiator.

s175
00:10:20.720 --> 00:10:26.640
Okay I think that's as much as I oh yeah this is if if you have seen uh Robin Hood men and tights you know this reference.

s176
00:10:26.640 --> 00:10:30.240
If you haven't then um I'm not going to show it anyway so don't worry.

s177
00:10:30.240 --> 00:10:31.360
Uh, all right.

s178
00:10:31.360 --> 00:10:32.640
We've got enough people in the room.

s179
00:10:32.640 --> 00:10:34.240
I'm gonna get officially started.

s180
00:10:34.240 --> 00:10:36.800
So, thank you all for uh joining.

s181
00:10:36.800 --> 00:10:39.360
Um, we're going to talk about that skill.

s182
00:10:39.360 --> 00:10:40.959
This is what I call the durable skill.

s183
00:10:40.959 --> 00:10:44.720
It's the last skill that the last software engineer needs.

s184
00:10:44.720 --> 00:10:46.959
Um, it's the last thing that you need to learn.

s185
00:10:46.959 --> 00:10:51.440
And if AI learns this skill, then I don't know what we're going to do.

s186
00:10:51.440 --> 00:10:53.120
Um, but it's not this.

s187
00:10:53.120 --> 00:10:56.240
So, um, this is the thesis.

s188
00:10:56.240 --> 00:11:04.800
When AI agents level the implementation playing field, which they are actively doing, then the differentiator becomes building the right thing.

s189
00:11:04.800 --> 00:11:07.040
Okay, I I'll I'll stop.

s190
00:11:07.040 --> 00:11:08.399
I see some people taking photos.

s191
00:11:08.399 --> 00:11:10.000
This is the slide to take a photo of.

s192
00:11:10.000 --> 00:11:10.800
This is my thesis.

s193
00:11:10.800 --> 00:11:13.920
The entire workshop depends on this slide.

s194
00:11:13.920 --> 00:11:16.399
I'll bring it up a couple times uh later.

s195
00:11:16.399 --> 00:11:19.120
All right, I hear some of you chuckling.

s196
00:11:19.120 --> 00:11:20.480
How many of you know what this slide is?

s197
00:11:20.480 --> 00:11:22.160
Have you seen this slide before?

s198
00:11:22.160 --> 00:11:22.640
Really?

s199
00:11:22.640 --> 00:11:24.240
Oh, how Okay, got a couple.

s200
00:11:24.240 --> 00:11:25.600
This is very exciting.

s201
00:11:25.600 --> 00:11:27.440
Um, I do this in every one of my talks.

s202
00:11:27.440 --> 00:11:29.200
I'm going to invite you to please stand.

s203
00:11:29.200 --> 00:11:32.480
If you're physically able to join us, please do.

s204
00:11:32.480 --> 00:11:42.240
Your blood needs to flow in your body for your brain to operate at peak efficiency, which you will need for the next two hours, hour and a half.

s205
00:11:42.240 --> 00:11:44.399
So, I want you to put your arms out in front of you like this.

s206
00:11:44.399 --> 00:11:46.079
Don't hit anybody, please.

s207
00:11:46.079 --> 00:11:48.240
Squat down like this and come back up.

s208
00:11:48.240 --> 00:11:50.028
This is called exercise.

s209
00:11:50.028 --> 00:11:51.200
[laughter]

s210
00:11:51.200 --> 00:11:52.079
This is air squat.

s211
00:11:52.079 --> 00:11:54.560
We'll do 12 of them together, and you need to count out loud with me.

s212
00:11:54.560 --> 00:11:56.959
If you're not counting loud enough, I will start over.

s213
00:11:56.959 --> 00:11:58.160
Ready?

s214
00:11:58.160 --> 00:11:58.880
One.

s215
00:11:58.880 --> 00:11:59.680
Oh, wow.

s216
00:11:59.680 --> 00:12:00.480
Two.

s217
00:12:00.480 --> 00:12:00.800
Great.

s218
00:12:00.800 --> 00:12:02.079
That threat really worked.

s219
00:12:02.079 --> 00:12:02.959
Three.

s220
00:12:02.959 --> 00:12:04.240
Four.

s221
00:12:04.240 --> 00:12:04.640
Five.

s222
00:12:04.640 --> 00:12:05.519
You're doing so great.

s223
00:12:05.519 --> 00:12:06.639
I see smiles.

s224
00:12:06.639 --> 00:12:07.600
You love this.

s225
00:12:07.600 --> 00:12:08.880
Seven.

s226
00:12:08.880 --> 00:12:09.200
Eight.

s227
00:12:09.200 --> 00:12:10.800
And you can go really deep if you want.

s228
00:12:10.800 --> 00:12:12.079
Nine.

s229
00:12:12.079 --> 00:12:12.320
10.

s230
00:12:12.320 --> 00:12:13.200
You know, this is so fun.

s231
00:12:13.200 --> 00:12:14.000
Let's start over.

s232
00:12:14.000 --> 00:12:14.399
One.

s233
00:12:14.399 --> 00:12:15.120
No, just kidding.

s234
00:12:15.120 --> 00:12:15.440
11.

s235
00:12:15.440 --> 00:12:15.839
And 12.

s236
00:12:15.839 --> 00:12:20.160
And then stretch over your head and then over to one side and over to the other.

s237
00:12:20.160 --> 00:12:20.639
All right.

s238
00:12:20.639 --> 00:12:21.360
Sweet.

s239
00:12:21.360 --> 00:12:22.240
Go ahead and sit down.

s240
00:12:22.240 --> 00:12:23.680
Thank you so much.

s241
00:12:23.680 --> 00:12:25.200
Your body needs blood flow.

s242
00:12:25.200 --> 00:12:27.680
Yes, that's right.

s243
00:12:28.018 --> 00:12:29.120
[applause]

s244
00:12:29.120 --> 00:12:32.560
Uh if if you're starting to feel a little sluggish, it is after lunch.

s245
00:12:32.560 --> 00:12:36.320
Um so if you're starting to feel a little sluggish halfway through, just stand up.

s246
00:12:36.320 --> 00:12:36.720
Do it.

s247
00:12:36.720 --> 00:12:38.240
We all know what's going on.

s248
00:12:38.240 --> 00:12:39.040
It's fine.

s249
00:12:39.040 --> 00:12:40.959
There are some seats up here in the front.

s250
00:12:40.959 --> 00:12:43.279
So if you're standing in the back, it's not awkward.

s251
00:12:43.279 --> 00:12:45.360
I already told them to be friendly to you.

s252
00:12:45.360 --> 00:12:47.600
So um they will be friendly.

s253
00:12:47.600 --> 00:12:48.240
All right.

s254
00:12:48.240 --> 00:12:53.600
We are going to for the workshop, we're going to be working through an example product, an example app.

s255
00:12:53.600 --> 00:13:01.760
And so, um, this is a QR code where you can submit your ideas for what that should be and upvote other people's ideas.

s256
00:13:01.760 --> 00:13:07.600
Um, I'm running a big risk because it means I wasn't able to practice the specifics of what we're doing.

s257
00:13:07.600 --> 00:13:19.760
So I'm going to need a lot of feedback from you on um the different ele how we apply the frameworks we're going to uh be talking about to the example app that we come up with and then also

s258
00:13:19.760 --> 00:13:23.680
this is the same QR code and uh this is how we're going to do Q&A.

s259
00:13:23.680 --> 00:13:32.240
Uh so I will be doing Q&A you can upvote each other's questions uh all there and I will put this same QR code at the bottom of every slide.

s260
00:13:32.240 --> 00:13:38.079
So uh if you are uh think of a question later then you can scan it.

s261
00:13:38.079 --> 00:13:50.160
Um and I'll come back to the example app uh that you come up with feel free to uh to if you think of any ideas or actually as we're working through this I want you to be thinking about your own application

s262
00:13:50.160 --> 00:13:59.680
taking notes about how these frameworks apply to your own application as well and I'll be asking you for um examples of how you've applied it to your own application.

s263
00:13:59.680 --> 00:14:02.320
So I'm expecting you to do that.

s264
00:14:02.320 --> 00:14:03.120
Okay.

s265
00:14:03.120 --> 00:14:06.560
So the the premise here is AI changes the scarce resource.

s266
00:14:06.560 --> 00:14:10.320
Uh it's no longer scarce to do implementation.

s267
00:14:10.320 --> 00:14:13.199
That is a um a commoditized thing.

s268
00:14:13.199 --> 00:14:16.000
Now it's again I'm being a little hand wavy.

s269
00:14:16.000 --> 00:14:18.240
I realize that agents aren't perfect and they make mistakes.

s270
00:14:18.240 --> 00:14:24.639
You still but um we are in a a place where agents are getting really really good and they're only getting better.

s271
00:14:24.639 --> 00:14:30.399
So the more valuable thing is deciding what to build, whether it's worth building in the first place.

s272
00:14:30.399 --> 00:14:33.519
And that applies not only to entire apps.

s273
00:14:33.519 --> 00:14:39.760
Like if you're here and you're like, "Well, Kent, I work at an enterprise and I'm it's like internal stuff.

s274
00:14:39.760 --> 00:14:41.760
Uh I'm not working on new apps every day."

s275
00:14:41.760 --> 00:14:42.800
That's not what this is about.

s276
00:14:42.800 --> 00:14:45.760
In fact, I'm more interested in your use case than the startups.

s277
00:14:45.760 --> 00:14:47.519
Um I'm interested in both.

s278
00:14:47.519 --> 00:14:51.279
But uh these all these principles we'll talk about apply to both of those.

s279
00:14:51.279 --> 00:14:54.160
So we're moving from can we build it to is it worth building?

s280
00:14:54.160 --> 00:15:05.600
What's interesting about that is that has always been important and the best software engineers, the most valuable software engineers at any company were the ones that could answer this question, could help the business get to this answer.

s281
00:15:05.600 --> 00:15:07.920
We'll talk about that a little bit too.

s282
00:15:07.920 --> 00:15:11.120
So, we already talked about the arrow metaphor for those of you who missed it.

s283
00:15:11.120 --> 00:15:15.279
We gotten really, really good at uh hitting targets uh with the agents that we have now.

s284
00:15:15.279 --> 00:15:17.519
There are tons more targets we could possibly hit.

s285
00:15:17.519 --> 00:15:33.040
And so the real question is um whether we which one of the targets we should actually try to aim for because maybe you do have infinite money but I don't and I can't aim for every one of those targets and even if you tried

s286
00:15:33.040 --> 00:15:38.160
you would probably overwhelm your users anyway and that would be a like a failure mode to use the AI.

s287
00:15:38.160 --> 00:15:44.800
Like I never used the words failure mode until AI started saying that to me but that would be another failure mode.

s288
00:15:44.800 --> 00:15:56.079
Um, so I'm going to actually be referencing a podcast that I've been running for the last couple months uh quite a lot in this workshop because they're just really really smart people uh and I really like their takes.

s289
00:15:56.079 --> 00:15:58.720
So this has been like this is an upcoming episode.

s290
00:15:58.720 --> 00:16:07.759
He said it's once development becomes cheap and easy and enables anybody to be able to do it especially non-technical people then it's really about the ideas that you have.

s291
00:16:07.759 --> 00:16:18.399
Now, I wouldn't say that we can take our tools that we have today and give it to a non-technical person and expect them to build like real software that can be maintained and like last for 35 years or whatever.

s292
00:16:18.399 --> 00:16:24.240
Um, but it's possible we get to that point and the the fact is that the implementation playing field is leveling.

s293
00:16:24.240 --> 00:16:25.519
It's in the process of that.

s294
00:16:25.519 --> 00:16:29.360
So, ideas have always been u one of the most valuable parts of it.

s295
00:16:29.360 --> 00:16:31.199
And then of course there's execution.

s296
00:16:31.199 --> 00:16:35.680
Um, but ideas are becoming more valuable relative to that execution.

s297
00:16:35.680 --> 00:16:39.120
Um, Julius, um, I'm not sure how to say Julius's name.

s298
00:16:39.120 --> 00:16:39.839
Sorry, Julius.

s299
00:16:39.839 --> 00:16:42.000
It's, uh, Marming, I think.

s300
00:16:42.000 --> 00:16:52.320
He works on T3 code and and, uh, a bunch of other T3 universe stuff, but he said, uh, that the hard question is deciding whether a feature is worth having or what the long-term consequences are.

s301
00:16:52.320 --> 00:16:56.959
That is an important part of ownership that we'll talk about here as well.

s302
00:16:56.959 --> 00:17:02.959
Um, so it's easy to build to spec and think that, okay, yeah, I'm a software engineer.

s303
00:17:02.959 --> 00:17:03.759
I build to spec.

s304
00:17:03.759 --> 00:17:04.880
That's what I do.

s305
00:17:04.880 --> 00:17:07.679
uh hand it off and then say my job is done.

s306
00:17:07.679 --> 00:17:09.360
Let me go grab the other thing.

s307
00:17:09.360 --> 00:17:13.520
But guess what you look like when you're just taking a ticket and turning it into an implementation.

s308
00:17:13.520 --> 00:17:17.439
If that's you, you look a awful lot like an agent to me.

s309
00:17:17.439 --> 00:17:18.160
Uh oh.

s310
00:17:18.160 --> 00:17:20.880
We're going to play some music, I guess.

s311
00:17:20.880 --> 00:17:23.199
Um, turn that off.

s312
00:17:23.199 --> 00:17:25.199
I think there's a little play button on this thing.

s313
00:17:25.199 --> 00:17:29.039
I've never seen what that does until now.

s314
00:17:29.039 --> 00:17:33.280
So, uh, very replaceable to just turn a ticket into an implementation.

s315
00:17:33.280 --> 00:17:46.559
Uh so a product engineer um I is able to recognize that you can still fail after finishing the implementation if it doesn't produce customer value and that's not just your PM's job.

s316
00:17:46.559 --> 00:17:56.559
Uh okay and then Wayne Allen brilliant guy this was an awesome episode he said that the product concern is building the right thing and the engineering concern seems to be building uh the thing right.

s317
00:17:56.559 --> 00:18:04.880
Um, however, building the right uh right thing is downstream of building the uh sorry I I said this wrong on the podcast too when he I repeated it to

s318
00:18:04.880 --> 00:18:08.640
building the thing right is downstream of building the right thing.

s319
00:18:08.640 --> 00:18:20.400
So it is very much in your interest as an engineer who wants your company to be successful so that you can continue to receive a paycheck and take your kids out to get milkshakes that night or whatever you know with the money that you get.

s320
00:18:20.400 --> 00:18:30.400
it's in your best interest to make sure that the upstream activities are coming down so that you have uh something that's actually going to be valuable to work on.

s321
00:18:30.400 --> 00:18:31.280
Okay.

s322
00:18:31.280 --> 00:18:32.960
Uh wh there we go.

s323
00:18:32.960 --> 00:18:36.480
Um Dax Rad uh creator of uh Open Code.

s324
00:18:36.480 --> 00:18:38.880
Uh he How many people use Open Code by the way?

s325
00:18:38.880 --> 00:18:40.559
Yeah, Open Code's pretty cool.

s326
00:18:40.559 --> 00:18:47.600
Um he says that the default place for our new coding agent abilities to go uh to is work on the wrong things.

s327
00:18:47.600 --> 00:18:49.520
Products go from good to bad faster than ever.

s328
00:18:49.520 --> 00:18:53.039
How many people have noticed some the products you use getting worse?

s329
00:18:53.039 --> 00:18:54.960
Yeah, I I relate to that.

s330
00:18:54.960 --> 00:18:56.720
The products that I build are getting worse.

s331
00:18:56.720 --> 00:18:57.600
No, just kidding.

s332
00:18:57.600 --> 00:18:59.919
Hopefully hopefully they're not the products you build.

s333
00:18:59.919 --> 00:19:00.320
Of course.

s334
00:19:00.320 --> 00:19:03.039
No, nobody in this room is doing that.

s335
00:19:03.039 --> 00:19:04.799
U it's just everybody else.

s336
00:19:04.799 --> 00:19:08.400
No, it's just so easy because the agent's not going to say, "Hold up, hold up."

s337
00:19:08.400 --> 00:19:11.840
Like, I think that we're expanding the system too bad or whatever.

s338
00:19:11.840 --> 00:19:22.000
They might in the future, but no, like it it is our responsibility to slow down and be intentional about what we're adding to our product so it doesn't bloat up to something too ridiculous.

s339
00:19:22.000 --> 00:19:32.799
And the the problem is that agents accelerate um bad practices throughout the codebase and really do need to be wrangled in uh as Shandai is per uh uh talking about here as well.

s340
00:19:32.799 --> 00:19:36.080
So you can move bad faster if you're not careful.

s341
00:19:36.080 --> 00:19:41.200
Um and Rita uh Koslav, she I just recorded this episode last week with her.

s342
00:19:41.200 --> 00:20:00.080
uh she is a the um oh shoot what a director of product or VP of product uh I think at Cloudflare uh she's a higher up and she's wonderful wonderful person but but um she said that it's very easy to get the look and feel like really quickly and just decide you know what why don't I just

s343
00:20:00.080 --> 00:20:10.880
ship this so she's on the the management side of uh things and it's really easy to feel like oh I can just ship this but then she says you can build something that on the surface layer feels functional but it's not actually scalable

s344
00:20:10.880 --> 00:20:12.720
doesn't address edge cases.

s345
00:20:12.720 --> 00:20:22.240
And uh the the real key and this is where we come in as product engineers is to actually take ownership of that to look at the prototype and be like yeah I'm going to throw this away.

s346
00:20:22.240 --> 00:20:33.280
We'll have cloud or cursor whatever reimplement this in in two hours and it'll uh you know with my framing my systems thinking and everything um so that I can take ownership and accountability over it.

s347
00:20:33.280 --> 00:20:47.120
The reason that matters is because if you know that you're going to be paged at two in the morning to deal with issues, then you're going to take a lot of care in the system, in the playground that you create for your agents, uh, to make sure that they're successful when they're implementing things.

s348
00:20:47.120 --> 00:20:52.159
And Aaron Francis, you can do the wrong thing incredibly fast and feel like you're making a ton of progress.

s349
00:20:52.159 --> 00:20:53.919
How many people relate to that?

s350
00:20:53.919 --> 00:20:54.559
Yeah.

s351
00:20:54.559 --> 00:20:56.000
So, so bad.

s352
00:20:56.000 --> 00:20:59.600
Because like you get so far into it and you're like, I can't stop now.

s353
00:20:59.600 --> 00:21:01.919
Sunk cost fallacy is not a thing.

s354
00:21:01.919 --> 00:21:10.480
And yeah, so how to do the right thing incredibly fast or at least directionally correct rather than 10,000 lines of code a day on the wrong freaking thing.

s355
00:21:10.480 --> 00:21:12.640
It's so irritating.

s356
00:21:12.640 --> 00:21:17.039
So, uh, a question that I get often, we're we're going to get to the interactive bits here in just a second.

s357
00:21:17.039 --> 00:21:19.760
I just want to establish things so nobody leaves the room.

s358
00:21:19.760 --> 00:21:25.120
Um, that we are not I'm not saying now it's time for you to all be product managers.

s359
00:21:25.120 --> 00:21:26.720
That is not what I'm saying at all.

s360
00:21:26.720 --> 00:21:31.679
So, let's talk about what where the line is between a product engineer and a product manager.

s361
00:21:31.679 --> 00:21:33.760
Um, okay.

s362
00:21:33.760 --> 00:21:46.159
So, your job as a product engineer is to connect the customers uh the understanding what the customer needs to the technical choices that are being made so that you don't paint yourself into a corner and you don't overbuild.

s363
00:21:46.159 --> 00:21:51.679
So, uh you're going to decide on the data model and the uh workflow shape, observability, constraints, failure modes, and the smallest slice.

s364
00:21:51.679 --> 00:22:01.440
All of these things are technical decisions that you make and you make them better by understanding the upstream uh where requests are coming in, where the ideas are coming from.

s365
00:22:01.440 --> 00:22:07.440
Um yeah, that and we'll dive into each one of these a little bit um more here in a second.

s366
00:22:07.440 --> 00:22:11.360
So um this is one of my favorite the smallest slice thing.

s367
00:22:11.360 --> 00:22:13.039
Um I just I love this.

s368
00:22:13.039 --> 00:22:15.840
Um so how many of you have seen this before?

s369
00:22:15.840 --> 00:22:17.520
I'm not I'm not sharing anything new, right?

s370
00:22:17.520 --> 00:22:19.440
Okay, so you got your waterfall.

s371
00:22:19.440 --> 00:22:21.679
Okay, well by the end we'll have something useful.

s372
00:22:21.679 --> 00:22:24.799
In agile it's like we got useful use, but it's different every single time.

s373
00:22:24.799 --> 00:22:27.120
We got to rebuild it from scratch almost every time.

s374
00:22:27.120 --> 00:22:37.280
In AI it's like here's the whole thing and uh your job is to whittle it down to um the actually useful thing that you're trying to uh to accomplish here.

s375
00:22:37.280 --> 00:22:42.080
And what's interesting actually this image also makes me think of uh how Instagram started.

s376
00:22:42.080 --> 00:22:46.640
Instagram was originally an app called Bourbon uh which was basically a foursquare clone.

s377
00:22:46.640 --> 00:22:49.679
Uh, so they were having people check in uh to to things.

s378
00:22:49.679 --> 00:22:51.919
I think it was uh alcoholrelated.

s379
00:22:51.919 --> 00:22:55.600
I don't drink alcohol, so I don't know that world very well, but I think that's what it was all bourbon.

s380
00:22:55.600 --> 00:22:56.080
That's out.

s381
00:22:56.080 --> 00:22:57.919
Yeah, I I'm just kidding.

s382
00:22:57.919 --> 00:23:07.440
Um, but uh yeah, so um they realized after watching you user behavior that the only feature anybody really cared about was sharing photos.

s383
00:23:07.440 --> 00:23:11.360
And so they just gutted everything else and just made it about sharing photos.

s384
00:23:11.360 --> 00:23:13.280
And they had a pretty nice exit.

s385
00:23:13.280 --> 00:23:14.511
So good for them.

s386
00:23:14.511 --> 00:23:15.440
[laughter]

s387
00:23:15.440 --> 00:23:16.240
Uh, okay.

s388
00:23:16.240 --> 00:23:23.280
Uncle uh Bob was also on the podcast just last week and uh he had this experience uh as a software engineer.

s389
00:23:23.280 --> 00:23:29.200
He was working making these little mini computers for uh a company that did like telephone lines and stuff.

s390
00:23:29.200 --> 00:23:34.720
And so he'd make these uh software for the uh guy who would go up the telephone pole to fix stuff.

s391
00:23:34.720 --> 00:23:37.600
And his boss said, "Uh, have you ever been on the truck?

s392
00:23:37.600 --> 00:23:39.520
Have you ever like seen what it's like?"

s393
00:23:39.520 --> 00:23:40.080
He's like, "No."

s394
00:23:40.080 --> 00:23:40.320
Okay.

s395
00:23:40.320 --> 00:23:41.840
Well, you're getting on the truck, you're going out.

s396
00:23:41.840 --> 00:23:50.400
And it he said it totally changed the way that he thinks about building software for people because that guy's up there trying to use his software while he's hanging off of this pole

s397
00:23:50.400 --> 00:23:56.799
and he's realizing okay so there's like 30 things I can think of to improve this software to make it easier for that guy to use this.

s398
00:23:56.799 --> 00:24:03.919
Um so seeing your uh users use your software um is humbling uh to say the least.

s399
00:24:03.919 --> 00:24:08.559
So in his mind a product engineer lives half in the technology and half in the customer's house.

s400
00:24:08.559 --> 00:24:10.640
There's a deeply human side to product engineering.

s401
00:24:10.640 --> 00:24:14.240
His podcast episode is coming out I think tomorrow or next week.

s402
00:24:14.240 --> 00:24:19.360
So if you're interested in hearing more from him or really any of every one of these episodes, it's really good.

s403
00:24:19.360 --> 00:24:20.880
You should definitely uh check it out.

s404
00:24:20.880 --> 00:24:29.279
Had Grady Buch on a couple weeks ago and he also feels that engineering judgment comes from technical experience married to human issues.

s405
00:24:29.279 --> 00:24:35.840
U so being able to attach what the human actually needs to technical expertise you have.

s406
00:24:35.840 --> 00:24:41.200
That's what makes a product engineer a product engineer, not a product manager.

s407
00:24:41.200 --> 00:24:42.559
Um, okay.

s408
00:24:42.559 --> 00:24:44.000
Like I said, we've got a ton of these.

s409
00:24:44.000 --> 00:24:49.360
I'm probably overindexed on the number of quotes, but Ronin is awesome, too.

s410
00:24:49.360 --> 00:24:50.640
It's about launching a product.

s411
00:24:50.640 --> 00:24:51.919
It's not about the cool API.

s412
00:24:51.919 --> 00:25:00.640
And I, in fact, like there are so many um so many instances that I can think of where somebody will come to me and just talk about their solution all the time.

s413
00:25:00.640 --> 00:25:04.400
Like, oh wow, like it does this and it does that and it does this and it does that.

s414
00:25:04.400 --> 00:25:07.600
And I'm thinking, none of those things matter to me whatsoever.

s415
00:25:07.600 --> 00:25:08.400
like it.

s416
00:25:08.400 --> 00:25:16.559
Uh, okay, that's kind of cool, but I already do this one thing and it's not uh so much better that I want to switch from my current workflow to that thing.

s417
00:25:16.559 --> 00:25:26.480
And it's just so clear to me that this these people are so excited about their solution that they've totally lost the plot of the problem that they're trying to solve for users.

s418
00:25:26.480 --> 00:25:30.480
So, um, it's really easy to fall into that as an engineer.

s419
00:25:30.480 --> 00:25:38.000
And therefore a great way for you to stand out as an engineer at your company is to be a product-minded engineer.

s420
00:25:38.000 --> 00:25:47.200
So you have the technical capabilities but you also understand what the user is ultimately trying to do and what your company is trying to do to solve uh their problems.

s421
00:25:47.200 --> 00:25:59.919
So um a a lot of the decisions that we make as engineers um shape what the product ends up being and um and the reason that is so important is because

s422
00:25:59.919 --> 00:26:02.159
changing architecture is really expensive.

s423
00:26:02.159 --> 00:26:12.400
You might think, well, no, Ken, like it's literally just like, I don't know, a million tokens, and I can be from, you know, one database to another, or I can be from monor repo to microservices

s424
00:26:12.400 --> 00:26:16.720
or or or uh mono monolith to microservices or whatever.

s425
00:26:16.720 --> 00:26:27.840
Uh it's depending on the size of your app, maybe that is uh the case, but um it is really expensive on the way that it impacts the rest of the team and on the way it impacts the user.

s426
00:26:27.840 --> 00:26:33.919
Um, change is expensive and and if you say, "Okay, well, it's just a couple million tokens.

s427
00:26:33.919 --> 00:26:39.919
What if I could hire you or I could hire somebody who made the right choice the first time because they understand the product?"

s428
00:26:39.919 --> 00:26:44.159
Yeah, I'm going to hire the person who can uh do it right the first time uh and cost me less.

s429
00:26:44.159 --> 00:26:45.520
So, it's all about the primitives.

s430
00:26:45.520 --> 00:26:46.640
This is Ree.

s431
00:26:46.640 --> 00:26:51.840
Uh he works uh actually he just made a startup uh and really cool dude.

s432
00:26:51.840 --> 00:26:56.480
But, uh he's he's saying it's all about having the right primitives that you can build upon.

s433
00:26:56.480 --> 00:27:00.960
It's really difficult to build a really great solution on top of those u wrong primitives.

s434
00:27:00.960 --> 00:27:03.840
So your engineering expertise matters.

s435
00:27:03.840 --> 00:27:10.080
Hopefully that uh uh that you know what dang this is going um I got to talk about Michael.

s436
00:27:10.080 --> 00:27:12.559
Who who here knows Michael at work OS?

s437
00:27:12.559 --> 00:27:13.360
This guy rocks.

s438
00:27:13.360 --> 00:27:15.039
Uh and work OS is really cool.

s439
00:27:15.039 --> 00:27:27.520
But he uh work OS is an authentication platform and he was deciding okay so with this new platform do I make it like a on-rem thing like I sell you a license now you can run this

s440
00:27:27.520 --> 00:27:43.600
uh authentication thing on prem or do I make it like an open source thing with like a commercial ARM or whatever and after interviewing and talking with a lot of customers he realized oh no like if there is some sort of security problem then we need to push out a fix immediately and we can't wait for people

s441
00:27:43.600 --> 00:27:54.559
to upgrade some npm package for that and so therefore I will make it a hosted solution and so your technical expertise and understanding married to the understanding of the

s442
00:27:54.559 --> 00:28:09.360
um the user and the actual problems is going to make a significant impact on the direction that you take the uh product um oh gosh you know what I'm going to it's been way too long since you've actually done anything so we're going to skip over this you need

s443
00:28:09.360 --> 00:28:19.600
hopefully you get the point there is a line between product manager and product engineer and both of them need to really understand the uh the actual problems that the user has.

s444
00:28:19.600 --> 00:28:33.760
So uh one of the one of the things that you are doing as a product engineer is um building a system for your underlings to be successful in your

s445
00:28:33.760 --> 00:28:42.559
uh even like years ago before we had AI agents you would have a team lead and their job was u to make sure that that system that you're working in is very efficient for you.

s446
00:28:42.559 --> 00:28:44.799
So you got good testing, you've got good architecture.

s447
00:28:44.799 --> 00:28:48.880
It's very obvious which Lego blocks to put together to build out uh different features.

s448
00:28:48.880 --> 00:28:50.960
These primitives are very important.

s449
00:28:50.960 --> 00:28:55.760
We are now all team leads over however many agents you're able to run at once.

s450
00:28:55.760 --> 00:29:01.200
And it's your job to make sure that that playground is a really nice playground and they don't make a ton of mistakes.

s451
00:29:01.200 --> 00:29:03.760
Okay, I'm going to skip uh ahead.

s452
00:29:03.760 --> 00:29:05.120
Here we go.

s453
00:29:05.120 --> 00:29:10.880
Um, and uh gosh, I really want to get you you all doing stuff here really quick.

s454
00:29:10.880 --> 00:29:12.799
So, I don't want to miss anything.

s455
00:29:12.799 --> 00:29:15.440
Oh, okay.

s456
00:29:15.440 --> 00:29:21.440
There's so many things that you miss if you don't have product understanding.

s457
00:29:21.760 --> 00:29:23.440
I just I got to tell this story.

s458
00:29:23.440 --> 00:29:24.559
Who knows Don Norman?

s459
00:29:24.559 --> 00:29:25.840
Who ever heard of Don Norman?

s460
00:29:25.840 --> 00:29:27.279
This guy legend.

s461
00:29:27.279 --> 00:29:28.320
He's 90 years old.

s462
00:29:28.320 --> 00:29:29.600
He comes on my podcast.

s463
00:29:29.600 --> 00:29:31.520
Like, that first of all is pretty wild.

s464
00:29:31.520 --> 00:29:39.200
and he has so much he this is the guy who invented the term user experience when he was at Apple.

s465
00:29:39.200 --> 00:29:44.240
So like yeah kind of a cool dude um and just delight to work with.

s466
00:29:44.240 --> 00:29:49.120
Well uh how many of you are familiar with the three mile island um incident?

s467
00:29:49.120 --> 00:29:49.520
Okay.

s468
00:29:49.520 --> 00:29:53.200
1979 in Pennsylvania, there's a nuclear reactor.

s469
00:29:53.200 --> 00:29:58.720
Something terrible goes wrong and um and like there's a cool issue and and whatever.

s470
00:29:58.720 --> 00:30:06.720
Uh it was a technical issue and these operators made some um bad decisions based off of the technical problems they were experiencing.

s471
00:30:06.720 --> 00:30:09.600
So they bring in Don to come and be like, "What happened?

s472
00:30:09.600 --> 00:30:12.960
Why did these operators make such a terrible decision?"

s473
00:30:12.960 --> 00:30:15.360
And he looked at the problem totally differently.

s474
00:30:15.360 --> 00:30:16.399
He said, "You know what?

s475
00:30:16.399 --> 00:30:18.000
Actually, it was not their fault.

s476
00:30:18.000 --> 00:30:18.880
They're smart.

s477
00:30:18.880 --> 00:30:20.240
Uh they're competent.

s478
00:30:20.240 --> 00:30:21.520
The problem was the system.

s479
00:30:21.520 --> 00:30:22.640
The system was wrong.

s480
00:30:22.640 --> 00:30:24.240
User error does not exist.

s481
00:30:24.240 --> 00:30:25.760
That is his assertion.

s482
00:30:25.760 --> 00:30:30.960
So if you want to learn more from Don, the design of everyday things is like the canonical book.

s483
00:30:30.960 --> 00:30:32.960
You absolutely should read or listen to that.

s484
00:30:32.960 --> 00:30:35.440
Uh and he also wrote design for a better world.

s485
00:30:35.440 --> 00:30:38.799
Also a really u great initiative there too.

s486
00:30:38.799 --> 00:30:45.039
So I sorry I had to jump like this guy is fabulous and really understands users.

s487
00:30:45.039 --> 00:30:45.919
Okay.

s488
00:30:45.919 --> 00:30:55.279
So before we get into um talking about the specific app and and the frameworks that we're going to get into, I wanted to double check if uh there are any questions.

s489
00:30:55.279 --> 00:31:04.880
So let me pull those up and um we'll we'll get to our example uh software here in just a second.

s490
00:31:04.880 --> 00:31:06.799
Um we've got one question in here.

s491
00:31:06.799 --> 00:31:10.240
How many features are too many for an MVP?

s492
00:31:10.240 --> 00:31:17.360
Um, that depends on like tons of factors, but I would I would actually say that that is the wrong question.

s493
00:31:17.360 --> 00:31:24.640
The the question is, and we'll get into some of the the frameworks here in a little bit that will kind of help answer this, but the question it really is

s494
00:31:24.640 --> 00:31:32.000
uh what is the minimal thing that you can do to demonstrate that you solve uh a problem for uh your target audience?

s495
00:31:32.000 --> 00:31:38.720
Uh and in particular, if you're entering a space that's already overcrowded, you want to niche down as well on that target audience.

s496
00:31:38.720 --> 00:31:40.880
So, we'll talk a little bit more about this.

s497
00:31:40.880 --> 00:31:47.360
Um, but since there are no other questions on here, does anybody have a question that they didn't put on here, but they want to ask?

s498
00:31:47.360 --> 00:31:47.760
Go ahead.

s499
00:31:47.760 --> 00:31:50.240
You can just yell it and I will repeat it.

s500
00:31:50.240 --> 00:31:51.360
Oh, we've got a couple.

s501
00:31:51.360 --> 00:31:52.480
Is there a link to my slides?

s502
00:31:52.480 --> 00:31:53.919
There will be at the end.

s503
00:31:53.919 --> 00:32:01.519
Um, right now it's local host, so I can't um but but at the at the very end there is actually a link to uh to the slides.

s504
00:32:01.519 --> 00:32:05.120
So, haha, you have to stay uh or watch it later.

s505
00:32:05.120 --> 00:32:10.640
Um uh how do you evaluate software engineers in the hiring process for product engineering?

s506
00:32:10.640 --> 00:32:21.679
Uh okay so I this is a great question hiring is really really tough to say the thing that everybody knows already.

s507
00:32:21.679 --> 00:32:31.440
Um having really good technical expertise as I explained is still very important.

s508
00:32:31.440 --> 00:32:34.640
Uh I would not be doing coding challenges at all.

s509
00:32:34.640 --> 00:32:43.519
Uh that is like I certainly wouldn't be doing any coding challenges where I say no you are not allowed to use a an agent to do this.

s510
00:32:43.519 --> 00:32:45.840
Like what are you hiring them to do?

s511
00:32:45.840 --> 00:32:49.360
Like how awful is it to work at your company that I can't use an AI agent?

s512
00:32:49.360 --> 00:32:49.919
Goodness.

s513
00:32:49.919 --> 00:32:53.120
So no instead you're going to be talking about systems.

s514
00:32:53.120 --> 00:33:02.240
um you're going to be talking about how to uh in fact I would probably take a couple of the things that we're going to do here in a second and I would just ask them like here's an example

s515
00:33:02.240 --> 00:33:10.640
what are the questions that you're going to ask to figure out what the the core problem is and then and then once they say okay yeah you got what the core problem is

s516
00:33:10.640 --> 00:33:22.640
now from there what are the system implications from the um that discovery of the core problem that would be how my my interview for that would go that's a great

s517
00:33:22.640 --> 00:33:23.840
Um, I lost it.

s518
00:33:23.840 --> 00:33:24.880
Uh, okay.

s519
00:33:24.880 --> 00:33:27.279
When do you stop talking to customers and start building?

s520
00:33:27.279 --> 00:33:28.640
Uh, I think that you do both.

s521
00:33:28.640 --> 00:33:29.200
Constantly.

s522
00:33:29.200 --> 00:33:29.840
Always.

s523
00:33:29.840 --> 00:33:31.039
You do not stop.

s524
00:33:31.039 --> 00:33:34.159
Uh, it's it's a feedback loop that continues to go.

s525
00:33:34.159 --> 00:33:40.240
We'll get into this a little bit uh as we go um further on in the workshop here as well.

s526
00:33:40.240 --> 00:33:43.120
Uh, not only what to build, but how to get users to use it.

s527
00:33:43.120 --> 00:33:44.880
Traction is the new moat.

s528
00:33:44.880 --> 00:33:45.919
Um, okay.

s529
00:33:45.919 --> 00:33:48.080
So, I don't actually talk about this specifically.

s530
00:33:48.080 --> 00:33:50.960
Distribution is is definitely a difficult problem.

s531
00:33:50.960 --> 00:33:53.600
Uh, you just get Theo to talk about it on his YouTube channel.

s532
00:33:53.600 --> 00:33:54.399
That's that's the answer.

s533
00:33:54.399 --> 00:33:55.200
No, just kidding.

s534
00:33:55.200 --> 00:33:57.919
Um, yeah, that's a that actually is a really difficult one.

s535
00:33:57.919 --> 00:34:05.200
Uh, I have heard that people will just spend an unreasonable amount of money on u marketing and advertisement and that seems to work for them.

s536
00:34:05.200 --> 00:34:12.079
So, um, yeah, this is not something we're uh really going to uh to talk in uh talk about and I'm not prepared to answer.

s537
00:34:12.079 --> 00:34:16.079
So, sorry to not be uh all that helpful with that one.

s538
00:34:16.079 --> 00:34:17.040
Uh, okay.

s539
00:34:17.040 --> 00:34:20.480
And we Oh, did I just mark one that was that I didn't answer?

s540
00:34:20.480 --> 00:34:21.839
What did that question say?

s541
00:34:21.839 --> 00:34:23.119
I think it moved on me.

s542
00:34:23.119 --> 00:34:25.040
There's a bad user experience right there.

s543
00:34:25.040 --> 00:34:27.839
Somebody who works at Slido can go fix that.

s544
00:34:27.839 --> 00:34:29.919
Um, okay.

s545
00:34:29.919 --> 00:34:32.480
Uh, I'll I'll answer like three more questions then we'll move on.

s546
00:34:32.480 --> 00:34:36.720
Would you recommend product managers stay in product teams and product engineers within Oh goodness.

s547
00:34:36.720 --> 00:34:44.560
Uh, as far as organization is concerned, um, often applications end up uh, looking a lot like their org chart.

s548
00:34:44.560 --> 00:34:55.520
So um I would huh uh I I definitely feel personally that your product u manager should be very integrated with engineering.

s549
00:34:55.520 --> 00:34:58.880
That communication layer should be very tight.

s550
00:34:58.880 --> 00:35:10.640
Um and your engineers should also not be siloed from other engineers in the organization because one important uh thing for a product engineer to do is not only look at their slice of the application

s551
00:35:10.640 --> 00:35:21.520
but also step back and see at least their neighbors because if you don't then you'll end up building things that are already solved for you by other primitives that are within the organization

s552
00:35:21.520 --> 00:35:24.800
and uh and so you're just rebuilding those same things.

s553
00:35:24.800 --> 00:35:29.839
So you need to at least go one neighbor out and one neighbor up and down.

s554
00:35:29.839 --> 00:35:43.520
Uh and this is actually the same with um programming languages like if you're a React developer then you need to understand how JavaScript and the browser work so that you can use React effectively even though you're not necessarily using all of the features

s555
00:35:43.520 --> 00:35:46.960
um that are included there because the framework is doing that for you.

s556
00:35:46.960 --> 00:35:49.040
So it's the same sort of idea.

s557
00:35:49.040 --> 00:35:51.760
Um tip highlight the questions to bring the focus.

s558
00:35:51.760 --> 00:35:52.480
How do I highlight?

s559
00:35:52.480 --> 00:35:55.280
Do I just click on it?

s560
00:35:55.280 --> 00:35:58.880
All right, we're doing a product lesson right now.

s561
00:35:58.880 --> 00:36:00.240
That is a a useful tip.

s562
00:36:00.240 --> 00:36:01.119
It seems like a good idea.

s563
00:36:01.119 --> 00:36:02.079
Oh, is that what that is?

s564
00:36:02.079 --> 00:36:03.280
All right, there we go.

s565
00:36:03.280 --> 00:36:04.480
User error.

s566
00:36:04.480 --> 00:36:05.520
No, it's not.

s567
00:36:05.520 --> 00:36:06.765
It doesn't exist.

s568
00:36:06.765 --> 00:36:07.599
[laughter]

s569
00:36:07.599 --> 00:36:08.480
Uh, okay.

s570
00:36:08.480 --> 00:36:10.000
So, if I highlight it.

s571
00:36:10.000 --> 00:36:10.720
Okay, that's cool.

s572
00:36:10.720 --> 00:36:13.119
Should all engineers be product engineers?

s573
00:36:13.119 --> 00:36:27.440
Um, well, in my estimation, if you're not a product engineer, if you don't have product sense or design sense, it's that's going to be really easy to replace you by a a as the agents get like continue to get competent.

s574
00:36:27.440 --> 00:36:30.560
I don't want to be like alarmist or anything like you know, whatever.

s575
00:36:30.560 --> 00:36:40.400
But it does seem like if all you're doing is listening to somebody tell you what to do and then turning that into a code implementation, that part seems like it's pretty replaceable.

s576
00:36:40.400 --> 00:36:46.079
If you don't understand the product, uh, then the system you build will not solve the user's problem.

s577
00:36:46.079 --> 00:36:46.880
Well, okay.

s578
00:36:46.880 --> 00:36:52.480
I think we will come back to these questions uh as they're voted up a little bit more later.

s579
00:36:52.480 --> 00:37:00.640
So, after um quite a while into the workshop, we're going to finally talk about what the workshop is going to um be talking about.

s580
00:37:00.640 --> 00:37:07.760
So, um what I want you to leave with is um frameworks for early idea validation.

s581
00:37:07.760 --> 00:37:09.520
How many of you have heard of the mom test?

s582
00:37:09.520 --> 00:37:10.160
Okay, sweet.

s583
00:37:10.160 --> 00:37:11.920
So, we'll be talking about that a little bit.

s584
00:37:11.920 --> 00:37:14.079
Framing user request jobs theory.

s585
00:37:14.079 --> 00:37:14.560
Anybody?

s586
00:37:14.560 --> 00:37:16.240
Okay, sweet.

s587
00:37:16.240 --> 00:37:18.160
And prioritizing software changes.

s588
00:37:18.160 --> 00:37:19.599
Kano model.

s589
00:37:19.599 --> 00:37:20.079
Okay.

s590
00:37:20.079 --> 00:37:20.320
Yeah.

s591
00:37:20.320 --> 00:37:22.800
So, I one of you raised your hand three times.

s592
00:37:22.800 --> 00:37:26.320
So, I invite you to come up and you can talk about this.

s593
00:37:26.320 --> 00:37:27.520
No, just kidding.

s594
00:37:27.520 --> 00:37:28.720
Uh, okay.

s595
00:37:28.720 --> 00:37:29.440
So, here we go.

s596
00:37:29.440 --> 00:37:30.560
This is the dangerous thing.

s597
00:37:30.560 --> 00:37:34.560
What is the idea that we're going to be framing things around?

s598
00:37:34.560 --> 00:37:35.760
And this is a risk.

s599
00:37:35.760 --> 00:37:39.280
I've never given this talk before so this might go really really poorly.

s600
00:37:39.280 --> 00:37:43.520
Um hopefully not but uh let's look at what we've got.

s601
00:37:43.520 --> 00:37:50.640
The top voted with fire is an app that gets people into conference workshop sessions.

s602
00:37:50.640 --> 00:37:51.920
Uh uh yeah.

s603
00:37:51.920 --> 00:37:52.480
Okay.

s604
00:37:52.480 --> 00:37:54.320
I I literally am going to do that.

s605
00:37:54.320 --> 00:37:56.240
Um okay.

s606
00:37:56.240 --> 00:37:57.520
What should we call it?

s607
00:37:57.520 --> 00:38:02.160
What's what's our short um how about Lightning Lane Workshop?

s608
00:38:02.160 --> 00:38:04.960
You know, call back to uh to Disneyland, right?

s609
00:38:04.960 --> 00:38:05.680
Lightning lanes.

s610
00:38:05.680 --> 00:38:08.000
Yeah, lightning lane.

s611
00:38:08.000 --> 00:38:09.119
Just get in.

s612
00:38:09.119 --> 00:38:09.359
Oh.

s613
00:38:09.359 --> 00:38:09.920
Oh, there.

s614
00:38:09.920 --> 00:38:10.560
That's so good.

s615
00:38:10.560 --> 00:38:12.000
Just get in.

s616
00:38:12.000 --> 00:38:14.640
Oh, I love that.

s617
00:38:14.880 --> 00:38:16.079
Just get in.

s618
00:38:16.079 --> 00:38:16.720
I don't know.

s619
00:38:16.720 --> 00:38:20.480
Like, are are some of you thinking like weird things?

s620
00:38:20.480 --> 00:38:23.119
The workshop.

s621
00:38:23.760 --> 00:38:25.626
And sick people.

s622
00:38:25.626 --> 00:38:27.040
[laughter]

s623
00:38:27.040 --> 00:38:29.359
I just heard everybody laughing like why are they laughing?

s624
00:38:29.359 --> 00:38:30.079
Oh, okay.

s625
00:38:30.079 --> 00:38:31.040
Gosh.

s626
00:38:31.040 --> 00:38:31.760
Okay.

s627
00:38:31.760 --> 00:38:37.599
So let's talk about early idea validation for the just get in the workshop app.

s628
00:38:38.079 --> 00:38:39.119
How do we validate this?

s629
00:38:39.119 --> 00:38:40.480
All right, throw out some questions.

s630
00:38:40.480 --> 00:38:41.440
What are question?

s631
00:38:41.440 --> 00:38:42.480
You're at a conference.

s632
00:38:42.480 --> 00:38:46.880
This is the perfect place to talk about the just get in the workshop app.

s633
00:38:46.880 --> 00:38:48.480
So you you've got all these people.

s634
00:38:48.480 --> 00:38:51.520
You're standing in line and you're like, go this is so awful.

s635
00:38:51.520 --> 00:38:55.359
What are the questions that you're asking people to validate that your idea is a good idea?

s636
00:38:55.359 --> 00:38:56.240
So just throw them out.

s637
00:38:56.240 --> 00:38:57.359
Raise your hands if you want to.

s638
00:38:57.359 --> 00:38:57.839
Yeah.

s639
00:38:57.839 --> 00:38:59.520
Why do we need that app for that?

s640
00:38:59.520 --> 00:39:01.440
Why do we need an app for that?

s641
00:39:01.440 --> 00:39:04.000
Okay, that so ask them, is it a problem?

s642
00:39:04.000 --> 00:39:04.800
Is this a problem?

s643
00:39:04.800 --> 00:39:05.680
That's Yeah.

s644
00:39:05.680 --> 00:39:06.880
How long have you

s645
00:39:06.880 --> 00:39:08.160
How long have you been waiting?

s646
00:39:08.160 --> 00:39:08.480
Yeah.

s647
00:39:08.480 --> 00:39:11.680
Okay, that that's getting really close to a really good question.

s648
00:39:11.680 --> 00:39:13.680
Other questions?

s649
00:39:13.680 --> 00:39:16.000
Sorry, what?

s650
00:39:16.079 --> 00:39:17.200
Why are you waiting?

s651
00:39:17.200 --> 00:39:17.680
Oh, okay.

s652
00:39:17.680 --> 00:39:18.560
Yeah, that's a good question.

s653
00:39:18.560 --> 00:39:20.560
Yeah.

s654
00:39:21.440 --> 00:39:23.280
Oh, could there be a pre-registration?

s655
00:39:23.280 --> 00:39:24.160
Like, yeah.

s656
00:39:24.160 --> 00:39:26.480
Wouldn't it be better if this was a pre-registration thing?

s657
00:39:26.480 --> 00:39:27.119
Yeah.

s658
00:39:27.119 --> 00:39:29.599
When was the last time you were stuck?

s659
00:39:29.599 --> 00:39:31.119
Oh, you've read the mom test.

s660
00:39:31.119 --> 00:39:31.440
All right.

s661
00:39:31.440 --> 00:39:31.839
All right.

s662
00:39:31.839 --> 00:39:33.839
He said, "When was the last time you were stuck?"

s663
00:39:33.839 --> 00:39:35.520
Yeah, that's that's okay.

s664
00:39:35.520 --> 00:39:35.839
Okay.

s665
00:39:35.839 --> 00:39:38.720
What What else do we have?

s666
00:39:40.720 --> 00:39:40.960
Sorry.

s667
00:39:40.960 --> 00:39:43.040
Say that again.

s668
00:39:43.040 --> 00:39:45.359
Oh, what's the max time we're expected to wait?

s669
00:39:45.359 --> 00:39:45.680
Yeah.

s670
00:39:45.680 --> 00:39:46.160
Okay.

s671
00:39:46.160 --> 00:39:48.000
Are there any open seats?

s672
00:39:48.000 --> 00:39:49.359
Are there any open seats?

s673
00:39:49.359 --> 00:39:49.760
Yeah.

s674
00:39:49.760 --> 00:39:50.640
Okay.

s675
00:39:50.640 --> 00:39:50.960
Okay.

s676
00:39:50.960 --> 00:40:03.599
So these are some questions that you might think okay if they what you're looking for or what we naturally are looking for is somebody to say oh yeah this is a huge problem for me I definitely want some I would I would spend

s677
00:40:03.599 --> 00:40:09.200
just untold sums of money to be able to get in the workshop really quickly and easily.

s678
00:40:09.200 --> 00:40:11.440
I I think you had another question.

s679
00:40:11.440 --> 00:40:17.359
I got another one like what is your what's what your experience while you wait

s680
00:40:17.359 --> 00:40:19.520
oh yeah how's your experience been while waiting?

s681
00:40:19.520 --> 00:40:23.920
So you're trying to like get a sense for uh for that overarching experience.

s682
00:40:23.920 --> 00:40:24.079
Yeah.

s683
00:40:24.079 --> 00:40:27.520
And that can be a good input into the solution.

s684
00:40:27.520 --> 00:40:29.040
Is this a solved problem?

s685
00:40:29.040 --> 00:40:29.760
Oh yeah.

s686
00:40:29.760 --> 00:40:33.520
Like have you heard of an app that could really have sped this process up?

s687
00:40:33.520 --> 00:40:35.440
I saw somebody holding up the mom test book.

s688
00:40:35.440 --> 00:40:38.593
You literally have it in your Unbelievable.

s689
00:40:38.593 --> 00:40:39.520
[laughter]

s690
00:40:39.520 --> 00:40:40.800
That's awesome.

s691
00:40:40.800 --> 00:40:43.833
Uh I I also carry my mom test book with me everywhere I

s692
00:40:43.833 --> 00:40:43.839
[laughter]

s693
00:40:43.839 --> 00:40:45.119
go.

s694
00:40:45.119 --> 00:40:45.760
Uh yes.

s695
00:40:45.760 --> 00:40:47.280
So this is the mom test.

s696
00:40:47.280 --> 00:40:51.599
You don't want to ask users to evaluate your idea or diagnose the problem for you.

s697
00:40:51.599 --> 00:41:01.599
This actually this is so common and it's so hard because when you think of an idea, you start thinking, okay, so we will we'll make it uh a mobile app.

s698
00:41:01.599 --> 00:41:03.200
Obviously, they have to install it on their phone.

s699
00:41:03.200 --> 00:41:06.480
Oh, wait, no. Uh maybe like it's a one-time use thing.

s700
00:41:06.480 --> 00:41:14.079
So, we'll also make a a website and then we're going to uh integrate with all of the different uh providers of all these different things.

s701
00:41:14.079 --> 00:41:24.240
And uh and so then you start talking with people and you say, "Would it be easier if it was already integrated with the app, maybe the conference had a link to it or maybe it was integrated?"

s702
00:41:24.240 --> 00:41:36.720
And so you're talking about the solution to your specific solution with this u potential possible user before they've even agreed that this is uh a problem worth solving.

s703
00:41:36.720 --> 00:41:37.200
Okay.

s704
00:41:37.200 --> 00:41:42.079
Um so here are a couple of the weak questions that may sound a little familiar.

s705
00:41:42.079 --> 00:41:43.839
Would you use this?

s706
00:41:43.839 --> 00:41:46.319
Is it a is this a problem in the first place?

s707
00:41:46.319 --> 00:41:47.200
What is the problem?

s708
00:41:47.200 --> 00:41:48.240
Help me define the problem.

s709
00:41:48.240 --> 00:41:54.000
Some better questions are tell me about the last time this happened and what did you do instead?

s710
00:41:54.000 --> 00:41:56.079
And what did it cost you to do that?

s711
00:41:56.079 --> 00:42:02.240
Now for ours, none of us I think would actually I don't know you all paid to come here to the conference.

s712
00:42:02.240 --> 00:42:08.720
So there is some amount of like I'm waiting I'm missing the first 20 minutes of this talk that I actually paid to attend.

s713
00:42:08.720 --> 00:42:10.079
Don't get mad at Sean.

s714
00:42:10.079 --> 00:42:11.680
This these things happen.

s715
00:42:11.680 --> 00:42:12.880
Sean's a friend of mine.

s716
00:42:12.880 --> 00:42:19.119
U but uh but yeah, like the the questions of what did you do instead?

s717
00:42:19.119 --> 00:42:20.480
What is your workaround?

s718
00:42:20.480 --> 00:42:21.920
And how much did it cost you?

s719
00:42:21.920 --> 00:42:24.079
Because a solution like this doesn't exist.

s720
00:42:24.079 --> 00:42:27.280
You do not talk about the solution at this stage.

s721
00:42:27.280 --> 00:42:28.400
It's so hard not to.

s722
00:42:28.400 --> 00:42:29.280
But you don't.

s723
00:42:29.280 --> 00:42:32.800
Instead, you start with um what did you do before?

s724
00:42:32.800 --> 00:42:34.160
What what is the past?

s725
00:42:34.160 --> 00:42:36.240
You do not want to ask them future questions.

s726
00:42:36.240 --> 00:42:37.920
Nobody can tell the future.

s727
00:42:37.920 --> 00:42:39.520
You don't want to say would you pay?

s728
00:42:39.520 --> 00:42:40.960
how much would you pay?

s729
00:42:40.960 --> 00:42:49.520
Whatever that often what that turns into is somebody's like, I kind of want to get out of this conversation and I'm just going to say yes to everything that they say.

s730
00:42:49.520 --> 00:42:51.839
And in the book they call this complimenting.

s731
00:42:51.839 --> 00:42:56.079
So that you're just you're you're asking you're fishing for compliments.

s732
00:42:56.079 --> 00:43:06.000
Uh so Don Norman said don't ask somebody what's a problem because they'll tell you the symptoms and often they will actually tell you solutions as well and their solutions will be just as ill-formed as your own

s733
00:43:06.000 --> 00:43:08.160
uh and maybe even more so.

s734
00:43:08.160 --> 00:43:11.359
Um, so you don't want to just ask generally what is the problem.

s735
00:43:11.359 --> 00:43:13.520
You want to ask about specific behavior.

s736
00:43:13.520 --> 00:43:21.520
So tell me about the last time this happened, what happened, give me specifics and who was involved, what made it annoying enough to remember.

s737
00:43:21.520 --> 00:43:28.160
Now our example is kind of um kind of funny because you if you're in line, you're literally experiencing the problem right now.

s738
00:43:28.160 --> 00:43:35.040
So, it's a little visceral, but most of the time you're not um like your idea to solve somebody's problem.

s739
00:43:35.040 --> 00:43:38.160
They're not actively experiencing that problem right at that very moment.

s740
00:43:38.160 --> 00:43:41.440
So, it's not necessarily going to be so much, well, tell me the last time it happened.

s741
00:43:41.440 --> 00:43:42.400
Well, it's happening right now.

s742
00:43:42.400 --> 00:43:43.200
Like, maybe it is.

s743
00:43:43.200 --> 00:43:44.640
And that's a pretty good signal.

s744
00:43:44.640 --> 00:43:48.480
Then, you're going to be able to get some really um direct answers to that.

s745
00:43:48.480 --> 00:43:54.640
But often, you're going to have to let them think back about like, oh yeah, why why was this so bad and who was involved?

s746
00:43:54.640 --> 00:43:57.200
What made it annoying enough that I can even remember?

s747
00:43:57.200 --> 00:44:04.880
If they can't remember, that's a signal and it's not the one that you want it to be, but you want that signal, that's for sure.

s748
00:44:04.880 --> 00:44:07.520
So, uh, you want to know what they did do instead.

s749
00:44:07.520 --> 00:44:08.400
So, what are the workaround?

s750
00:44:08.400 --> 00:44:09.040
Oh, whoops.

s751
00:44:09.040 --> 00:44:10.400
Ah, back up.

s752
00:44:10.400 --> 00:44:11.680
Uh, how much does it cost?

s753
00:44:11.680 --> 00:44:14.000
How often is this happening to them?

s754
00:44:14.000 --> 00:44:18.880
Uh, how much effort are they putting into solving this problem already?

s755
00:44:18.880 --> 00:44:20.560
And that is gold.

s756
00:44:20.560 --> 00:44:26.319
That is, um, a really strong evidence that this is a problem we're solving.

s757
00:44:26.319 --> 00:44:38.160
if they're already putting a bunch of effort and paying a bunch of money to solve this problem with some workaround um that's a really good opportunity if they are like yeah I I don't remember the last time I know it's happened

s758
00:44:38.160 --> 00:44:46.720
yeah I don't really remember u and I just I kind of gave up that that that's a pretty good signal too like that's a signal that they're probably not going to pay

s759
00:44:46.720 --> 00:44:59.280
uh and and change their workflows or whatever uh to get a solution now um the uh yeah u the the point here is that as software engineers, we've gotten kind of used to just like sitting in our corner.

s760
00:44:59.280 --> 00:45:10.079
Well, some of us I'm kind of an extrovert, so I I like to talk to people, but uh some of us are just like, "Yeah, I got into software engineering because I never have to talk to anybody and I can just like take my jar ticket and turn it into implementation."

s761
00:45:10.079 --> 00:45:13.760
I that I'm sorry, that's not the way that software engineering ever was.

s762
00:45:13.760 --> 00:45:16.079
The best software engineers were not that way.

s763
00:45:16.079 --> 00:45:27.200
Um, I mean, you could turn out some pretty amazing code, don't get me wrong, but the software engineer who really talks to people and understands their problems is the one who's going to build a system that can solve those problems better.

s764
00:45:27.200 --> 00:45:36.960
Uh, and uh, one one thing that Michael did actually in our conversation, he was talking to so many people validating his idea before he uh, built work OS.

s765
00:45:36.960 --> 00:45:40.880
Uh, and he would ask them what are they losing out by not having a solution to this?

s766
00:45:40.880 --> 00:45:42.640
What's the cost of not doing that?

s767
00:45:42.640 --> 00:45:44.560
And that worked out for him.

s768
00:45:44.560 --> 00:45:50.640
So the output is uh like actual evidence rather than somebody just saying, "Oh yeah, that's a good idea."

s769
00:45:50.640 --> 00:45:56.319
And this is why it's called the mom test because your mom, of course, she loves you and she's going to be like, "Yeah, of course this is wonderful.

s770
00:45:56.319 --> 00:46:03.839
I would absolutely use it and I would and so now you have two users, yourself and your mom and uh or dad or whatever."

s771
00:46:03.839 --> 00:46:04.960
And and that's wonderful.

s772
00:46:04.960 --> 00:46:05.680
Good for you.

s773
00:46:05.680 --> 00:46:10.720
Um so um here's a a fun story from Wayne.

s774
00:46:10.720 --> 00:46:12.480
Uh he he lives down in Australia.

s775
00:46:12.480 --> 00:46:21.040
He worked for a real estate uh company down there and they were building a product uh that was kind of an experiment in in this bigger uh organization.

s776
00:46:21.040 --> 00:46:36.800
They he was on a team that's building this product and in the process of that they ended up building a really highcale thing and they in fact decided what if every single person in Australia logged in at the exact same time we need to make sure that we handle that.

s777
00:46:36.800 --> 00:46:39.040
And uh they spent a year working on this.

s778
00:46:39.040 --> 00:46:41.440
It was like microservices the whole thing.

s779
00:46:41.440 --> 00:46:56.640
Um, and after it was 1.2 million u Australian dollars that they spent on this, which is like almost 900,000 US dollars, um, they uh ended up like nobody used it, not a single person.

s780
00:46:56.640 --> 00:47:04.800
So like not only did they not have to handle every person in Australia, they they didn't even have to handle one person in Australia, nobody ended up using this thing.

s781
00:47:04.800 --> 00:47:13.280
And so he told me that uh something he learned from that is we probably could have launched something in two weeks, not a year and done the integration manually.

s782
00:47:13.280 --> 00:47:16.000
The integration is always the most painful part anyway.

s783
00:47:16.000 --> 00:47:21.920
Uh and we could have tested the market, learned a lot of things very very quickly and not invested a whole team to work on this.

s784
00:47:21.920 --> 00:47:27.680
So project validation really important and it's again not just the PM's job to do this.

s785
00:47:27.680 --> 00:47:33.440
It's your job to really understand okay so this and and we're not just talking about startups here either.

s786
00:47:33.440 --> 00:47:37.599
this is a product feature that you've been asked to do.

s787
00:47:37.599 --> 00:47:50.720
And if if that feature request doesn't come with some sort of validation of why this exists, then you as a product engineer need to go back to the PM and say, I need to understand a little bit more about why this exists

s788
00:47:50.720 --> 00:47:59.599
for not only to validate that this is going to be worth engineering time um but also for some of the other questions that we're going to talk about here in a second.

s789
00:47:59.599 --> 00:48:08.560
But part of that is um understanding what like where this request is coming from will help you shape the system that you use to build it.

s790
00:48:08.560 --> 00:48:12.880
Is this just like a one-off experiment or is this something that's going to be core to our business?

s791
00:48:12.880 --> 00:48:15.920
I'm going to make different technical trade-offs based on that.

s792
00:48:15.920 --> 00:48:18.240
Like how much investment do I need to spend?

s793
00:48:18.240 --> 00:48:20.240
Should I spend a year working on this?

s794
00:48:20.240 --> 00:48:23.119
Is this like the business will go out of business if we don't have this?

s795
00:48:23.119 --> 00:48:23.200
Okay.

s796
00:48:23.200 --> 00:48:23.359
Okay.

s797
00:48:23.359 --> 00:48:34.800
Well, I'm going to uh spend uh a little bit more time and attention to detail uh while still like balancing the um MVP style like let's just get some validation.

s798
00:48:35.839 --> 00:48:37.599
Okay.

s799
00:48:37.599 --> 00:48:40.480
I'm really glad I grabbed that water.

s800
00:48:40.480 --> 00:48:43.040
Uh yeah, this is something Michael did as well.

s801
00:48:43.040 --> 00:48:43.760
Very easy.

s802
00:48:43.760 --> 00:48:44.079
Yeah.

s803
00:48:44.079 --> 00:48:46.079
Gosh, this is so relatable.

s804
00:48:46.079 --> 00:48:51.839
It's very easy to convince yourself that something is a good idea just sitting in your room by yourself hacking on it for weeks and weeks and weeks.

s805
00:48:51.839 --> 00:48:53.680
Uh, so you need to show it to people.

s806
00:48:53.680 --> 00:48:57.680
Uh, I How many people relate to that just like Yeah.

s807
00:48:57.680 --> 00:48:59.760
Uh, it's fun, right?

s808
00:48:59.760 --> 00:49:01.280
Like why you got to rate my parade?

s809
00:49:01.280 --> 00:49:02.640
I'm having fun doing this thing.

s810
00:49:02.640 --> 00:49:04.880
And then you go show it to somebody and they don't get it.

s811
00:49:04.880 --> 00:49:06.160
So you don't want to talk about it.

s812
00:49:06.160 --> 00:49:08.160
You just want to work on it yourself.

s813
00:49:08.160 --> 00:49:10.153
That was a signal right there.

s814
00:49:10.153 --> 00:49:10.240
[laughter]

s815
00:49:10.240 --> 00:49:16.559
Like if they if they don't get what you why you're so excited about it, then that yeah, you want to pay attention to that feeling.

s816
00:49:16.559 --> 00:49:20.160
Um, and then we need to close the loop with user feedback.

s817
00:49:20.160 --> 00:49:22.800
is part of like shrinking things down.

s818
00:49:22.800 --> 00:49:35.280
So like the validation of the idea that the and the mom test questions we talked about earlier that is like telling you whether it's worth um working or solving the problem in the first place and now we're getting into

s819
00:49:35.280 --> 00:49:41.599
how do you know whether your solution is doing that effectively and uh Ruben says that user feedback is how you do that.

s820
00:49:41.599 --> 00:49:45.839
So uh and also what Michael was doing and and and um Wayne as well.

s821
00:49:45.839 --> 00:49:49.760
So you try to get something some prototype to those early users.

s822
00:49:49.760 --> 00:50:03.040
Early adopters are like totally fine especially if if the solution that you build has a ton of gaping holes then yeah they're going to be like pretty um their reaction to that

s823
00:50:03.040 --> 00:50:04.480
is going to tell you a lot.

s824
00:50:04.480 --> 00:50:13.280
If there are a lot of gaping holes but like it gets them like almost there or like just gets them over to what they're trying to do and they're still excited about it.

s825
00:50:13.280 --> 00:50:15.359
That is very significant.

s826
00:50:15.359 --> 00:50:20.160
if that doesn't quite get them there and they kind of give up early or something and they're like, "Ah, I don't know.

s827
00:50:20.160 --> 00:50:24.640
I could just, you know, I had a couple of these, you know, paper cut issues and so I stopped."

s828
00:50:24.640 --> 00:50:26.400
Okay, that's a really good signal, too.

s829
00:50:26.400 --> 00:50:34.960
The the U problems that are really worth solving are the ones where people will just run through the glass cutting themselves and they're like, "We made it."

s830
00:50:34.960 --> 00:50:37.119
Now, that's what you're trying to uh to go for.

s831
00:50:37.119 --> 00:50:41.200
And and getting early user feedback um through prototypes helps a lot with that.

s832
00:50:41.200 --> 00:50:43.200
So, why product engineers need the MOM test?

s833
00:50:43.200 --> 00:50:58.640
because it helps you translate what the PM is saying um into the shape of the system that you build uh and select the appropriate architecture um especially understanding what do users do now or what what do our potential customers do now to solve this problem

s834
00:50:58.640 --> 00:51:10.400
um can help you determine the architectural shape okay so um there's a lot of opportunity here and um there's going to be a lot of downtime so we need to pay attention to how we do migrations or whatever

s835
00:51:10.400 --> 00:51:16.559
uh okay and yeah it's very easy to end up building the wrong abstraction without that context.

s836
00:51:16.559 --> 00:51:18.319
So, I'm going to pause really quick.

s837
00:51:18.319 --> 00:51:20.800
We'll see what questions you all have.

s838
00:51:20.800 --> 00:51:25.119
Um, so far the most upvoted ones are the ones I'm going to focus on.

s839
00:51:25.119 --> 00:51:28.640
Um, and back here, here we go.

s840
00:51:28.640 --> 00:51:31.680
So, three main things that, uh, let's just highlight that.

s841
00:51:31.680 --> 00:51:32.160
There we go.

s842
00:51:32.160 --> 00:51:35.200
Three main things that distinguish a product engineer from product manager.

s843
00:51:35.200 --> 00:51:36.720
Do we need both?

s844
00:51:36.720 --> 00:51:37.920
The three main things.

s845
00:51:37.920 --> 00:51:46.880
Um, so yeah, kind of talked about this a little bit, but I would say that the product engineer is the one that's I'm not sure if I'm going to get three, but I'll just say a bunch of things.

s846
00:51:46.880 --> 00:51:49.359
Um, product engineer is the one that is actually in the code.

s847
00:51:49.359 --> 00:51:58.800
You may not necessarily be looking at the code anymore, but you are thinking about the system and you're thinking about the uh data types that are available, the primitives that you have available.

s848
00:51:58.800 --> 00:52:09.200
Oh, we do have like a cron job system or Q but there are some problems with that and I and you understand what those constraints are and and what the limitations of your current infrastructure are

s849
00:52:09.200 --> 00:52:20.079
you understand what the available uh primitives are that and so when a request comes in and you understand what's trying to be solved here you can categorize that into whether it fits in the existing architecture

s850
00:52:20.079 --> 00:52:29.760
or if you need to expand beyond that and understanding um that um the ultimate user goal or the job to be the foreshadowing.

s851
00:52:29.760 --> 00:52:37.280
Um that is going to help you decide whether we should actually expand the system with a couple more primitives or if you can reuse existing ones.

s852
00:52:37.280 --> 00:52:46.800
Um that's probably the the number one thing that distinguishes a product engineer from the product manager is just that technical expertise and understanding of the system.

s853
00:52:46.800 --> 00:52:55.920
Uh which a product manager could probably just talk to an AI about over and over and over again to like kind of ramp themselves up, but they should actually be doing other stuff.

s854
00:52:55.920 --> 00:53:01.920
Um and so yeah, I would say um the to answer the question, do we need both?

s855
00:53:01.920 --> 00:53:03.200
This is actually interesting.

s856
00:53:03.200 --> 00:53:04.960
So um I am Oh, sorry.

s857
00:53:04.960 --> 00:53:06.640
I I am self-employed.

s858
00:53:06.640 --> 00:53:11.200
So I do not have a product manager or a product engineer or a CFO or what?

s859
00:53:11.200 --> 00:53:12.480
Like I just work for myself.

s860
00:53:12.480 --> 00:53:13.520
I am all the things.

s861
00:53:13.520 --> 00:53:16.480
And you start a startup often it's just like three of you.

s862
00:53:16.480 --> 00:53:21.280
Sometimes you all three of you are technical or maybe one of you is the the business person or whatever.

s863
00:53:21.280 --> 00:53:26.640
And as the company grows, you eventually start divvying out those responsibilities and you have multiple jobs.

s864
00:53:26.640 --> 00:53:28.319
So yeah, um do we need both?

s865
00:53:28.319 --> 00:53:35.680
It highly depends on where you're at in the process of uh your scaling of your business and and whether you ever want to get there.

s866
00:53:35.680 --> 00:53:46.319
So um I think that now uh you can actually build a much bigger business without splitting out into a much bigger organization uh than you used to.

s867
00:53:46.319 --> 00:53:48.160
Uh great question.

s868
00:53:48.160 --> 00:53:48.559
Okay.

s869
00:53:48.559 --> 00:53:51.200
How small do you think teams Oh, literally just said that.

s870
00:53:51.200 --> 00:53:53.119
Um, pretty small.

s871
00:53:53.119 --> 00:53:56.720
I like I don't know how I can give you a more specific answer to that.

s872
00:53:56.720 --> 00:53:57.440
Uh, okay.

s873
00:53:57.440 --> 00:54:01.440
How would people get promoted from junior to senior in the age of a AI?

s874
00:54:01.440 --> 00:54:07.599
Um, well, huh, I'm not sure how to answer this question.

s875
00:54:07.599 --> 00:54:16.240
I I think uh juniors and seniors are both um like we're all ultimately trying to do the same thing.

s876
00:54:16.240 --> 00:54:30.240
even all the way back before AI um uh the junior was just trying to emulate the s senior as much as they could until they s suddenly or over time become a senior engineer and I think that nothing about that has changed

s877
00:54:30.240 --> 00:54:45.760
uh as far as like actually getting promoted that's just going to be a conversation with your uh with your person uh I should actually stop really quick because I I'm just noticing people walking out and my timer went to zero this session is a two-hour session it's just back toback and I'm planning on just going straight through

s878
00:54:45.760 --> 00:54:49.680
But if there are other sessions you wanted to see, you're welcome to stand up and leave.

s879
00:54:49.680 --> 00:54:54.480
I will just take a mental image of you and frown at you in the hallway later.

s880
00:54:54.480 --> 00:54:54.960
Just kidding.

s881
00:54:54.960 --> 00:54:55.599
I won't do that.

s882
00:54:55.599 --> 00:54:59.200
You're you're welcome to take off if if there was another session you wanted to go to.

s883
00:54:59.200 --> 00:55:03.680
Um because there are so many good ones.