Build the Right Thing: Product Engineering (Part 2) — Kent C. Dodds, EpicProduct.engineer

AI Engineer · 52 min · 831 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

1. 00:13So, a as some of you are uh heading out, um actually maybe we let's make this less awkward.
2. 00:19I don't want anybody feel awkward.
3. 00:21Uh let's move again.
4. 00:22Everybody stand up really quick.
5. 00:24Yeah, I I know you're all like, "Oh, I can't.
6. 00:26We just did this like 20 minutes ago."
7. 00:27Yep.
8. 00:28There you still got to move.
9. 00:29Um, we're not going to do the full uh stretch or uh air squat thing, but you can just stretch a little bit.
10. 00:35Make it less awkward for the people who want to leave.
11. 00:37It's okay.
12. 00:38We don't want them to feel bad.
13. 00:39Okay.
14. 00:40Yep.
15. 00:40You can stretch your legs, whatever you want to do.
16. 00:42Just don't hurt yourself.
17. 00:43Uh, and then go ahead and sit.
18. 00:50Okay.
19. 00:52So, we've got uh I think I've got 50 minutes.
20. 00:565-0.
21. 00:56So that's what you're uh in for if you decide to stay.
22. 01:01So now we've got the just get in the workshop implemented and we're rapidly adding features based on user requests.
23. 01:07So we we've moved our uh business life cycle.
24. 01:10We're in this phase now and a request comes in example request which we will fill in here in just a second.
25. 01:16Um and and as we're going through this I want you to see if you can apply uh some of these principles to your own uh product as well.
26. 01:23Um, okay.
27. 01:24So, what should the live request be?
28. 01:27What did what did somebody ask us to add to just get in the workshop?
29. 01:33Just throw it out or raise your hand.
30. 01:36Yeah.
31. 01:42Oh, so like a way to know how long the queue is.
32. 01:46Okay.
33. 01:46Yeah.
34. 01:51Oh, okay.
35. 01:52A little bit like the Disney Fastpass thing then.
36. 01:54Yeah.
37. 01:54Like a remote weight list.
38. 01:56Yeah.
39. 01:56Okay.
40. 01:59Oh, yeah.
41. 02:00To to know how much uh CO2 you're going to be breathing when you get in there.
42. 02:03Yeah.
43. 02:04That's Yeah.
44. 02:08Oh, okay.
45. 02:10That that one.
46. 02:11So, that would really speed up the process of getting in the workshop.
47. 02:14Okay.
48. 02:14Facial recognition.
49. 02:15I think everybody good with that one?
50. 02:17Okay.
51. 02:18Um, facial recognition.
52. 02:23I think we all get that.
53. 02:24All right.
54. 02:26So, let's first start with what Jack Ryan says uh here to dig uh past the requested button or whatever it is that they're requesting.
55. 02:36So, uh ultimately this is just you keep asking why.
56. 02:39What does it matter?
57. 02:40Why does it matter?
58. 02:41So, why why do we care that there's facial recognition?
59. 02:44What what does that mean in the context of our app?
60. 02:49slow.
61. 02:50It's slow.
62. 02:50So, we're Okay.
63. 02:51So, the problem uh that we're we're thinking about is it's slow.
64. 02:55Okay.
65. 02:55What else?
66. 02:57Is that like the main thing or is there anything else?
67. 03:00Like maybe there are other side features to this too like um it will take your picture and post it to social media.
68. 03:06I don't know.
69. 03:07That could be awful.
70. 03:09Yeah.
71. 03:15Oh, the schedule could be the problem like the fact that Tis was speaking across the hall from me.
72. 03:20That really bugged me.
73. 03:21Um because I like TIS a lot and uh yeah.
74. 03:25Any other like Yeah.
75. 03:30Yeah.
76. 03:42Yeah.
77. 03:42Yep.
78. 03:42That's true.
79. 03:44The um the the core is just getting people into the room and knowing who came in, right?
80. 03:49Yeah.
81. 03:50Um and and why is that important?
82. 03:53Let's let's go one deep one level deeper.
83. 03:56Yeah.
84. 04:01Oh, good question.
85. 04:02So the question is in in this role playinging that we're doing here, who are we?
86. 04:06We are the product engineer.
87. 04:07And so we're trying to we're we're talking with the user who's making the request or maybe the PM is doing this and now we're talking to the PM and asking them these questions.
88. 04:16But yeah, so you have a followup.
89. 04:18What's my motivation?
90. 04:18Am I trying to get
91. 04:22Oh yeah.
92. 04:24Yeah.
93. 04:25So the question is what's the motivation?
94. 04:26Like why do I care that we're we're solving these problems?
95. 04:30the reason that you care is because the app that you're building is what pays your paycheck.
96. 04:34The like if we're talking about capitalistic reasons, um but maybe you have some sort of innate thing inside of you that you just hate the idea of somebody missing the first 20 minutes of a workshop
97. 04:45and so um that is motivating you.
98. 04:57Yeah.
99. 04:58Yeah.
100. 04:58Why is there not a that kind of goes back to this idea too?
101. 05:00Why is there a virtual queue in the first place that actually that's one of the uh things that we as software engineers need to ask ourselves frequently is like
102. 05:10okay I'm going to I'm going to go in here and I'm going to optimize the heck out of this thing.
103. 05:13It's going to be amazing when we really should step back and think why does this exist in the first place?
104. 05:18Oh, it's solving this problem.
105. 05:19Well, why does that problem or oh okay so that that's why this matters.
106. 05:24If we had known at the time that we created this solution um that it would cost this much because this other offshooting problem, then we never would have gone this direction anyway.
107. 05:32We would have gone a different direction.
108. 05:33I called that the problem tree.
109. 05:35You got a problem.
110. 05:36Oh, uh and then you solve that with this and then that has a bunch of problems off of those and each one of those span out and you go until the problems aren't big enough anymore.
111. 05:44Uh, and sometimes you can back up when when you get all the way down here, you're like, "Oh, um, that uh like that's really hard problem to solve.
112. 05:52If I'd known that all the way back here, then I would have gone this way instead."
113. 05:56And we often don't do that.
114. 05:57So, you do need to ask yourself, okay, why does this exist in the first place?
115. 06:01Okay, so we go down why a bunch of times.
116. 06:03This is um how you get down to what ultimately is called the job to be done.
117. 06:08So, jobs to be done theory uh is from Klay Christensen, amazing person um who uh wrote competing against luck among many other books.
118. 06:17How many people have listened or read to at least three of his books?
119. 06:21You have not listened or read to enough of his books.
120. 06:23He has like a bunch and they're all fabulous.
121. 06:26Competing against luck is very good.
122. 06:28Um but the idea of jobs to be done theory or jobs theory is people hire products to help them make progress in a specific situation.
123. 06:36And I have a a video on my YouTube channel.
124. 06:39By the way, I just started getting serious about YouTube and like caring about thumbnails and stuff, like all the stuff you're supposed to do.
125. 06:45So, go subscribe on YouTube.
126. 06:47I'm I'm like weekly u more than weekly videos and they're all really really great.
127. 06:52Uh or if they're not great, go watch it and tell me why and I'll make it better.
128. 06:55Um but uh yeah, so I go deep into uh jobs theory um on that uh deeper or Yeah, it's it's great.
129. 07:03You should go look.
130. 07:04Okay.
131. 07:05So, um first facial recognition that is the request that's not the job.
132. 07:10These are two different things.
133. 07:12Um so, Rita said people will say you guys should build this feature and then you ask them what are you trying to solve?
134. 07:18That's that is your first question back to them.
135. 07:21Uh and if you do say yes to every one of these features, you end up with a huge level of complexity.
136. 07:26So, you do not just turn feature requests into implementation.
137. 07:30I know some of you here in San Francisco are building an app right now that has like takes the user's request and sends it into an agent and just automatically implements it.
138. 07:41Stop.
139. 07:42Okay.
140. 07:43I actually don't know if anybody's actually doing that, but that would be a bad idea.
141. 07:46You would end up with a really terrible product with no vision whatsoever, no taste, all judgment out the window.
142. 07:52It would be awful.
143. 07:53Uh so, uh each feature request requires your product judgment.
144. 07:58So, we don't take the feature request and nail it onto the thing we've already got.
145. 08:02It wouldn't be a good product.
146. 08:03There's lots of judgment that happens here.
147. 08:04And some of that judgment is going to be on the the product manager, of course, like they they're often going to be the one who gets the request first.
148. 08:11Not necessarily.
149. 08:12We'll talk about that, too.
150. 08:14But, um, often that is going to be kind of where things are coming from.
151. 08:17And they should do a good job of um applying the vision of the product to the requests as they come in.
152. 08:25Uh, and so, uh, hopefully they are are kind of slowing things or or at least giving them a better shape for you.
153. 08:32But even still, you you still don't want to just take everything that they give to you because it is going to have an impact on the system that you design.
154. 08:40And now you have to maintain that long term and it's going to slow you down on other things that might be more important.
155. 08:45And so, you have to put everything through the lens of the job to be done to make sure you understand the problem and realize uh that it's actually a valuable problem to be solved.
156. 08:55So, here are the questions that we're uh you ask to reveal what the job is.
157. 09:00And we we're going to come down to a job statement uh here for our facial recognition thing.
158. 09:04So, who is this for?
159. 09:06What who's the facial recognition for?
160. 09:08If we're if we're the conference and we're building an app to get people in, who is the uh the app for?
161. 09:15Attendees.
162. 09:17It's actually partially for the attendees.
163. 09:18It's also for whoever's manning the the door, right?
164. 09:21But um yes, the attendees ultimately are the ones being served.
165. 09:25Um and then when does this problem happen right here when people are going getting into a workshop?
166. 09:34Okay.
167. 09:35And then what progress do they want?
168. 09:37What is the progress that the user is trying to do?
169. 09:41Get in.
170. 09:42That's why we named our our product this just get in to the workshop.
171. 09:46Um okay.
172. 09:47And then finally, what do they do currently?
173. 09:51Well, it's a QR code scan.
174. 09:53Yeah,
175. 09:53blocking request.
176. 09:54A blocking request.
177. 09:56That's right.
178. 09:57We could do parallelization.
179. 09:58That could also solve this problem, right?
180. 10:00And so that hang on to that.
181. 10:02Put a pin in that.
182. 10:03Um run MCP guy.
183. 10:05I like that.
184. 10:05That shirt is cool.
185. 10:06You you know U Michael uh G. Yeah.
186. 10:09Okay.
187. 10:10Um so let's let's try and write the job statement when situation help me make progress so I can get desired outcome.
188. 10:17And often you'll add without some other thing.
189. 10:19So when there's a workshop going on at 1010, help me get into the workshop quickly um so that I can learn about product engineering um without missing the first 20 minutes.
190. 10:32Okay, there you go.
191. 10:34That's the job statement.
192. 10:35That is not that has nothing to do with facial recognition, right?
193. 10:39And so now that we understand what the job statement is, it will inform the decision that we take to uh to go forward with this.
194. 10:48And again, some of you are thinking, well, that's the product manager's job is to like give me the job statement.
195. 10:52Yeah.
196. 10:53Okay.
197. 10:53Like, I'm not going to say that's that's wrong.
198. 10:55But if you get a request that doesn't have a job statement attached or you've never heard you don't know which job statement this request is coming from, then that's going to be
199. 11:04uh you are not understanding the the problem well enough.
200. 11:08Or maybe the PM just skip that part and you're going to spend three months building something that doesn't actually hit users because it's it's not solving a real problem for them.
201. 11:19Or maybe you're solving a problem that seemed like a good idea to one person but wasn't generally applicable to everybody else.
202. 11:26So getting down to the Y why and then turning that into a job statement uh helps a lot with making sure you're building the right thing.
203. 11:32So uh another thing that we do is to break that job once we have that into three dimensions.
204. 11:37functional, social, and emotional.
205. 11:40And uh yeah, we got uh Okay, so functional is like does it actually do the thing?
206. 11:45Social is who is involved when the user is using this product to do the thing?
207. 11:50And then how do they feel when they're using it?
208. 11:52Are they embarrassed or are they like shy or are they excited?
209. 11:57Are they what?
210. 11:59And and these things actually will have will map to technical decisions that you as a product engineer will make.
211. 12:05And so you actually need answers to these questions.
212. 12:07We'll talk about that here in just a sec.
213. 12:09Uh, oh, and I also mapped this to how React one um, by the way.
214. 12:14So, how many of you hate it when I say React one?
215. 12:18I'm just kidding.
216. 12:19Some of you probably don't like React.
217. 12:21I like React and React one.
218. 12:22It's over.
219. 12:23The the game is actually over.
220. 12:25Um, there will be something that comes after React, of course.
221. 12:28Like, it's it's not like it's going to last forever.
222. 12:30But will we need to learn that thing?
223. 12:32No. The answer is no. we will not be learning whatever it is that comes after React.
224. 12:36that it it has changed into like what is the toy that you give your pet agent to be happy like that that's all that really matters at this point as far as that's concerned and so react one
225. 12:49here's here's the reason this is what my video is about react one because it started with functional that was that was the thing at the time everybody was either backbone
226. 12:58maybe Ember uh AngularJS that's what I was using and uh React was just faster uh considerably faster despite all the like rerenders and everything we talk about now, but it was way faster
227. 13:10and uh it gave a much simpler mental model for state management and uh composition was just fabulous and it still is.
228. 13:18Uh React really solves composition.
229. 13:20Being able to create a thing, have it be like a total mess on the inside of that but have a very solid interface so that that mess inside doesn't really affect anything else.
230. 13:29That that level of encapsulation composition was really good.
231. 13:33So then um eventually React like totally dominated 2015 2016 now React is like the top and everybody's using it.
232. 13:42That's when React moved from winning because of functional reasons into social reasons.
233. 13:47Well everybody else is using it.
234. 13:48My boss uh wants me to use it.
235. 13:50My team uh if I want to get a job in this industry I've got to learn React.
236. 13:55And so everybody is learning React.
237. 13:57Now of course there are still others out there and Angular 2 came out and know people are using those.
238. 14:02But React was just so dominant.
239. 14:04The ecosystem was so big.
240. 14:05Yeah, I'm gonna use that.
241. 14:06And how do I feel when I'm using it?
242. 14:07Well, maybe now you feel like, "Oh, I hate this thing or whatever."
243. 14:10But like at the time it was um you know, everybody else is using this.
244. 14:14I feel pretty good like and it's kind of cool because there are all these conferences I can go to and feel really excited with everybody else.
245. 14:20That's why React totally won.
246. 14:22And that's why when React started stumbling a little bit on the functional aspects of things and okay, well now we got SolidJS.
247. 14:29we don't have rendering problems anymore or we got view and it's like easier to learn.
248. 14:34That's arguable.
249. 14:35But like whatever whatever it is, all these functional differences, it didn't matter anymore because it already had just nailed itself with the network externalities.
250. 14:46So anyway, Jobs theory applied to something that you might not think, but open source libraries definitely a product.
251. 14:54Um that's pretty interesting.
252. 14:56Okay.
253. 14:58So, um, one thing that Jack Ryan suggests is, uh, having a Slack channel or some some mechanism for you to just get all of the user feedback.
254. 15:07Some of you work on apps that have hundreds of thousands of users or millions of users.
255. 15:11I worked at PayPal.
256. 15:12I would ship something and instantly it was out to millions of users and that was pretty exciting and cool thing.
257. 15:17But getting feedback from those users is really, really difficult.
258. 15:21Um, and uh, and making sure that you're actually building the right things.
259. 15:24And at the time it was fine that I didn't really spend too much time doing that because the you know it's a big company and we're going to be successful no matter what I do I guess.
260. 15:32Um but uh uh what you can do in those situations or even others is to have a select channel.
261. 15:39Uh you can have a specific one with uh individuals that you really respect their decisions and and um they're like high value customers of yours or you can have one that's just like takes Reddit threads and and X and like all the social media brings it into one place and you just like you don't have to necessarily
262. 15:56answer everything or whatever.
263. 15:57That's not necessarily your job.
264. 15:59But seeing all of this you get a sense for where the rough edges, the paper cuts and all the problems are in your product.
265. 16:06And that can kind of help inform, okay, we're not actually uh solving the pro the job that we were hired to do uh in this product and give you a a sense for okay,
266. 16:16so I understand why they're complaining about that.
267. 16:19I don't believe in user error because Don Norman said it doesn't exist and so how can I change the system so it can support what they think it should be doing?
268. 16:28And sometimes it's going to be, oh well, we just need to put a button right here in instead of just having it right there and and now it will be a lot easier.
269. 16:34Other times it's going to be, oh yeah, there's no way we can do that right now and now now I'm doing my systems thinking of how can we make that
270. 16:42so that those two systems can talk to each other or something.
271. 16:45So um yeah, just let that context wash over you.
272. 16:48Um Michael says there's something very important in thinking about the storytelling of the thing you're building and how it situates in somebody's life.
273. 16:56How does somebody feel when they're using your product?
274. 16:59um is and and not just how do they feel when they're using your product, but uh how does it how do they feel when they know other people know that they're using your product?
275. 17:08So if you're making some really embarrassing things that people need to use like I don't know it depends or something I I don't know bad example but um then uh like you need to take that into account in the way that you uh package your implementation.
276. 17:23Okay.
277. 17:23So let's talk specifically uh about our um our idea.
278. 17:27Did I?
279. 17:28No, I didn't.
280. 17:29Dang it.
281. 17:29So, our uh idea was the facial recognition thing.
282. 17:32That was the original request.
283. 17:33Um and then we brought that into just getting people into uh the room faster.
284. 17:39So, what are the functional um aspects of that?
285. 17:42I need you to throw out these things.
286. 17:44So, like what is the state workflow integration sorts of things that we need to think about if we're building this app?
287. 17:50Uh just get in the workshop.
288. 17:53Start on time.
289. 17:54Start on time.
290. 17:55So, like the speaker always starts on time or Okay, so I messed that up is what you're saying.
291. 18:01[laughter]
292. 18:03Well, in my defense, uh, people weren't gone until, uh, 2 minutes after I should have started.
293. 18:08The solution,
294. 18:09you're right.
295. 18:10Yeah, the solution's not there.
296. 18:11Yeah.
297. 18:12So, actually, that's a really good um metric that we can track as well is like how much are we starting on time that will tell us whether the solution is actually solving the ultimate problem.
298. 18:21Great.
299. 18:22Thank you.
300. 18:22other what are the other functional like let's let's talk more systems things like what is this actual like database table that you would need to um to speed up this let's let's say that we do decide to do um to add
301. 18:37uh to help us solve this job we are going to add NFC no let's you know what let's stick with the face recognition idea um just because I think it'll be fun for something later um but uh yeah so we're going to add face facial recognition
302. 18:49what is some of the state uh that we're going to need to maintain for that.
303. 18:55Yeah.
304. 18:55Database of everybody's faces.
305. 18:56That's going to uh impact a lot of things.
306. 19:00Yeah.
307. 19:01What else?
308. 19:03A what?
309. 19:04Encryption.
310. 19:04Yeah.
311. 19:04Okay.
312. 19:05So, we're encrypting things now.
313. 19:06The government won't get mad.
314. 19:07Um just kidding.
315. 19:09No. Uh other other things we're going to need.
316. 19:11Yeah.
317. 19:12Number seats.
318. 19:13Yeah.
319. 19:13Number Yeah.
320. 19:14So like venue um capacity uh and and attendee uh capacity and and maybe well yeah we won't get into that.
321. 19:22Okay cool.
322. 19:23And then on the social side of things um who whe when this software is being used this particular feature of the software is being used um how does that affect
323. 19:33or the social aspects like who is using it and who are they interacting with when they're using it?
324. 19:38I'm going to have two lines.
325. 19:40You're going to have two lines.
326. 19:41Oh yeah, that I mean now we're talking about parallelization uh which also saw uh is another solution to that same job that we're trying to uh to accomplish.
327. 19:50Yeah.
328. 19:51Um I I would say that the person who's doing the checking in that would be uh one person uh that's involved there.
329. 19:58Um and pretty much just just the person there like other people who are around in line.
330. 20:03Maybe the speaker is involved as well um somehow.
331. 20:06So that can kind of inform some of the implementation as well.
332. 20:10And then our uh emotional side of this.
333. 20:12Hopefully after our solution's implemented, we're all feeling like good about that.
334. 20:16But maybe some people are feeling a little uncomfortable, right?
335. 20:19Like creeped out by the fact that the conference somehow has your facial recognition information.
336. 20:25Uh and that might be a good spot for you as a product engineer to be like, "How about not
337. 20:30[laughter]
338. 20:32let's come up with a different uh solution for this particular job."
339. 20:37Uh okay.
340. 20:39So, um, this kind of goes back to Oh, oh, sorry, let me back up.
341. 20:44Um, in jobs theory, there's actually, um, two times that your product is hired or two categories of hiring.
342. 20:51One is, uh, called the big hire.
343. 20:53That's when you actually sell the product and the person gives you the money.
344. 20:56And two is when they use the product and if they use it multiple times like, uh, I went to Smashburger or not, Super Duper, Super Burger, what, whatever.
345. 21:05It's really close and it's really delicious.
346. 21:06Uh, go mob them later.
347. 21:08They're really good.
348. 21:09Um, but uh and they were really nice to me.
349. 21:11So, seriously, go give them your money.
350. 21:13Um, but uh so the the big hire is when I uh paid my money for the burger and received that burger.
351. 21:19The little hire was every bite that I took out of that burger.
352. 21:22And if I stop early and throw it away, I'm not going to be standing on stage telling you all that it was delicious um because I didn't like it.
353. 21:28So, you actually care a lot about users actually using your software.
354. 21:32It's not just about getting them to pay for it in the first place.
355. 21:35So um you define your success by repeated progress.
356. 21:39Uh so you need to ask questions that will in influence the uh system but uh like how do you measure progress?
357. 21:46Do they do it more than once?
358. 21:48Sometimes they're not going to do this more than once.
359. 21:49Maybe this is the only workshop you come to and then you're going to leave and never go to another workshop ever.
360. 21:53Probably not, right?
361. 21:54So there's there's some potential to use it more than once.
362. 21:57And actually if we were actually building this app, our customer would be the the conference.
363. 22:01It wouldn't be all of you.
364. 22:02Um and so is the conference going to use it more than once?
365. 22:05Uh what should we build, change or instrument, defer or not build?
366. 22:09Uh and what does facial recognition become after we understand the job?
367. 22:14I think once we now that we understand the job, we probably don't need the complexity of facial recognition and maybe like NFC chips inside the badges could work.
368. 22:22Um but like then there's a whole other host of other problems that we now have to associate to that.
369. 22:28Um but by understanding the job to be done, it expands your horizons on what you could possib you're focused on the problem uh and less on the solution until like you've decided okay which direction do we want this problem tree to go.
370. 22:44Uh so why do you need to know this?
371. 22:46Uh it gives you uh language uh for systemshaped questions.
372. 22:50If if a task comes to you and it doesn't have uh some sort of job to be done on uh attached to it, then you go back to your PM and you say, I don't know enough about what this task is all about to design a good system for this.
373. 23:04Um and having the language like well what are the functional impacts and the social and the emotional progress points um that are going to impact um my implementation and architectural
374. 23:15needs.
375. 23:15Um that is going to help make sure that you end up building the right thing.
376. 23:19Um and without the job uh you can just build a requested feature totally missing um building the parts of the system or expanding the primitives that are available uh that users actually need.
377. 23:32So let's get back to some questions here really quick and then I have one more framework that we're going to talk about uh and then we will be done
378. 23:41which means we might actually be in a pretty good place as far as time is go.
379. 23:47Okay.
380. 23:48How do you decide what to build uh without metrics uh in this era?
381. 23:52So this era probably talking about like before you've shipped anything, you don't have any users.
382. 23:56So how do you decide what to build in that uh phase?
383. 23:59That's where the mom test comes in really handy.
384. 24:01Um because you're able to and actually because we have AI, we can prototype things really really quickly.
385. 24:07So you validate the problem exists and that it's an interesting enough problem.
386. 24:12Then you build a simple solution that has like all kinds of paper cuts and whatever, but see if uh people uh resonate with that.
387. 24:20Watch them use your software and just if you can't get anybody to try it out, then um that's a signal.
388. 24:26Um and so you want to find something that's um important enough for somebody to want to try it out.
389. 24:32Um if you are at an existing company and you're evaluating um adding a feature to a product uh same sort of thing except in that case you have customers that um are using your product you need to uh get in contact with those and hopefully your PM
390. 24:46has some mechanism for doing that.
391. 24:48Um and then um through those connections you do that same sort of thing.
392. 24:53Make sure you understand the problem really well.
393. 24:55Build a prototype and see how they feel about that.
394. 24:58Um hopefully that answers that question.
395. 25:01Okay.
396. 25:02So, three tips uh from going from a product role to a product engineering role.
397. 25:07Oh, wow.
398. 25:07Okay.
399. 25:08So, I'm assuming you mean by product role you're like product owner, product manager thing.
400. 25:12Um I would um I would talk to engineers about the system and just interrogate them and like if if I I'm a very open person so I would just literally say I want to get into engineering.
401. 25:26Uh so could you help me understand the system and like what I need to know about engineering at this company um and and just spend a lot of time
402. 25:35um trying to understand what the constraints of the infrastructure are uh how the system is uh designed and um I don't know have lunch with them like hang out with the engineering team
403. 25:47uh stop being so siloed uh okay sweet uh what do you do if your colleagues say there isn't a problem to solve uh but there is oh yeah Okay.
404. 25:57Is the user always right?
405. 25:58No, the user is not always right.
406. 25:59Uh or if they uh you you can have like let's say you your uh company you have 50 users um like that are businesses or something and you've got one who's like really harping on you about this.
407. 26:12Uh if if they're the one that's like covering 90% of your income, then yes, they are right.
408. 26:18Whatever.
409. 26:18Yes, sir.
410. 26:19Um but uh but often that's not the case.
411. 26:22you're going to you you definitely have situations where one user like really cares a lot about this particular thing.
412. 26:27Uh and it's not representative of other users.
413. 26:30So, how you sus that out is just by talking to other people like what was the last time when was the last time you had this problem?
414. 26:36Um and what what did you do about it?
415. 26:38Like mom test it to a bunch of your other users.
416. 26:41Um and it you know if you end up realizing oh yeah like there's not a lot of people who feel this way then yeah maybe the user was wrong in that case.
417. 26:51Um I do not think that the user is always right though.
418. 26:54Uh often they are directionally right like we really really want this.
419. 26:58Like often the user is going to come to you with a solution.
420. 27:01Um they're going to say I have this problem and then here's what would solve it.
421. 27:05Um or they they'll just skip the solution.
422. 27:07They'll just be like I need an export to PDF button right here.
423. 27:11And and that's as far as they'll go.
424. 27:13And um you need to be the one to ask questions and try to get at the heart of what they're trying to do.
425. 27:18Once you get to the heart of what they're trying to do in that job, then you can go around to the other customers and see like does this sound like
426. 27:26uh well, you don't ask is this a problem for you?
427. 27:28You just say when was the last time you experienced this pain and what did you do instead?
428. 27:33Oh, you're like clicking 50 menu items to get to that.
429. 27:35Great.
430. 27:36That is a good signal.
431. 27:37We are going to uh solve that for you.
432. 27:39Hopefully that answers your question.
433. 27:41Okay, we'll do like two or three more.
434. 27:43Um, do I really need to understand the code and architecture for incident response if I can mitigate issues without that knowledge?
435. 27:49Okay.
436. 27:50Um, so I mean it depends on how many millions of dollars you're losing a minute.
437. 27:55Um, I guess so like it kind of depends on the scale of your business.
438. 27:59Uh, if I when I was at PayPal, if I broke um the the the part of PayPal I worked on was uh crossber transactions.
439. 28:07Um, and if I broke that, like people couldn't send money, uh, to people, then yeah, we're losing millions of dollars rapidly.
440. 28:13Um, and so, uh, if we're talking about incident response, I mean, I actually, interestingly, I got attacked by, uh, some, um, bot on a digital ocean box, uh, recently on my my personal website,
441. 28:27and so I recorded a video of what I did to mitigate that problem.
442. 28:31Uh, so that's going up on my YouTube channel soon.
443. 28:33Go subscribe youtube.comkensyods-vids trying to get the just kydods and they are not doing that for me.
444. 28:41But um anyway uh so uh I literally just said went over to my agent and I like it knows what I mean when I say this.
445. 28:49I just said production is down.
446. 28:50Figure it out.
447. 28:51And it and it did.
448. 28:52And it like it pretty much did everything.
449. 28:54It it figured out and told me here's what's going on.
450. 28:56You're getting attacked by a bot.
451. 28:57And I said uh do you know how to fix it?
452. 28:59Fix it.
453. 29:00And it did.
454. 29:00So like there is an element of maybe our agents can kind of just handle things but again like this is my personal website which I I do get a lot of traffic to my website um and so it is important and it's actually a part of my business because it's the only way people can join my discord
455. 29:16server and like there it is important for my website to be up and it's not like a simple you know just hosted on Cloudflare uh pages sort of thing like it there is some complexity going on there
456. 29:26um but uh but it's not super important I'm not losing anything, however long the agent takes to fix it, like I' I'll let the agent do it and I'm going to go off and do something else.
457. 29:36Um, but uh but if it was really important, then yeah, you better understand how that system works and you better be able to very quickly diagnose what the problem is and guide the agent to like even if you're using the agent to debug and ultimately fix the issue,
458. 29:49um you're going to be a lot more uh successful if you really understand the code and architecture.
459. 29:55Uh okay, let's see.
460. 29:58Um, I'll answer this one really quick.
461. 30:00Do you feel systems these days are getting less secure, stable, and reliable due to the use of AI?
462. 30:04Uh, I think it it um I don't think AI has changed this necessarily.
463. 30:11Um, I think that it has made um security problems easier to um to what's the word I'm looking for?
464. 30:20Um,
465. 30:23exploit.
466. 30:24That's the word.
467. 30:24Thank you.
468. 30:25Uh, easier to exploit.
469. 30:27um and also like easier to identify.
470. 30:29Um and it's like the lazy developer who um who didn't really care too much about security before is able to have a bigger blaster radius now.
471. 30:40Um and so like in that way, yes.
472. 30:43Um but we always kind of had these problems.
473. 30:46It's just it is getting easier to exploit them though.
474. 30:49Okay, great.
475. 30:50And mythos.
476. 30:51Yeah, I just got to say that I guess.
477. 30:53Um sweet.
478. 30:56So, we just have one more and then I can um rest.
479. 31:00I will, by the way, I will be around all day the rest of this afternoon.
480. 31:04I'm leaving tomorrow morning.
481. 31:06So, if you do have questions or anything that you want to ask just oneonone later, uh you got to find me.
482. 31:11I'll be I'll just probably stay in the hall over here.
483. 31:13Uh you got to find me today because I I also have stickers that I brought as well.
484. 31:17So, I after this I'm just gonna run out there because they've got other people.
485. 31:21I think Kevin, do you have somebody else after this?
486. 31:25Yes, maybe if if nobody else is after this, then I'll just stay here.
487. 31:29But um if not, then I'll I'll just rush out.
488. 31:31Okay, so prioritizing software changes.
489. 31:37Uh now we've moved on.
490. 31:39We've gotten across the the chasm uh of uh early adopters and everything.
491. 31:44Now we're in the hands of the majority.
492. 31:46We are getting lots of requests.
493. 31:47So what are some requests that we could be getting in just get in the workshop?
494. 31:54Uh, actually we've kind of talked about this already.
495. 31:55You know what?
496. 31:56I'm going to skip that.
497. 31:56And we're actually going to write these down here in a little bit anyway.
498. 31:59So, um, we'll skip that really quick.
499. 32:02Um, not all feature requests, uh, create value in the same way.
500. 32:07So, um, Wayne and I were talking about Bun on the podcast and he said, uh, Bun had all these other ideas which are really exciting.
501. 32:16They've been shipping like nuts in the last year.
502. 32:19Um although it seems I don't know about you you all but I feel like it slowed down considerably after the acquisition.
503. 32:25Am I alone?
504. 32:25Anybody noticed that?
505. 32:27I'm alone.
506. 32:28Nobody who knows what bun is.
507. 32:30All right.
508. 32:30Okay.
509. 32:31Uh all right.
510. 32:32So anyway, they've been shipping like tons of really interesting useful features.
511. 32:36However, they missed some foundational ideas.
512. 32:38The reason that Wayne said this was because I'd mentioned that I'd used bun for some things, but I hadn't migrated some of my old node stuff over to bun because they were missing some pretty key features that I needed uh to be able to do that.
513. 32:50And so that's why he brought this up.
514. 32:52Uh and so there's this mental model for this called the Kano model.
515. 32:55And so that's what we're going to talk about for this last one.
516. 32:58Uh it is a 1984 paper about um attractive quality and must be quality.
517. 33:04Uh where not all features are created the same way.
518. 33:06Uh, and I wrote a I've got another YouTube video about this titled I hate sprint planning.
519. 33:11Uh, and this is how you fix sprint planning.
520. 33:14So, um, let's open up the demo.
521. 33:19So, this is the Kano model right here.
522. 33:21So, in the KO model, you have um these four quadrants that are on these two axes, the implemented and satisfied.
523. 33:31And um you have actually five categories but two of them don't appear on this graph because they should never appear in your codebase in your product at all.
524. 33:39We'll talk about them but these three are the delighters performance needs and the basic needs.
525. 33:44And um the interesting thing about basic needs is that no matter how much you've implemented it you can never get um nobody will be satisfied until it's at fully implemented.
526. 33:55And it is possible to overimplement those and we'll talk about that in a little bit.
527. 33:59performance things.
528. 34:00You hit like a a basic level and people are like, "Yeah, okay."
529. 34:03But if you keep on delivering on that, then people are going to be pretty jazzed about that.
530. 34:08Um and and it will make them happier and happier.
531. 34:11There's probably a limit to this as well.
532. 34:13Uh and then delighters, u people aren't even expecting these.
533. 34:16It's like if you have it even a little bit barely implemented, then yeah, okay, uh I'm film I like that a lot.
534. 34:23And then you can uh continue to add to lighters and eventually u crosses a performance needs.
535. 34:28um the specifics of where all this is.
536. 34:30You'll notice there are no units on this graph.
537. 34:32So like it's a little bit hand wavy on that.
538. 34:35Um but the this is the basic idea.
539. 34:38So let's uh remove the examples and we're going to add a couple requests.
540. 34:43So um uh let's see.
541. 34:46You know what?
542. 34:47Yeah, I don't I don't know that I have time for us to do those requests.
543. 34:50So we're going to use the pre pre-built ones.
544. 34:52And look away.
545. 34:54Don't look.
546. 34:54These are already categorized.
547. 34:55I I knew I I should have done this.
548. 34:57Dang it.
549. 34:58Okay, don't don't look.
550. 34:59Stop.
551. 34:59I see you looking.
552. 35:01Okay, here we are.
553. 35:03So, we've got these requests.
554. 35:06This is for a food delivery app.
555. 35:07So, we're changing shifting gears.
556. 35:09We're delivering food now.
557. 35:11We've expanded beyond just get in.
558. 35:13It's just get in the house to deliver them food.
559. 35:15That's what it is now.
560. 35:18It's getting weird.
561. 35:19Uh, all right.
562. 35:20So, uh, we've got these these items.
563. 35:23Email.
564. 35:23So, order confirmation emails are sometimes not sent.
565. 35:26Okay, real time driver GPS tracking as a feature seems like pretty useful for food delivery.
566. 35:32Estimated delivery time accuracy, like how accurate do we need to be?
567. 35:36Is it within two minutes?
568. 35:38Is it within like 10 minute accuracy?
569. 35:40Uh and then surprise discount on next order, like you finished your order and oh sweet, 25% discount next time.
570. 35:47Awesome.
571. 35:48And then shared cart.
572. 35:49So you're like a team ordering lunch together.
573. 35:51Um, maybe we can add some sort of shared card experience to make it so you don't have to do a 40 text message thread with and somebody has to awkwardly ask for money afterward.
574. 36:01So, um, based on my my mouse keeps on disappearing.
575. 36:04Is that happening for you all?
576. 36:05No, it's not.
577. 36:06I'll look at this screen instead.
578. 36:07Um, okay.
579. 36:09So, based on the Kano model, the basic things the the things that absolutely must be there.
580. 36:15Uh, raise your hand and say which of these things would you say are basic needs?
581. 36:20GPS.
582. 36:21GPS.
583. 36:23Okay.
584. 36:23GPS is a really interesting one.
585. 36:25U we're going to put it with basic, but I want to talk about that one.
586. 36:27What's next?
587. 36:30What?
588. 36:30ETA.
589. 36:31Okay.
590. 36:32ETA is an interesting one.
591. 36:33I'm not going to fill that one in uh just yet.
592. 36:35Um
593. 36:37I want to get the other There's one other basic that I'm looking for.
594. 36:40Email.
595. 36:41Yeah.
596. 36:42If if I'm not getting my email confirmation, then I'm like, who are you?
597. 36:46Like, what what have you done in in the last year?
598. 36:49So if we uh pop up just email right here anywhere down here I am not satisfied.
599. 36:56I'm getting like only some of the emails or something.
600. 36:58No, that's not going to do it for me.
601. 37:00Every other app, not just food delivery, but every other app has reliable email.
602. 37:06I'm expecting that.
603. 37:06That absolutely is required.
604. 37:13Like a notification.
605. 37:14Yeah.
606. 37:14Yeah.
607. 37:15Fair point.
608. 37:16um
609. 37:17notification and we'll go tada.
610. 37:22A notification with confetti.
611. 37:25Yeah.
612. 37:26Okay, that's very good point.
613. 37:28All right, so um I do want to talk about GPS, but I want I'm going to bring that one in last.
614. 37:33So, let's talk about performance.
615. 37:35Now, somebody mentioned ETA was a good basic.
616. 37:38I would actually put it as a performance.
617. 37:40And the reason is that as long as you're within like a reasonable range, like if you're within uh two minutes of people arriving on time, then like okay.
618. 37:50Yeah.
619. 37:50Like I realize that there are stop lights and whatnot.
620. 37:53Like that's that does happen.
621. 37:55But if you somehow manage to like get me Well, let's first start with u before.
622. 38:01If you say it's going to be 10 minutes and it's actually like 30, I'm pretty ticked.
623. 38:05If you say, even if you overd deliver and you underpromise, overd deliver and you say, "Well, it's going to be 30 minutes."
624. 38:11And then they're there in 10.
625. 38:13I'm like, "Bro, I'm not even home yet."
626. 38:15Like, gosh.
627. 38:16So, it's not necessarily overpromise, underdel or what?
628. 38:20The opposite of what I just said.
629. 38:22Uh, you need to hit some level of like satisfaction there or implementation.
630. 38:27But, you know, if it if I'm like looking at my phone and it's counting down with every step that he takes and then he drops it on my doorstep,
631. 38:35right when it hits zero, like, wow, that's pretty cool.
632. 38:39I don't know that I'm necessarily going to like be looking at my phone all the time, but I will be impressed.
633. 38:43Uh, so there is like a level where depending on what you're looking for, this is going to add a a certain amount of satisfaction for people and surprise and and
634. 38:53um they'll be excited about that.
635. 38:54Another good example of this, how many of you all are web developers?
636. 38:56You serve your app over HTTP.
637. 38:59Wow, a lot less than I expect.
638. 39:00Well, what did I expect?
639. 39:02Well, in in that world, I know some of you are native developers and you're like, "Yeah, people down my 30 gigabyte app and it takes them 20 minutes and nobody complains about that."
640. 39:12Well, us in the web world, if it takes less more than a second, then people leave.
641. 39:18Uh, so there you are.
642. 39:20So in the web world, it depends on what it is that you're building.
643. 39:24But you, one performance metric, literally performance need is how fast your website loads.
644. 39:30And in some situations being like uh 1 second.
645. 39:33All right, that's good.
646. 39:34That's going to put me at satisfactory.
647. 39:36I'm I'm happy.
648. 39:37But you can um push that further.
649. 39:39Okay, 250 milliseconds and 100 milliseconds now.
650. 39:42Like whoa, I'm really impressed by that.
651. 39:44So that's that's what we're talking about.
652. 39:46We're talking performance needs.
653. 39:47as long as for ETA as long as you're reasonably good then that's fine but you can push beyond that.
654. 39:54Uh whereas with a basic need if you go beyond the implemented then it gets to be a little bit much like all right so I received three notifications he's
655. 40:04uh he's just picked it up and now he's driving to you and he just passed the stop sign and then like it can be too much.
656. 40:10So that's kind of the difference between basic and performance is um whether or not it uh really matters if you get more and more of it.
657. 40:18Okay.
658. 40:18So now let's talk about uh discount and group order.
659. 40:22Where where should those be categorized?
660. 40:26Yeah, both of these are delighters.
661. 40:28Uh I would say uh the group order I think we're starting to get used to being able to like have more collaborative software.
662. 40:34So I would actually suggest that one's starting to move into the performance realm.
663. 40:38Um but uh yeah.
664. 40:40So, for delighters, these ones, like if you even have a little bit, um, I'm gonna be like, "Oh, yeah, sweet.
665. 40:46I'll take a 5% discount.
666. 40:47Oh, man.
667. 40:48I'll take a 90% discount.
668. 40:50Heck yeah."
669. 40:50Like, I'm I'm loving that.
670. 40:52Uh, or actually once you pass this, it's like, "Is there something wrong with this company?"
671. 40:57Like, so that like you could maybe go a little bit too far with that, too.
672. 41:01Um, and yeah, group ordering.
673. 41:03Oh, that's nice that they have that feature.
674. 41:04Oh, wow.
675. 41:05Like, it's really easy to use.
676. 41:06See, that's where we're going with that.
677. 41:08Okay.
678. 41:09Okay, I think we all pretty much get the delighters.
679. 41:10The uh the delighters are the interesting one because that's the one that's most exciting to work on, right?
680. 41:16Who wants to work on notifications?
681. 41:18Like zero people in this room want to work on that.
682. 41:21But like if you don't have that, then your users will also go down to zero.
683. 41:26Like they will not be happy with that.
684. 41:27So you have to have those delighters or those basic needs in place.
685. 41:30You cannot uh overdelight uh your basics.
686. 41:34Like if you don't have the basics, then it doesn't matter what your delighters are.
687. 41:37There's a little bit of an exception with the early adopters because they are willing to like experience all the paper cuts to get to that solution that you're you're doing for them.
688. 41:48Um but uh yeah at once you start getting to the level where you're trying to scale it out to majority uh you absolutely have to have those uh basics in or you will not be successful.
689. 41:58Now let's talk about GPS.
690. 41:59I agree that this is a basic need, but um back in 2015, this would have been a delighter.
691. 42:05Like I you mean I can watch my driver where they are right now and so yeah, you have an ETA and it's like right around here, but I can judge for myself and I know they're coming up to that uh train station and like there there's no way they're getting through that.
692. 42:20They stopped at train station.
693. 42:21So now like this is what was really really awesome in 2015 but uh it eventually moved down uh into performance and now like I'm expecting it's got to be there like it just has to exist and if you can make it more and more accurate like okay I'm feeling pretty good about that.
694. 42:36Yeah, that's pretty good.
695. 42:37Um but now I would agree I think it's a basic need if you don't have it at all.
696. 42:42Uh, and and at least at I don't know.
697. 42:44I I do think that maybe it's kind of sitting between performance and and basic need because like you can be more accurate, but honestly, am I going to really care that much that it's like that much more accurate than the next person?
698. 42:54As long as it's within a reasonable accuracy, which we could call fully implemented, then I'm fine with that.
699. 43:00So uh the the point there is that your um the the different items that you uh support from your application uh or your software solution, those pieces can move on the KO model, but they only move one direction.
700. 43:16It goes from delighter and it so if you implement a delighter, this is like a really nice feature.
701. 43:21Nobody else has it.
702. 43:22Your competitors don't have it yet.
703. 43:23Um it can either go from delighter down to performance or it can just pop off because nobody cared about it.
704. 43:29Uh, and so you're like, "Nope, we're we're going to remove that feature.
705. 43:31Nobody was using it anyway."
706. 43:33Um, but once it goes down to performance, then it's going to go down to basic need.
707. 43:37Uh, and now you have to do it.
708. 43:39Uh, so this does happen uh over time, which means you can't just be like, okay, we prioritized all of our tasks and we're going to just leave it there.
709. 43:47No, you have to regularly be looking at, okay, what have our competitors doing?
710. 43:51Oh, they added this delighter.
711. 43:53But we probably better start thinking about like looking into that and and seeing if we can solve the problem in our in our own take on that problem.
712. 44:01So that is uh the basics of the Kano model.
713. 44:04Here's a couple of things uh about that.
714. 44:06So uh Wayne was the one who introduced the Kano model to me and he gave one of my favorite analogies for this.
715. 44:12If there's no toilet paper in my hotel, that's a bad experience.
716. 44:15But if I have a hundred rolls of toilet paper, it doesn't increase my experience with the hotel.
717. 44:20Uh and actually he added I should have added it to this the thing but the next thing he said was and it might make me worry about the food.
718. 44:27So so um there is like if we were to take the the KO model and uh follow this line I would say continuing this the green line the basic needs would probably go down the more it's fully implemented
719. 44:41uh yeah it starts to become a deterrent for you.
720. 44:44Um, so but that that said, uh, slide.
721. 44:48There we go.
722. 44:49Uh, you have to have um the foundations.
723. 44:52So like don't bother wasting a bunch of time on the de delightful stuff before getting foundations right.
724. 44:58F focus on foundations and primitives.
725. 45:01Make sure you have good feedback systems so that you know that you're covering your bases as far as basics are concerned.
726. 45:07Uh, okay.
727. 45:09So uh, did I Yeah, there we go.
728. 45:11So why product engineers need the Kano model?
729. 45:13U product managers are responsible for identifying the value.
730. 45:16Um you also have some responsibility there as well but like in the perfect world product manager is going to find where that value is going to come from and they're going to deliver that to you and your job is to make sure you understand that
731. 45:29uh and then uh convert that into the technical requirements as the product engineer.
732. 45:34So you decide the reliability, reversibility, like how much does it matter?
733. 45:37What's the uh performance constraints?
734. 45:39Uh and so in in each one of these categories, it helps you make the right uh decision.
735. 45:44Not just which one should I do first, but how much technical um uh expertise or or uh effort do I put into it.
736. 45:52So um for basics, we're going to focus on reliability, completeness, ownership.
737. 45:57Like these things we absolutely have to nail and we have to do it right.
738. 46:01We don't want notifications not showing up for performance.
739. 46:05These are measurable quality gradients.
740. 46:07What that is is like how much does ETA actually matter?
741. 46:12Where is the level of satisfied do I need to get to?
742. 46:15Um because like if you have those performance metrics under the line, then that's going to be a problem as well.
743. 46:22So you need your basics up, you need your performance up at least to that line.
744. 46:25And once you have all of that, now you can start playing around with uh adding things to that.
745. 46:30So, you're going to need to measure things to make sure that you're hitting um those uh goals on the performance metrics to make sure that you're at least at the level you should be.
746. 46:39Uh and then delight uh delighters.
747. 46:41Um these are like kind of experiments.
748. 46:44Okay, so maybe you you do think that this is going to be kind of core to your identity and everything, but still a delighter is not a basic and you need to make sure that you're covering your bases on the basics.
749. 46:54So you're going to be thinking about okay with this experiment how much do I want to invest in making sure that this is solid reliable every I mean of course we want everything to be reliable right but there's only so much time there's only so many tokens
750. 47:06uh and so this is where you're going to um maybe have a little less commitment from an architectural standpoint maybe you kind of instead of integrating it into the existing system you have this other uh system where there you can do a lot of learning and then once you've learned and kind of gotten the shape of this
751. 47:21new thing you can find a way to bring it back into the full system.
752. 47:25All right.
753. 47:26Now there like I said there are other two other parts of the Kano model.
754. 47:29One is uh the indifference category.
755. 47:32So these are the things where the user like you decide that it doesn't really make a difference whether you have this or not.
756. 47:38I don't care.
757. 47:39I don't need that feature.
758. 47:40Uh so you don't want to make this like a a permanent part of your product and in fact you want it out of your product as quickly as possible.
759. 47:48It's just dead weight that you have to maintain as an engineer.
760. 47:51It's not useful.
761. 47:52And this is why it's important for you to understand the Kano model because you need to go to your your boss and say or your product manager say listen we spent two weeks maintaining this feature.
762. 48:02Is anybody using that?
763. 48:04Like does anybody even care about this thing?
764. 48:06Because if they don't maybe I should not spend two weeks working on figuring out how to get that migrated over to the new system.
765. 48:13So uh in different category you definitely want to know about.
766. 48:16And then uh reverse.
767. 48:17This is the sort of thing that you just really don't want in your product at all.
768. 48:21So, we were talking about the the facial recognition thing where like, oh yeah, maybe when you sign in, it could like automatically post to their social media saying, I'm going to this event.
769. 48:32Like, they would probably really hate that.
770. 48:34I I would hate that a lot.
771. 48:36Um, and so these are the sorts of thing that you as an engineer like, uh, let me ask you this.
772. 48:42There are products that do shady things that we don't like as users.
773. 48:47That that's a given.
774. 48:48I think we can all agree on that.
775. 48:50Who built that?
776. 48:52We did.
777. 48:53Software engineers are building that.
778. 48:55And and I'm not saying that the buck stops here necessarily, but it kind of does.
779. 48:59Like we're the ones who are building it.
780. 49:01So like, could you please push back on those stupid ideas that are like making our lives awful?
781. 49:08like software can be just this wonderful thing that can make the fulfill our lives so much better.
782. 49:15And it's software engineers just like us that that uh do the opposite as well.
783. 49:20And so um like take a stand and be like, "No, I'm not going to do that."
784. 49:26And yes, I realize that maybe you're Yeah, thank you.
785. 49:30Uh I realize like, okay, yeah, Ken, that's that's nice as a self-employed person.
786. 49:34Um, but uh I don't I don't know.
787. 49:38I feel like um have a little respect for yourself.
788. 49:41There you go.
789. 49:42So, uh I'm gonna answer a couple more questions.
790. 49:45We have four minutes, I think, until I'm I'm booted out of here.
791. 49:48And before I do this, uh I'm just going to look.
792. 49:52Ah, shoot.
793. 49:53You know what?
794. 49:53I don't have time for questions.
795. 49:54Here's what we're going to do.
796. 49:56Uh I I don't know if there's somebody in this room after me.
797. 49:59Is there anybody?
798. 50:00Is there
799. 50:00There is.
800. 50:01There is.
801. 50:02Thank you.
802. 50:03Sorry.
803. 50:03What?
804. 50:03It's PayPal.
805. 50:04It's PayPal.
806. 50:05I loved PayPal.
807. 50:06Loved working at PayPal.
808. 50:07So, we're not going to be a problem for PayPal.
809. 50:09I'm gonna run.
810. 50:10I'm not going to talk to any of you and I'm just going to run over to the hall over here and then if anybody who wants to talk can talk.
811. 50:16So, let me just wrap up.
812. 50:17I don't want you leaving yet.
813. 50:18You haven't clapped yet.
814. 50:19Gosh.
815. 50:20[laughter]
816. 50:21So, just really quick.
817. 50:23Um, the mom test helped us with idea early idea valuation or validation.
818. 50:29It's not just for startups.
819. 50:31It's also for any new product idea that you have in your company.
820. 50:35Um, framing user requests with jobs theory will help you make sure that you're not just building the solutions as the they come to you from the user, but that you can manage actually to solve many user requests with a single solution.
821. 50:47And the Kano model will help you prioritize all of the uh changes that are coming down the pipe and make sure that you're not shipping anything that's in the reverse category at least.
822. 50:57And with that, um, I want you to remember my original thesis.
823. 51:01When AI agents level the implementation playing field, the differentiator becomes building the right thing.
824. 51:07Um, and this is why I say this is the last skill you all need to learn.
825. 51:12uh once the AI agents figure out how to do this then who know like this AGI we I don't know what we do in this industry but this what's really cool about this is even if I am wrong
826. 51:22and agents stop right here they don't get any better than they are right now and we're still having to babysit them and all the stuff that we're doing now if I'm wrong
827. 51:30if you do this you are one of the most valuable members of your team at your company because this is where the actual value comes from when you can take your technical expertise and marry it to the actual problems your users are having, then you're the most valuable person at your company.
828. 51:46And that is it.
829. 51:47Thank you all so very much.
830. 51:49[applause]
831. 52:06[music]