The Pipeline Is Dead - Iris ten Teije, Sky Valley Ambient Computing

AI Engineer · 19 min · 206 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

1. 00:00As we're online today, you can't raise your hands, but just nod at your screen.
2. 00:04How many of you have spent a significant part of your career making making software move from one computer to another?
3. 00:12CI pipelines, packagers, registries, container images, app store reviews.
4. 00:18That entire stack exists to solve exactly one problem.
5. 00:23Get a frozen artifact, that code, from machine where it was built to the machine where it runs, safely, reproducibly, once.
6. 00:34And here's what I think everyone is missing.
7. 00:36That entire stack is built around one idea that is so old that is so old that we stopped seeing it as a choice.
8. 00:42One version of your software for everyone.
9. 00:46We've shipped it that way for so long that almost nobody asks why anymore.
10. 00:51I'm Iris.
11. 00:52I'm one of the co-founders of Differ.
12. 00:54My co-founder Noam was the first engineer at JFrog.
13. 00:58He helped build the pipeline that I'm about to tell you is dying.
14. 01:01I came at it from the other end.
15. 01:04I spent a decade in fintech.
16. 01:06I was early at a digital bank that we scaled and exited.
17. 01:09Shipping software in the environments least willing to tolerate what I'm about to propose.
18. 01:15So, I totally get any reservations and I've had them myself as well.
19. 01:20However, I'm telling you it's coming anyway.
20. 01:23And that's not a warning.
21. 01:26It's the best thing that's happened to software in a long time.
22. 01:30Every piece of distribution infrastructure that you've touched encodes the same assumption.
23. 01:36Software is produced in one place.
24. 01:38It runs in another.
25. 01:40Anything in the middle, the artifact, is frozen.
26. 01:45And that assumption was correct.
27. 01:47For decades, it was just true.
28. 01:50Why?
29. 01:52Because producing a correct change was expensive.
30. 01:55It took skilled humans hours or days.
31. 01:58So, you did it rarely.
32. 01:59It was a central event.
33. 02:00You verified it and froze it.
34. 02:03And you shipped that frozen thing to everyone.
35. 02:06So, the one-way pipeline is not arbitrary.
36. 02:09It's the direct consequence of production of software being expensive and risky.
37. 02:15And of course, the frozen artifact has some advantages.
38. 02:19You get reproducibility, previewability, rollback.
39. 02:25And every guarantee that we lean on in production flows from one fact.
40. 02:29There is one artifact and it doesn't change after we ship it.
41. 02:34That's the deal.
42. 02:35One version for everyone, frozen.
43. 02:39We call it reliability.
44. 02:41But the price was that the software couldn't really be for anyone in particular.
45. 02:47And nobody ever made that decision.
46. 02:49There was never a meeting where someone put one version for everyone versus a version for each person and picked the first one.
47. 02:59And it wasn't because the second option if a version for each person was worse.
48. 03:04It just wasn't an option.
49. 03:06Giving to users different software meant forking the code base and hand maintaining both.
50. 03:12A version per user at any real scale wasn't really a viable option.
51. 03:18And one version for everyone never had to win an argument.
52. 03:21It was just how it was.
53. 03:23It was the only shape software could take.
54. 03:25And we started treating it as a fact about software like gravity, like something that that is just true.
55. 03:32But it was never really a fact about software.
56. 03:34It was a fact about cost and budget and the economics and that cost just changed.
57. 03:40Now, you might you might expect me to go into AI 10 code, but everyone has already said it and that's stable stakes and to me that's not really the interesting part.
58. 03:49The interesting part is where and how cheap The cost of producing a correct and scope change is collapsing towards zero and just as importantly the production of the software
59. 04:04no longer has to happen in one place up front before anyone runs it.
60. 04:10Part of it can be run on the server, part of it on the client, part in the user's live session.
61. 04:15And as each step stops being a decision you freeze at build time and and now starts becoming more of a real-time one that also means that each each each piece can be placed wherever it makes most sense
62. 04:30including right in front of the user in their context.
63. 04:35And so this whole one-way pipeline existed because making software was the expensive, central, and rare event and running it was cheap.
64. 04:44So you separated development from distribution.
65. 04:48But now as making a change becomes as cheap as running one and it can happen in the same place as um as where you run it, the reason to separate them
66. 04:59is dissolving.
67. 05:00So far I've talked about the supply side.
68. 05:04Producing code has become cheap easy.
69. 05:07We can now make a change on a per user basis.
70. 05:10But I also want to address the demand side.
71. 05:14People have always wanted software that fits them and we have decades of proof for that.
72. 05:21It just wasn't really possible for most types of software because of cost.
73. 05:26To start with one example, the forward deployed engineer.
74. 05:30Enterprise software has always had a line item called professional services and a whole industry exists around that.
75. 05:38If you are a big client of a company like Salesforce, you probably have consultants that are helping you implement custom setups, configurations.
76. 05:48You have an engineer living in your Slack channel, and it's not that smaller customers can't benefit from this type of customization.
77. 06:00It just didn't make sense financially until today.
78. 06:04To give another example that might resonate with you as engineers, think about your dot files, your editor config, key bindings.
79. 06:15You rebuild every tool you touch into your tool by hand on every machine.
80. 06:21It's another example of a demand for personalized software.
81. 06:26And lastly, Excel, the most successful business software ever created.
82. 06:31And Excel isn't really a static program.
83. 06:38It's millions of people that all build their own programs on top of it.
84. 06:44So, that's where.
85. 06:45Give people the power to make software to make their software theirs, and they take it.
86. 06:49Seen in the social feed that per user wins on wins from one size fits all on every metric that mattered.
87. 06:58This is more of an example of content versus software.
88. 07:01But now that we have uh better coding again now we have coding agents and better coding agents, we can move this also to the software layer.
89. 07:09So, my point is none of this is really new demand.
90. 07:12We've seen it for decades.
91. 07:13There have been predecessors like feature flags, segmentation, AB testing.
92. 07:19There's an enormous industry around uh uh around this, and we've been trying to make software diverge for many years.
93. 07:29But we go forced into a specific shape, that of creating buckets and segments that you declare in advance.
94. 07:39And now, for the first time, we can make software truly adaptive.
95. 07:46So, to get back to the title of this talk, when the agent is the runtime, when the thing that runs your software can also modify it, development and distribution stop being two phases.
96. 07:58The boundary blurs and it's gone.
97. 08:02And the shape that we bet on at Differ is that instead of one code base gated by flags and shipped to everyone, you deploy one canonical stem and every user runs her own divergence of it.
98. 08:16Same origin, but individually adapted live.
99. 08:22It's going from the least first version for everyone to the best version for anyone.
100. 08:31Now, if you're any of such a person, you might be a little worried.
101. 08:34Your stomach might be turning because I have just deleted the frozen artifact and the frozen artifact was what was holding up the entire the the entire building.
102. 08:45And I do we do get these objections, for example, from a call that I had with a CTO recently who is right to be skeptical.
103. 08:54What he said roughly was, "I can already barely reason about one AI-generated code base.
104. 09:00And you want me to run millions of these?
105. 09:02You're not describing a capability.
106. 09:04You're describing my worst problem multiplied."
107. 09:07And if that's your reaction, that's not surprising.
108. 09:10It's the right instinct, but perhaps aimed at the wrong target.
109. 09:13Here's the distinction that we make.
110. 09:15The brittleness that you're picturing is a specific type of failure mode.
111. 09:20It's unmanaged divergence inside a single artifact.
112. 09:23Thousand-line files, everything can touch everything else, no boundaries.
113. 09:28And that's not not that's not necessarily brittle because it's AI-generated.
114. 09:32It's brittle because there's no structure separating things.
115. 09:38In our In our vision, we've we're thinking about per-user divergences.
116. 09:43And don't write that is the opposite of that.
117. 09:45You've got a stem plus divergences.
118. 09:47The divergences are bounded, isolated, and individually reversible.
119. 09:52A bad variant can silently corrupt the stem or reach another user.
120. 09:56Which means that the blast radius of a changes in the system is one context.
121. 10:01And any single divergence can roll back life with no deploy.
122. 10:06So, the answer to the previous objection isn't trust us, AI is good at coordination or coding.
123. 10:13The honest answer is you're brittle because there's a tangled artifact with no boundaries.
124. 10:22In our case, we don't ship you a thousand single artifacts.
125. 10:24We ship one stem and bounded divergences.
126. 10:27Each isolated, each reversible.
127. 10:30The thing that you're afraid of is the thing that this architecture exists to prevent.
128. 10:37And as a developer, you can also set controls and boundaries.
129. 10:42What can and cannot be adapted.
130. 10:44To give a small example, um we can have a scenario where we've got a form and the form can be adapted in order to improve conversion rate, for example.
131. 10:56However, as a developer, you can always indicate that specific fields can never be dropped or parts of your app, like off or payments, should always be out of um
132. 11:09uh uh should always be off limits for any sort of adaptation.
133. 11:14So, give another example to make adaptive software a bit more concrete.
134. 11:20Uh think of, for example, um a CRM.
135. 11:24In this case, uh we've got an investor who's using a CRM.
136. 11:28Whereas, the CRM was mostly built with a salesperson in mind.
137. 11:31And as an investor, you might use it slightly differently.
138. 11:35So, this investor, she often logs founder intros, and she's always logging who intro'd her to which deal.
139. 11:45So, the system observes that and creates a um create creates a intro path.
140. 11:53The system can also observe that she's always skipping specific fields.
141. 11:57She never fills them out.
142. 11:59So, over time, the system learns and doesn't surface those fields, but instead it surfaces fields that she that she cares about more.
143. 12:10Another example can be the fact that she is always um always checking specific types of deals or founders, and it doesn't exactly follow the prioritization that the system sets
144. 12:25by default.
145. 12:26So, again, can can we learn and make that smarter so that the information that the user cares about is surfaced first.
146. 12:34Not only can a system observe, but there is also that the user can proactively request changes, and as long as these changes are within the boundaries that the developer sets, and as long as they are within
147. 12:47the spirit of the software, within the purpose of what the software was originally made for, it can be implemented without having to go back to the developer.
148. 12:58So, you can imagine for a for a horizontal SaaS like a CRM, you can address a much wider number of customer personas without increasing your R&D spend.
149. 13:13Now, of course, there is there are many hard parts when it comes to bringing this to life.
150. 13:21And I'm going to discuss a couple of hard challenges and problem and problems that we're uh that we're working on as we're making adaptive software real at scale.
151. 13:31We haven't solved all of them, but we have a we have a point of view on each of them, and it's that it's exactly what we're working on day in, day out.
152. 13:40So, what changes when there's no single artifact?
153. 13:44Firstly, the source of truth.
154. 13:48So, what when there's no single artifact, you can wonder what is the software?
155. 13:54In our case, we consider the software to be the stem plus the plus all the mutual immutable divergences.
156. 14:02But of course, that creates a lineage problem.
157. 14:06What is this user running and why?
158. 14:09That now becomes more of a graph query versus a version number.
159. 14:14A bug report describes a program that exists for that specific user.
160. 14:19How do you debug or inspect that?
161. 14:22The answer is every divergence is immutable, inspectable, attributable.
162. 14:28And we also need to trace any version back to the signal that to this specific recommendation and this exact adaptation.
163. 14:40That's one of the parts that we're working on.
164. 14:44Second is correctness.
165. 14:46Like how do you test that's the um how do you test that the code change that you've just implemented works, that UI is correct, and yeah, testing at a much larger scale
166. 15:01of users means that you need to reason about the stem and also every possible divergence of it.
167. 15:09And then there's desirability because perhaps you made a code change, it's perfectly correct and working, but you also need to know that whether whether this was desirable.
168. 15:21Was it actually a good change?
169. 15:24Because anyone can make a code change now, but the hard part is knowing whether you actually found an improvement, an uplift.
170. 15:36And yeah, that that is something that is extremely important to to keep track of and to measure and to consider what the goals for the company are.
171. 15:45And this is not going to be the same for every single piece of software.
172. 15:48In some cases, it might be retention or less churn or lowering the number of support tickets.
173. 15:57So, it's extremely important to keep track of what are what are the goals that we're chasing with adaptive software and do the adaptations reach to improve improve improve the metrics that matter.
174. 16:11Next is autonomy versus control.
175. 16:13And the conservative answer here would be start with just recommendations and don't make any autonomous changes.
176. 16:23And that's not a wrong strategy, but in our case, it's not really our vision.
177. 16:29The vision is a system that understands the user well enough to act without asking the developer for permission every time first.
178. 16:38So, for us, the challenge isn't building more control.
179. 16:41It's winning enough trust that you don't have to.
180. 16:45And it's a hard problem, but also also very interesting one.
181. 16:49And you know, how do you make a system good enough, legible enough, reliable enough that humans in the loop choose to step back.
182. 17:00And that's certainly what we are building towards.
183. 17:06And lastly, coordination.
184. 17:08Everyone on their own version, how do you push new updates?
185. 17:12How do how do changes propagate to like a million different versions?
186. 17:17And this is one of the challenges that we've been thinking hardest about.
187. 17:22Yeah, and the answer that we keep coming back to is don't merge code, merge intent, merge outcome.
188. 17:30Which means not everyone has to run the same commit or this exact same piece of code, but everyone converges on the same goal through their own path.
189. 17:41And if it wasn't already clear, the challenges that I just addressed are the hard challenges.
190. 17:46Generation has become easy, and I would say that's actually the easy 80%.
191. 17:51Calling a model to write some code is something that everyone can do.
192. 17:56The other part, observability, validation, coordination, that is the entire business.
193. 18:06Anyone can call an LM, but the substrate is stem plus diver plus divergences, provenance, validation, that is really the hard part, and something that we're working on every single day.
194. 18:25To close off with, when I started out in fintech, a bank with no branches sounded reckless.
195. 18:33A decade later, the the branch is the weird part.
196. 18:38My co-founder, Noam, used to fight with engineers about why you need builds and CI.
197. 18:47We've seen this We've seen these shifts before, and adaptive software is the same type of shift.
198. 18:54It doesn't feel obvious until it does.
199. 18:58To go back to the start of the talk, the pipeline didn't fail because it didn't work anymore, but the constraint it was built for went away.
200. 19:09The assumption underlying of that software is expensive, so we need to freeze it and ship it once.
201. 19:15That assumption stopped being true.
202. 19:19And when making software get as cheap as running it, the line between distribution and development isn't a line anymore.
203. 19:30We spent 20 years getting good at shipping one version for everyone.
204. 19:34The next 20 years are about shipping the right version to anyone with the isolation and provenance that makes it safe instead of terrifying.
205. 19:43I'm Iris, this is different and that's what we're building.
206. 19:46Thank you for watching.