WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=TJPInBjhE4Q NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/TJPInBjhE4Q.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:00.480 --> 00:00:01.400 Hi everyone. s2 00:00:01.400 --> 00:00:03.960 I'm Sachin Gupta and I'm a software engineer. s3 00:00:03.960 --> 00:00:06.640 The title is exactly what it sounds like. s4 00:00:06.640 --> 00:00:09.480 Your coding agent is creating review debt. s5 00:00:09.480 --> 00:00:12.680 Before I start, sit with the title for a second. s6 00:00:12.680 --> 00:00:14.480 Notice what it is not saying. s7 00:00:14.480 --> 00:00:17.680 I'm not saying that coding agents are bad. s8 00:00:17.680 --> 00:00:20.280 I'm not saying they don't make us faster. s9 00:00:20.280 --> 00:00:25.400 I'm saying they they're creating a kind of debt that nobody is measuring. s10 00:00:25.400 --> 00:00:28.320 And that debt is going to come to you. s11 00:00:28.320 --> 00:00:31.280 Over the next few minutes, here's what I'm going to do. s12 00:00:31.280 --> 00:00:33.400 I will define what review debt is. s13 00:00:33.400 --> 00:00:37.040 I'll walk five signal families that are going to compose it. s14 00:00:37.040 --> 00:00:45.640 I'll score three real pull requests side by side and I will show you a cross repo scan of over 500 PRs, which is nothing but three public code bases. s15 00:00:45.640 --> 00:00:47.920 So let's go. s16 00:00:49.440 --> 00:00:50.600 Look here. s17 00:00:50.600 --> 00:00:52.840 Here's the gap nobody's measuring. s18 00:00:52.840 --> 00:01:03.120 GitHub's 2025 October's report that covers almost every public pull request on the planet shows that the commits climbed 25% year over year. s19 00:01:03.120 --> 00:01:07.720 And now over the same year, comments on the comments dropped 27%. s20 00:01:07.720 --> 00:01:11.680 Now, these comments are nothing but the proxy for review activity. s21 00:01:11.680 --> 00:01:16.320 Code production volume actually went up, but the review attention went down. s22 00:01:16.320 --> 00:01:19.800 They moved in opposite directions that to in the same year. s23 00:01:19.800 --> 00:01:23.920 Now, look at the teams for this along the AI adoption curve. s24 00:01:23.920 --> 00:01:28.280 Faros AI tracks this cohort in their 2026 benchmark. s25 00:01:28.280 --> 00:01:41.680 Median PR review time is up by 441.5% and if you see like if you calculate, you'll figure out that the reviewed PRs take 5.4 times longer than what they used to. s26 00:01:41.680 --> 00:01:46.480 And then 31% more PR are now merged with no review at all. s27 00:01:46.480 --> 00:01:49.840 So AI produce AI is producing the code very fast. s28 00:01:49.840 --> 00:01:56.680 AI is producing the pull request very fast, but humans cannot responsibly review them at that pace. s29 00:01:56.680 --> 00:01:59.240 This gap is called as review debt. s30 00:01:59.240 --> 00:02:01.040 It actually accrues quite a bit. s31 00:02:01.040 --> 00:02:05.520 It compounds and right now nobody has a number for it. s32 00:02:05.520 --> 00:02:11.440 But by the end of this talk, I will make sure that you will have a certain number. s33 00:02:12.240 --> 00:02:17.959 Now, if you look at this particular slide, here is the story every team is telling right now. s34 00:02:17.959 --> 00:02:20.680 PR per developer are up 16%. s35 00:02:20.680 --> 00:02:27.200 That's the Faros AI acceleration Whiplash benchmark and that from April 2026. s36 00:02:27.200 --> 00:02:30.920 There were like 22,000 developers, 4,000 teams. s37 00:02:30.920 --> 00:02:34.400 And the median PR size is up 63%. s38 00:02:34.400 --> 00:02:38.200 44 lines to 72 lines per pull request. s39 00:02:38.200 --> 00:02:46.760 That is basically 16 months of long-term data from DX 2026 study, which includes 400 organizations. s40 00:02:46.760 --> 00:02:54.120 Now, if you see the cycle time, open to merge, that is modestly down and it is coming from the same DX study. s41 00:02:54.120 --> 00:02:55.720 The framing is really generous. s42 00:02:55.720 --> 00:03:02.200 PR throughput grew actually about 8% and the AI usage rose about 65%. s43 00:03:02.200 --> 00:03:08.239 The gain that we see today is real, but it is actually smaller than the hype that that is present. s44 00:03:08.239 --> 00:03:10.400 Every one of these numbers are real. s45 00:03:10.400 --> 00:03:14.160 None of them is a lie, but everyone is a vanity metric. s46 00:03:14.160 --> 00:03:18.560 PR count goes up when one PR splits into seven PRs. s47 00:03:18.560 --> 00:03:22.600 Median PR size going up is not a benefit, it's actually a bloating. s48 00:03:22.600 --> 00:03:25.959 Cycle time going down when reviewers stop pushing back. s49 00:03:25.959 --> 00:03:28.440 These things tell you the speed of production. s50 00:03:28.440 --> 00:03:32.000 They do not tell you the speed of trust. s51 00:03:32.000 --> 00:03:36.480 Now, we will go into the next slide to see more. s52 00:03:42.600 --> 00:03:47.480 Now, if you go looking for the second story, here is what you're going to find. s53 00:03:47.480 --> 00:03:48.800 Reviewer fatigue. s54 00:03:48.800 --> 00:03:52.320 Engineers carry far more review load than a year ago. s55 00:03:52.320 --> 00:03:54.280 They're not happier about it. s56 00:03:54.280 --> 00:03:55.800 Late-night merges. s57 00:03:55.800 --> 00:04:03.880 PRs are sitting unreviewed for 3 days, 4 days, and then they suddenly get a thumbs up at 11:00 p.m. before a Friday deadline. s58 00:04:03.880 --> 00:04:05.160 Test theater. s59 00:04:05.160 --> 00:04:11.080 Tests are getting added, but they're actually asserting what the code did, not what the code should do. s60 00:04:11.080 --> 00:04:15.040 They lack in behavior, including bugs. s61 00:04:15.040 --> 00:04:16.480 Architectural drift. s62 00:04:16.480 --> 00:04:21.519 Again, the same problem that is solved in three different ways in three different files. s63 00:04:21.519 --> 00:04:25.800 Nobody is actually holding up the architectural trade. s64 00:04:25.800 --> 00:04:27.720 Finally, incident lag. s65 00:04:27.720 --> 00:04:33.720 When something breaks, the bug lags week or months after month uh after merge. s66 00:04:33.720 --> 00:04:37.120 Nobody connects the dot back to the AI authored change. s67 00:04:37.120 --> 00:04:42.640 These costs don't show up in any dashboard until now. s68 00:04:42.640 --> 00:04:46.360 We'll look into the next slide to know more about it. s69 00:04:46.360 --> 00:04:48.680 So now, let me give you the definition. s70 00:04:48.680 --> 00:04:50.520 Review debt. s71 00:04:50.520 --> 00:04:59.919 It's the accumulating gap between code your agent has produced and the code humans have actually reviewed, trusted, and understood. s72 00:04:59.919 --> 00:05:05.240 It rhymes with technical debt, but it is more like a financial debt because it compounds. s73 00:05:05.240 --> 00:05:08.400 It actually accrues interest, but this interest is not money. s74 00:05:08.400 --> 00:05:10.680 This interest is paid in human attention. s75 00:05:10.680 --> 00:05:13.160 It compounds because of the three feedback loop. s76 00:05:13.160 --> 00:05:19.800 First, the agent learns from your code base, fine-tuning, rag grounding, in-context suggestions. s77 00:05:19.800 --> 00:05:24.600 Code that was not deeply reviewed yesterday grounds tomorrow's PR. s78 00:05:24.600 --> 00:05:27.520 This debt becomes generative in nature. s79 00:05:27.520 --> 00:05:31.240 Second one, reviewers seed the architectural call. s80 00:05:31.240 --> 00:05:36.840 When most of a PR was generated, attention contracts to syntax and obvious bugs. s81 00:05:36.840 --> 00:05:40.240 Big picture decision moved from review time to never. s82 00:05:40.240 --> 00:05:43.560 Third one, velocity expectations reset. s83 00:05:43.560 --> 00:05:48.560 Once leadership sees the new throughput, you don't get to hire reviewers in proportion. s84 00:05:48.560 --> 00:05:51.320 There is no slack left to pay the debt back. s85 00:05:51.320 --> 00:05:54.000 Each loop on its own is survivable. s86 00:05:54.000 --> 00:06:00.440 Together, you will have a runway, but the next question is, how are you going to measure it? s87 00:06:00.680 --> 00:06:03.040 So, how do we measure review debt? s88 00:06:03.040 --> 00:06:07.160 There are five signal families that has like 10 deterministic checks. s89 00:06:07.160 --> 00:06:09.840 Remember, the keyword here is deterministic. s90 00:06:09.840 --> 00:06:14.320 Every check is computable from a pull request and its repository. s91 00:06:14.320 --> 00:06:17.360 No, we don't need any language model. s92 00:06:17.360 --> 00:06:18.040 Why? s93 00:06:18.040 --> 00:06:23.320 Because LLM, when it acts as a judge, it possibly can break two things. s94 00:06:23.320 --> 00:06:26.680 Number one, the score becomes a moving target. s95 00:06:26.680 --> 00:06:34.560 The same PR, basically the score that you got for the same PR, will score differently when your model will change. s96 00:06:34.560 --> 00:06:38.919 Two, the score stop being defensible in an engineering review. s97 00:06:38.919 --> 00:06:40.560 You cannot put it on a slide. s98 00:06:40.560 --> 00:06:43.040 You cannot put it in front of a leadership. s99 00:06:43.040 --> 00:06:44.240 So, what do you want? s100 00:06:44.240 --> 00:06:51.440 You want a number that is traceable to a deterministic computation, different size and coupling. s101 00:06:51.440 --> 00:06:53.800 Second, test evidence gap. s102 00:06:53.800 --> 00:06:56.960 Third, directory and ownership spread. s103 00:06:56.960 --> 00:06:59.800 Fourth, AI authorship indicators. s104 00:06:59.800 --> 00:07:02.600 Fifth, evidence and rational gaps. s105 00:07:02.600 --> 00:07:08.040 These are the five families that will have 10 checks under them. s106 00:07:08.040 --> 00:07:11.520 Let's walk each of them. s107 00:07:11.800 --> 00:07:16.240 The first one, the first signal that we have is dev size and coupling. s108 00:07:16.240 --> 00:07:18.880 This is the simplest one to calculate, basically. s109 00:07:18.880 --> 00:07:21.240 It's also the most often misread. s110 00:07:21.240 --> 00:07:29.919 What it measures is the net lines changed, the files touched, whether changes cluster in one module, or is it sprawling across many. s111 00:07:29.919 --> 00:07:31.560 Why the agent is struggling here? s112 00:07:31.560 --> 00:07:34.720 Agents are biased toward fix at the call site. s113 00:07:34.720 --> 00:07:37.400 A human engineer routes a fix to the root cause. s114 00:07:37.400 --> 00:07:39.720 That keeps the difference very small. s115 00:07:39.720 --> 00:07:43.800 But an agent will reach into many files looking for the same symptom. s116 00:07:43.800 --> 00:07:48.240 The review cost of a sprawling difference is not proportional to the size. s117 00:07:48.240 --> 00:07:50.600 It is actually much steeper. s118 00:07:50.600 --> 00:07:56.040 So, the cross-file coupling explodes the mental model and reviewer has to hold. s119 00:07:56.040 --> 00:08:00.920 We'll look into the second signal, which is test evidence gap. s120 00:08:03.840 --> 00:08:05.560 So, what is test evidence gap? s121 00:08:05.560 --> 00:08:11.840 Test evidence gap is nothing but test lines added divided by the production lines added per pull request. s122 00:08:11.840 --> 00:08:16.600 So, AI authored PRs, they ship with a far lower test to code ratio. s123 00:08:16.600 --> 00:08:19.040 Sometimes there is no test file in place. s124 00:08:19.040 --> 00:08:22.480 And if there is any, it's minimal. s125 00:08:22.480 --> 00:08:24.880 So, this gap is actually a consistent gap. s126 00:08:24.880 --> 00:08:31.840 And the problem with this particular thing is the number The reason this number is basically brutal is the agent generate test. s127 00:08:31.840 --> 00:08:33.360 They generate a lot of test. s128 00:08:33.360 --> 00:08:36.800 But these test basically assert what the code is doing. s129 00:08:36.800 --> 00:08:42.159 It base It does not generate the test cases what the code should actually do. s130 00:08:42.159 --> 00:08:44.600 They lock in the behavior including the bugs. s131 00:08:44.600 --> 00:08:47.720 The ratio does not capture the quality gap. s132 00:08:47.720 --> 00:08:51.680 It just measures whether the test showed up at all. s133 00:08:51.680 --> 00:08:56.480 So, the deeper signal is the next step that is a layer down. s134 00:08:56.480 --> 00:09:02.120 We'll see the next one, which is directory and ownership spread. s135 00:09:03.880 --> 00:09:06.400 So, what is directory and ownership spread? s136 00:09:06.400 --> 00:09:10.440 Count the distinct code owner teams whose file appear in the difference. s137 00:09:10.440 --> 00:09:12.280 Now, what does it mean? s138 00:09:12.280 --> 00:09:16.640 Basically, a well-shaped PR concentrates in one team directory. s139 00:09:16.640 --> 00:09:19.960 But a sprawling one reaches across many team file. s140 00:09:19.960 --> 00:09:22.640 And the review cost is enormous in this scenario. s141 00:09:22.640 --> 00:09:26.400 No single human hold the whole mental order model. s142 00:09:26.400 --> 00:09:31.200 You need multiple approvals from multiple engineers in multiple different context. s143 00:09:31.200 --> 00:09:37.040 The coordination overhead easily exceeds the time the agent save producing the code. s144 00:09:37.040 --> 00:09:40.360 This is where you start to feel the economics of review debt. s145 00:09:40.360 --> 00:09:43.520 The agent gave you three hours of typing. s146 00:09:43.520 --> 00:09:48.160 You spend those three hours by multi-party reviewer attention. s147 00:09:48.160 --> 00:09:53.240 And the next one we are going to see is AI authorship indicators. s148 00:09:53.920 --> 00:09:56.040 So, what is AI authorship indicator? s149 00:09:56.040 --> 00:09:58.960 Before everyone gets defensive, this is not for the blame. s150 00:09:58.960 --> 00:10:02.080 We are not flagging that engineers use coding agents. s151 00:10:02.080 --> 00:10:07.400 We wait the scores for a PR shape like agent assisted authorship get extra reviewer attention. s152 00:10:07.400 --> 00:10:11.800 So, that way you know, right like, okay, this is the PR that is directly coming from the agent. s153 00:10:11.800 --> 00:10:14.440 And there are like the very common three detection mode. s154 00:10:14.440 --> 00:10:18.000 I must I I think everyone must have seen it by now. s155 00:10:18.000 --> 00:10:21.960 One is the co-authored footer, which is basically one of your strongest signal. s156 00:10:21.960 --> 00:10:24.360 You see that co-authored by Copilot. s157 00:10:24.360 --> 00:10:26.000 Second one is the branch name pattern. s158 00:10:26.000 --> 00:10:30.400 Generally, you must have seen it says codex, Copilot, cursor prefixes. s159 00:10:30.400 --> 00:10:35.120 Third one is basic generated by assisted by, which is nothing but your PR body. s160 00:10:35.120 --> 00:10:36.920 your commit message phrases. s161 00:10:36.920 --> 00:10:42.440 We did actually a real data check on three public repos, which comprised of 524 PRs. s162 00:10:42.440 --> 00:10:52.440 I'm not going to name the company, but for the sake uh we are just calling it A, B, and C. Now, if you see the highest signal was the co-authored footer. s163 00:10:52.440 --> 00:11:01.720 And the second one uh the lowest signal was 0% on the repo C. Even though it was done by coding agent, but they might have blocked it uh saying any co-authored s164 00:11:01.720 --> 00:11:11.400 or uh like generated by etc. etc. The next thing that we are going to see is evidence and rational gaps. s165 00:11:12.240 --> 00:11:14.080 So, evidence and rational gaps. s166 00:11:14.080 --> 00:11:19.120 I think this is one of the most deterministic that I personally feel. s167 00:11:19.120 --> 00:11:23.400 It's the one that destroys review ability the fastest. s168 00:11:23.400 --> 00:11:24.800 What it measures basically? s169 00:11:24.800 --> 00:11:28.720 Does the PR explain the why or just the what? s170 00:11:28.720 --> 00:11:30.839 On the left, if you see, we have an high gap. s171 00:11:30.839 --> 00:11:32.800 On the right, we have a low gap. s172 00:11:32.800 --> 00:11:40.400 The title says "Fix leaky test" and all the data that you're seeing here is actually from some public repos. s173 00:11:40.400 --> 00:11:44.840 Now, the PR body length is 18 characters, the commit message says updates. s174 00:11:44.840 --> 00:11:48.240 Obviously, I will not be if you give it to me, I won't be able to review this. s175 00:11:48.240 --> 00:11:50.160 Okay, I cannot accept this. s176 00:11:50.160 --> 00:11:56.880 But then on the right, if you see when there is actually a low gap, the title actually is telling you what the change is all about. s177 00:11:56.880 --> 00:12:01.840 The body has a symptom, the diagnosis, the change, and the link to the benchmark. s178 00:12:01.840 --> 00:12:04.680 Now, a reviewer can do the job. s179 00:12:04.680 --> 00:12:11.360 So, in our regression fixtures and hands code teams, this signal destroys reviewability fastest. s180 00:12:11.360 --> 00:12:16.120 In the real open source data, PR bodies follow conventional commit format. s181 00:12:16.120 --> 00:12:18.080 So, it fires most rarely there. s182 00:12:18.080 --> 00:12:21.720 But if you see the regression fixtures are just the demos. s183 00:12:21.720 --> 00:12:25.160 Okay, let's see how those five signals combine. s184 00:12:25.160 --> 00:12:28.600 You will only have one number that is going from zero to 100. s185 00:12:28.600 --> 00:12:31.560 And the exact weights are basically your defaults. s186 00:12:31.560 --> 00:12:35.360 Now, if you're planning to adopt this, I would like you to do this first. s187 00:12:35.360 --> 00:12:39.920 Run it backwards over the last 200 PRs that have been merged in your company. s188 00:12:39.920 --> 00:12:43.600 Calibrate the weights against your team's actual reviewer experience. s189 00:12:43.600 --> 00:12:46.640 The score has to feel right against your actual guts. s190 00:12:46.640 --> 00:12:49.880 We you you are going to divide it into four bands. s191 00:12:49.880 --> 00:12:54.120 If you are zero to 24, which means you have very low review burden. s192 00:12:54.120 --> 00:12:59.120 If you have 25 to 49, you are normal, proceed with the standard care. s193 00:12:59.120 --> 00:13:03.560 52 74, you need evidence from the author before senior review. s194 00:13:03.560 --> 00:13:08.800 And if you're 75 and above, definitely it's high split or request more context. s195 00:13:08.800 --> 00:13:13.280 Same shape as technical debt categories, but the unit is different. s196 00:13:13.280 --> 00:13:18.520 Now, we are going to see what the scanner actually did. s197 00:13:23.920 --> 00:13:27.440 Now, if you see this is the clean PR. s198 00:13:27.440 --> 00:13:32.760 Unfortunately, I won't be able to give you a live demo today, but I will walk you through a CLI. s199 00:13:32.760 --> 00:13:41.280 I wanted you to show three scored pull requests side by side, the level where the framework become useful as a repeatable review conversation. s200 00:13:41.280 --> 00:13:43.080 This is the first one. s201 00:13:43.080 --> 00:13:46.720 If you see, this is what the scanner says when a PR is all changed. s202 00:13:46.720 --> 00:13:48.720 Score out zero out of 100. s203 00:13:48.720 --> 00:13:51.160 The burden is basically nil. s204 00:13:51.160 --> 00:13:51.960 It's zero. s205 00:13:51.960 --> 00:13:53.880 We have a low review burden. s206 00:13:53.880 --> 00:13:55.800 The estimated minutes is six. s207 00:13:55.800 --> 00:13:57.800 No checks were fired. s208 00:13:57.800 --> 00:14:02.360 And if you look the structure here, that is why the list is empty. s209 00:14:02.360 --> 00:14:03.960 Reviewer focus is empty. s210 00:14:03.960 --> 00:14:05.600 Author next action is empty. s211 00:14:05.600 --> 00:14:10.160 The framework only generate advice when it has something specific to say. s212 00:14:10.160 --> 00:14:14.440 A healthy PR produces a tiny, almost ceremonial report. s213 00:14:14.440 --> 00:14:16.080 That's exactly what you want. s214 00:14:16.080 --> 00:14:20.680 And the takeaway is most healthy PRs produce zero noise. s215 00:14:20.680 --> 00:14:23.920 They get the work done without making any noise. s216 00:14:23.920 --> 00:14:27.040 This is not a tool that complains by default. s217 00:14:27.040 --> 00:14:31.160 Running it on every PR cost you one comment that says look good. s218 00:14:31.160 --> 00:14:33.320 That's it. s219 00:14:33.320 --> 00:14:36.720 Let's see the high debt PR now. s220 00:14:37.680 --> 00:14:43.480 Now, you see what you see actually on the screen is from a public repo. s221 00:14:43.480 --> 00:14:48.480 This is a real scenario from the scanner regression suite. s222 00:14:48.480 --> 00:14:51.720 If you see, the score is here 60 out of 100. s223 00:14:51.720 --> 00:14:54.080 It requires evidence. s224 00:14:54.080 --> 00:14:56.520 It's on the audit and it says needs evidence. s225 00:14:56.520 --> 00:15:00.480 86 estimated minutes of review effort. s226 00:15:00.480 --> 00:15:01.839 Look at the structured output. s227 00:15:01.839 --> 00:15:03.440 It's actually not a score. s228 00:15:03.440 --> 00:15:05.800 A wire list explaining what fired. s229 00:15:05.800 --> 00:15:09.760 A reviewer focus list telling the reviewer what to do next. s230 00:15:09.760 --> 00:15:14.520 An author next action list telling the author how to bring the score down. s231 00:15:14.520 --> 00:15:17.520 This is the part most PR quality tool miss. s232 00:15:17.520 --> 00:15:19.400 The score alone is useful. s233 00:15:19.400 --> 00:15:23.680 The structured advice is what actually moves the team behavior. s234 00:15:23.680 --> 00:15:29.480 Now, if you read the bullet point, it says PR has soft indicators of AI assisted authorship. s235 00:15:29.480 --> 00:15:36.920 This is information only, not a definitive claim, and not a penalty on its own. s236 00:15:37.760 --> 00:15:42.720 This particular step sentence is in the scanner output. s237 00:15:42.720 --> 00:15:49.160 The AI indicator check contributes five of the 60 points, which is nearly about 8%. s238 00:15:49.160 --> 00:15:54.400 The other 55 came from a different size claim mismatch and missing test. s239 00:15:54.400 --> 00:16:00.040 Those would be high burden signals on any PR, agent authored or not authored. s240 00:16:00.040 --> 00:16:02.160 This is not an anti-AI scorecard. s241 00:16:02.160 --> 00:16:05.600 Basically, this is more of a review burden scorecard. s242 00:16:05.600 --> 00:16:12.840 The agent did not cause the score, but the shape of the PR that is created by the agent did this. s243 00:16:12.840 --> 00:16:15.520 Let's see the next one. s244 00:16:20.360 --> 00:16:24.080 This is a PR, basically, which is AI authored. s245 00:16:24.080 --> 00:16:29.960 It's very well shaped, and the score is seven, seven, which is low review burden. s246 00:16:29.960 --> 00:16:34.760 Now, if you see the same agent, it has done the same kind of work, but the test were added. s247 00:16:34.760 --> 00:16:36.080 So, the CI is green. s248 00:16:36.080 --> 00:16:39.160 The risky path is called out in the PR description. s249 00:16:39.160 --> 00:16:43.000 That's why we got the score as seven out of 100. s250 00:16:43.000 --> 00:16:46.400 It's very low burden, took only 14 minutes. s251 00:16:46.400 --> 00:16:52.120 The AI indicator check still fires, but it contributes 2.1. One, low severity. s252 00:16:52.120 --> 00:17:02.839 Second, the framework explicitly says, quoting the report itself, "Information only, not a definitive claim, and not a penalty on its own." s253 00:17:02.839 --> 00:17:08.360 So, when your team is going to ask, "Will this tool penalize us for using coding agents?" s254 00:17:08.360 --> 00:17:12.319 No, the answer is on your screen, low review burden. s255 00:17:12.319 --> 00:17:15.280 It's a very well shaped AI PR. s256 00:17:15.280 --> 00:17:22.839 We are going to see the next one, which is basically our three public repos over 524 PRs. s257 00:17:22.839 --> 00:17:29.800 I wanted to show you a 90-day AI search slope across three public repos and 524 pull request. s258 00:17:29.800 --> 00:17:34.000 The review burden climbs even when the AI authorship doesn't. s259 00:17:34.000 --> 00:17:37.360 If you see this particular scan, we observe three things. s260 00:17:37.360 --> 00:17:40.520 One of them is the volume is the actual variable. s261 00:17:40.520 --> 00:17:46.800 AI authorship was flat and it was 5 to 20% steady state, but the review burden was not. s262 00:17:46.800 --> 00:17:54.240 If you see one repo accumulated 186 senior reviewer hours in 27 days and another one took 43. s263 00:17:54.240 --> 00:17:57.280 The window length is same, but the volume was not the same. s264 00:17:57.280 --> 00:17:59.160 The burden was not the same. s265 00:17:59.160 --> 00:18:01.160 That is what PR volume is. s266 00:18:01.160 --> 00:18:11.000 Second is amplifier only holds up under real data, which means AI indicators fired on 5 to 20% of PRs every week across all three repos. s267 00:18:11.000 --> 00:18:16.000 None of those AI authored PRs disproportionately landed in the high burden band. s268 00:18:16.000 --> 00:18:19.440 The positioning contract holds against real code bases. s269 00:18:19.440 --> 00:18:35.280 Third one is complexity drives burden, not authorship, which means across 524 PRs four landed in needs evidence or high bands, which means all four were structural changes, which were large migration, SDK rewrites, multi-team refactors. s270 00:18:35.280 --> 00:18:37.880 The framework score complexity fairly. s271 00:18:37.880 --> 00:18:42.920 AI driven volume creates the condition under which these accumulate. s272 00:18:42.920 --> 00:18:46.880 So now, what 524 PR look like under the lens? s273 00:18:46.880 --> 00:18:49.840 Where does the review debt actually show up? s274 00:18:49.840 --> 00:19:04.720 In absolute terms, the scanner saw 524 real pull request in three public repos, 228 senior reviewer hours cumulative across three public repos that were scanned over 27 to 90-day windows. s275 00:19:04.720 --> 00:19:10.000 Second one is nine PR per day sustained merge rate the high velocity repo. s276 00:19:10.000 --> 00:19:16.040 So, if you see, it's all about the volume that is basically driving the burden. s277 00:19:16.080 --> 00:19:18.200 Third one is 5,036. s278 00:19:18.200 --> 00:19:31.480 These are the minutes that are basically spent on a single PR, which is nothing but 84 hours of review effort estimated for one single pull request. s279 00:19:31.480 --> 00:19:38.560 The score that we received there was 73, which said it requires an evidence band. s280 00:19:38.560 --> 00:19:45.560 Now, 5 to 25% of these PR every week across every week fired at AI indicator signal. s281 00:19:45.560 --> 00:19:54.400 Again, remember this is volume, which is causing the review debt, which is changing the game altogether. s282 00:19:55.520 --> 00:19:57.360 We'll move to the next slide. s283 00:19:57.360 --> 00:19:58.640 Measure the problem. s284 00:19:58.640 --> 00:20:00.200 We have shown the cost. s285 00:20:00.200 --> 00:20:02.840 Now, what do we do about it? s286 00:20:02.840 --> 00:20:06.480 So, first thing is you should have one logical change per PR. s287 00:20:06.480 --> 00:20:09.240 I'm not saying like not small PR in the abstract. s288 00:20:09.240 --> 00:20:11.840 One logical change, that is enough. s289 00:20:11.840 --> 00:20:14.360 Second, test ship with the change. s290 00:20:14.360 --> 00:20:26.560 Even if the agent wrote the code, even if the agent wrote the test, the human author confirms the test assert what the code should do, not what the code is doing, what the code is supposed to do. s291 00:20:26.560 --> 00:20:27.640 Okay? s292 00:20:27.640 --> 00:20:29.800 Third, stay in one owner territory. s293 00:20:29.800 --> 00:20:34.080 Cross-cutting work splitting into per per team PR. s294 00:20:34.080 --> 00:20:35.480 Remain in your own territory. s295 00:20:35.480 --> 00:20:38.560 One approval, one context, one mental model. s296 00:20:38.560 --> 00:20:40.120 Author writes the why. s297 00:20:40.120 --> 00:20:42.560 The agent should not write the PR body. s298 00:20:42.560 --> 00:20:47.560 That's the moment the human author commits to understanding what they are actually shipping. s299 00:20:47.560 --> 00:20:51.120 Same review standard for the AI PRs as human PRs. s300 00:20:51.120 --> 00:20:57.040 The AI indicator amplifier only fires when other signals are weak. s301 00:20:57.040 --> 00:20:58.600 Keep the other four strong. s302 00:20:58.600 --> 00:21:00.600 The amplifier is invisible. s303 00:21:00.600 --> 00:21:04.480 So, make sure there is no exception for AI. s304 00:21:04.480 --> 00:21:08.080 None of these is actually requiring you to have a new tool. s305 00:21:08.080 --> 00:21:10.560 These are the moves that you already are aware of. s306 00:21:10.560 --> 00:21:13.920 You just need to implement it. s307 00:21:15.800 --> 00:21:20.240 Now, how are you planning to adopt this? s308 00:21:20.240 --> 00:21:22.720 If you see, we again have five steps. s309 00:21:22.720 --> 00:21:27.520 Backfill, threshold, surface, aggregate, and talk about it. s310 00:21:27.520 --> 00:21:28.480 Backfill. s311 00:21:28.480 --> 00:21:32.000 So, you run the scorer over your last 200 merge PR. s312 00:21:32.000 --> 00:21:36.000 Look at the one basically that has the highest score. s313 00:21:36.000 --> 00:21:37.320 Second, threshold. s314 00:21:37.320 --> 00:21:39.560 Set a basically justify line. s315 00:21:39.560 --> 00:21:42.000 Default, let's say you want to give it 50. s316 00:21:42.000 --> 00:21:45.560 So, any PR scored at 50 or above requires a comment from the author. s317 00:21:45.560 --> 00:21:46.960 As simple as that. s318 00:21:46.960 --> 00:21:48.200 Surface it. s319 00:21:48.200 --> 00:21:50.920 Post the score as a PR comment on every PR. s320 00:21:50.920 --> 00:21:52.240 Don't block it. s321 00:21:52.240 --> 00:21:57.400 Just for the visibility purpose, surface it so that everybody should know what's happening over there. s322 00:21:57.400 --> 00:21:59.280 Aggregate weekly per team. s323 00:21:59.280 --> 00:22:03.480 Which means the slope of each team deadline is the leading indicator. s324 00:22:03.480 --> 00:22:06.520 That's what your engineering manager should watch. s325 00:22:06.520 --> 00:22:07.560 Talk about it. s326 00:22:07.560 --> 00:22:12.040 Bring the number to a retrospective, every roadmap view. s327 00:22:12.040 --> 00:22:15.480 In every meeting, you should discuss about that number. s328 00:22:15.480 --> 00:22:19.520 Like, what was your score when you did this particular PR. s329 00:22:19.520 --> 00:22:21.800 That's it. s330 00:22:26.000 --> 00:22:27.760 Why this number matter? s331 00:22:27.760 --> 00:22:29.520 Because of this conversation? s332 00:22:29.520 --> 00:22:31.280 Absolutely not. s333 00:22:31.280 --> 00:22:34.440 Without a number, what you are just conversing is why. s334 00:22:34.440 --> 00:22:37.680 But if you have a number, it's actually structured. s335 00:22:37.680 --> 00:22:44.920 You can tell like, okay, with AI coding agents, we have a rollout which has added X percentage of the throughput. s336 00:22:44.920 --> 00:22:51.200 But you need to make sure you are also adding the Y points of the review debt that has been occurred over Z weeks. s337 00:22:51.200 --> 00:22:58.400 Now, at your current slope, that's roughly N senior N senior engineer hours that you need to talk about. s338 00:22:58.400 --> 00:23:01.280 That moves the discussion from feeling to measurement. s339 00:23:01.280 --> 00:23:04.280 So, you should always have a number. s340 00:23:04.280 --> 00:23:07.520 If you see 2026, we are adopting things. s341 00:23:07.520 --> 00:23:12.760 But, 2027 is the year when this conversation will shift to the governance model. s342 00:23:12.760 --> 00:23:15.080 And, it's actually starting right now. s343 00:23:15.080 --> 00:23:17.120 Can you trust the code that you're shipping? s344 00:23:17.120 --> 00:23:19.240 If yes, we are good. s345 00:23:19.240 --> 00:23:24.120 If no, then who is accountable when an AI authored change causes an incident? s346 00:23:24.120 --> 00:23:25.480 Where is the audit trail? s347 00:23:25.480 --> 00:23:29.520 So, reviewed it is basically the bridge between these two columns. s348 00:23:29.520 --> 00:23:38.560 It's the first number that lets you have the right-hand side conversation without abandoning the left-hand side gains. s349 00:23:38.640 --> 00:23:41.920 We'll look into the final slide now. s350 00:23:44.520 --> 00:23:48.480 Now, three things I would like you to take away. s351 00:23:48.480 --> 00:23:50.720 First one is, measure the gap. s352 00:23:50.720 --> 00:23:52.240 Don't build the tooling yet. s353 00:23:52.240 --> 00:23:58.080 Take 20 PRs from the last week, score them using the five signals that we discussed. s354 00:23:58.080 --> 00:24:03.920 You will basically get a number so that you will understand, okay, whether the framework is working right or not. s355 00:24:03.920 --> 00:24:05.600 Try to make the slope visible. s356 00:24:05.600 --> 00:24:09.160 The slope of a reviewed it over time matters most than the level. s357 00:24:09.160 --> 00:24:16.720 So, plot it weekly, and then discuss it with your team maybe in a meeting or in a sprint planning, whatever the way you feel like. s358 00:24:16.720 --> 00:24:17.880 Have this conversation. s359 00:24:17.880 --> 00:24:20.160 Bring the number to your next review. s360 00:24:20.160 --> 00:24:25.240 Like, say that, okay, we do have extra report, wide it, and this is the slope. s361 00:24:25.240 --> 00:24:30.720 And, then you are basically moving away from the wide culture to the measurement culture. s362 00:24:30.720 --> 00:24:33.640 One quick callout on anti-patterns to avoid. s363 00:24:33.640 --> 00:24:36.680 First thing is, approve with comment merges. s364 00:24:36.680 --> 00:24:38.160 The AI did the boring part. s365 00:24:38.160 --> 00:24:40.120 No. We'll catch it in QA. s366 00:24:40.120 --> 00:24:41.760 No. PRs smaller now. s367 00:24:41.760 --> 00:24:46.320 No. Stop writing LGTM on the PRs. s368 00:24:46.320 --> 00:24:53.160 Make sure whatever we have discussed, the habits that we have discussed, we are going to follow them. s369 00:24:53.160 --> 00:24:55.880 This is the entire point of the talk. s370 00:24:55.880 --> 00:24:59.080 Thank you very much for listening.