WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=s-aixZYJG4c NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/s-aixZYJG4c.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:12.480 --> 00:00:15.600 Are you all set up with your AI factory? s2 00:00:15.600 --> 00:00:17.720 Everything is smooth. s3 00:00:17.720 --> 00:00:22.400 As you're sitting here, your code is being deployed, errors growing, right? s4 00:00:22.400 --> 00:00:24.560 If it's like that, raise your hand. s5 00:00:24.560 --> 00:00:25.040 No? s6 00:00:25.040 --> 00:00:25.880 Oh, okay. s7 00:00:25.880 --> 00:00:27.480 Two people, great. s8 00:00:27.480 --> 00:00:41.040 Um Let us know like before and after this talk what they are and and do you feel like there's a bottleneck, a new bottleneck that is not on writing code, rather somewhere else else in the STLC? s9 00:00:41.040 --> 00:00:47.360 Is that your biggest thing to tackle if if you are tackling that right now? s10 00:00:47.360 --> 00:00:48.800 Okay, so you're in the right place. s11 00:00:48.800 --> 00:01:03.880 And what about like code review, verifying that the code work according to your intent, according to your architecture, standards, best practices, etc. Is this something that you're tackling day-to-day or week-by-week? s12 00:01:03.880 --> 00:01:05.840 Okay, so you're you're in the right place. s13 00:01:05.840 --> 00:01:08.920 So, I'm Adam Ouriel Friedman, the CEO and co-founder of Codo. s14 00:01:08.920 --> 00:01:14.440 Um I don't share it too much, but Codo stands for quality of development optimization. s15 00:01:14.440 --> 00:01:31.560 Our mission and and uh is to help you all have a code governance code review platform that understand your code base, your tribal knowledge, your best practices, and that does not come off the shelf s16 00:01:31.560 --> 00:01:32.800 from a model. s17 00:01:32.800 --> 00:01:40.520 It requires a system that optimize for you as we go, and that's uh why we decided to name our company uh this way. s18 00:01:40.520 --> 00:01:47.880 Uh so, uh I'm going to talk about the last human code review, and what do we need to do in order to get there? s19 00:01:47.880 --> 00:01:48.720 Okay? s20 00:01:48.720 --> 00:01:53.520 And um can you switch instead of seeing myself seeing the presentation here? s21 00:01:53.520 --> 00:01:54.320 Thank you. s22 00:01:54.320 --> 00:02:02.920 Um so, I I think like first of all, we need to agree on why do we have code review at all? s23 00:02:02.920 --> 00:02:08.640 So, I think basically these are two buckets that we should agree on, and I think I think it's quite common. s24 00:02:08.640 --> 00:02:21.360 One is we want to validate the code that is in high quality, safe, maintainable, the right architecture, according to our best practice, etc. The second re- reason is actually alignment and learning. s25 00:02:21.360 --> 00:02:21.560 Right? s26 00:02:21.560 --> 00:02:33.760 Like where senior developers, for example, has one last chance, a gateway, a gatekeeper before uh it's being uh code is being pushed to production to have that alignment and teaching, s27 00:02:33.760 --> 00:02:34.280 right? s28 00:02:34.280 --> 00:02:42.800 So, these are the two reasons, and then that's what you need to think if you're trying to automate the code review, can human still do that? s29 00:02:42.800 --> 00:02:51.280 And this if this is the right place, the the code review process, the pull request process, is that the right place to still doing these two things? s30 00:02:51.280 --> 00:03:04.000 If you have uh new tools, new processes that will help you unblock this bottleneck, but let you still do these two buckets of tasks, then you're on the right path. s31 00:03:04.000 --> 00:03:13.200 Uh so, we're here to ask yes, no, is human code review still optional end of 2026? s32 00:03:13.200 --> 00:03:16.959 Uh is it becoming optional or is it still mandatory? s33 00:03:16.959 --> 00:03:19.000 Okay, that's that's what we're here to answer. s34 00:03:19.000 --> 00:03:29.640 Do you think like every PR or the majority of PRs are going to be reviewed line by line or, you know, bucket by bucket in the in the chunks and the hunks s35 00:03:29.640 --> 00:03:31.200 by our developers, raise your hand? s36 00:03:31.200 --> 00:03:38.400 Or do you think Do you think by end of this year your developers are still going to review diff by diff? s37 00:03:38.480 --> 00:03:39.080 Okay. s38 00:03:39.080 --> 00:03:41.959 So, try to think why why is that happening? s39 00:03:41.959 --> 00:03:52.680 Now, I I I wanted to share with you that yesterday night we did a drone show, and I had to uh opportunity to talk to different people during that drone show, different uh engineering leaders. s40 00:03:52.680 --> 00:04:02.560 And what I can tell you is that I saw two very different groups of school of thoughts, okay? s41 00:04:02.560 --> 00:04:08.200 They both agree that bugs are coming in different shapes. s42 00:04:08.200 --> 00:04:10.320 For some, this is fine. s43 00:04:10.320 --> 00:04:15.560 And we're just going to fix that quickly after it hits the production and for some not. s44 00:04:15.560 --> 00:04:18.320 So so actually we do see two teams. s45 00:04:18.320 --> 00:04:21.400 Those that are thinking about the room is split into two. s46 00:04:21.400 --> 00:04:31.120 Those are thinking like, "Hey, we have to let make sure that every piece of line is is trusted and and the human must review that. s47 00:04:31.120 --> 00:04:47.160 Or the other group somewhat reckless or so saying, "Let's let's like push those bugs into production and we quickly fix that and that's how we actually do things because it's much faster. s48 00:04:47.160 --> 00:04:51.680 Velocity is more important than getting getting things right. s49 00:04:51.680 --> 00:04:55.200 And I think like you need to think like where do you sit? s50 00:04:55.200 --> 00:05:05.680 Of course I put it in two two sides of the spectrum but there's some somewhere in in between and you you need to think what what's your philosophy because that will lead you to different s51 00:05:05.680 --> 00:05:16.919 milestones or different tools that you need to use in order to get that confidence that you can skip over a human review in the pull request in the code review. s52 00:05:16.919 --> 00:05:20.160 So let's start talking about what is the process? s53 00:05:20.160 --> 00:05:21.720 How do we need to start thinking about it? s54 00:05:21.720 --> 00:05:26.720 So I claim that models are not not a barrier anymore. s55 00:05:26.720 --> 00:05:35.720 Like it's not a matter of of the you know a model doing a good good good job or not, having the right reasoning or not. s56 00:05:35.720 --> 00:05:47.600 Basically, the models are improving but I'm telling you I just came from one of the leading labs where we are inspecting how benchmarks for code review did not change a lot s57 00:05:47.600 --> 00:05:49.920 throughout the latest model. s58 00:05:49.920 --> 00:05:53.560 The The key here is actually context. s59 00:05:53.560 --> 00:05:53.880 Okay? s60 00:05:53.880 --> 00:06:07.280 Like the models, if you give them the right context, and what is that right context we're going to talk about, they could already reason pretty well over what is the issues that we need What are the issues that we need to surface s61 00:06:07.280 --> 00:06:09.840 for a certain change in the code? s62 00:06:09.840 --> 00:06:16.280 Otherwise, if you don't have the context, even the best model out there, they will give you different types of of uh bugs and issues. s63 00:06:16.280 --> 00:06:23.840 Some of them are really good, but in many cases they will simply tell you, "Hey, did you consider error handling uh uh like or not?" s64 00:06:23.840 --> 00:06:27.840 By the way, error handling could be like a really good thing to handle. s65 00:06:27.840 --> 00:06:28.720 Depends. s66 00:06:28.720 --> 00:06:31.400 In some cases it's critical, in some cases are not. s67 00:06:31.400 --> 00:06:34.320 And again, the context is what what matters. s68 00:06:34.320 --> 00:06:39.080 Right now, our context is like spread all across. s69 00:06:39.080 --> 00:06:42.919 Like we have agents.mds, cloud.mds, skills.mds. s70 00:06:42.919 --> 00:06:52.240 And And the thing is that each each one of them like has like different standards, uh different or organization and sub organization are dealing with different differently. s71 00:06:52.240 --> 00:06:56.760 Even within a certain team, you might be using that that differently. s72 00:06:56.760 --> 00:07:07.040 And you're actually maybe using like the same uh one one team is using uh the same agent to do code review and sorry, coding and code review. s73 00:07:07.040 --> 00:07:09.480 The other might be using something else. s74 00:07:09.480 --> 00:07:15.000 And all of that does not bring you the trust and consistency that you're looking uh towards. s75 00:07:15.000 --> 00:07:27.640 Um By the way, like you also might have like coding agents that are running in your IDE, but you're probably building the AI factory that running those agents like in workflows to automate some of the coding. s76 00:07:27.640 --> 00:07:37.640 I see teams that are already uh having more lines of code being uh shipped that are not generated from the CLI or or the IDE. s77 00:07:37.640 --> 00:07:39.800 So, how do you control all that? s78 00:07:39.800 --> 00:07:46.960 Like that's that's missing uh like in right now like in our in our tools in our infrastructure. s79 00:07:46.960 --> 00:07:54.520 And let alone if you add those MCPs and and and rag like style context. s80 00:07:54.520 --> 00:07:57.560 I don't know if you have like great visibility. s81 00:07:57.560 --> 00:07:59.360 There are ways to tackle that. s82 00:07:59.360 --> 00:08:00.960 There's great talks out there. s83 00:08:00.960 --> 00:08:09.520 Check how you can have like MCP versioning and have data sets for every like a benchmark for every MCP change, but that's hard to manage. s84 00:08:09.520 --> 00:08:15.000 We're missing like a governance layer for us to move to the next level. s85 00:08:15.000 --> 00:08:19.160 Okay, like where we can actually trust the code without human reviewing it. s86 00:08:19.160 --> 00:08:21.840 So, where where is that context? s87 00:08:21.840 --> 00:08:22.760 Where is that context? s88 00:08:22.760 --> 00:08:31.800 Basically, I would say experience, tribal knowledge, wisdom of your developers is a lot in their heads. s89 00:08:31.800 --> 00:08:33.599 They are in some documents. s90 00:08:33.599 --> 00:08:42.440 There are documents, infrastructure documents, but a lot By the way, a lot of them is our slacks or or teams or or so. s91 00:08:42.440 --> 00:08:47.120 The data is there, but a lot of the information are are in your developer heads. s92 00:08:47.120 --> 00:08:50.440 And we need like to to codify them. s93 00:08:50.440 --> 00:09:05.000 Now, um I think like basically another thing that I'm saying like when I'm saying that we need to codify human knowledge, what we're actually saying is that we're trying to build an interface s94 00:09:05.000 --> 00:09:11.280 for agents an interface for humans to collaborate each other on that on that knowledge. s95 00:09:11.280 --> 00:09:15.400 And that's a very important a very important point. s96 00:09:15.400 --> 00:09:28.400 When you want to extract that information, the tribal knowledge from your like senior developers, etc., and codify that, do you codify that only in agents' language? s97 00:09:28.400 --> 00:09:38.360 Which is very maybe verbose and structured, or you want to codify that in a wiki style get started and all that what developers love love doing. s98 00:09:38.360 --> 00:09:49.880 And the answer is that you probably need to build your context lake, your context engine, as I mentioned, that's the the the the real like gold mine here to to get the code review s99 00:09:49.880 --> 00:09:51.760 like auto automated. s100 00:09:51.760 --> 00:09:54.200 You have to have it fitting for both. s101 00:09:54.200 --> 00:10:06.360 So, what you're seeing here, for example, is that Codo as an example, but you can use other tools, help you collect all the rules and standards that your team s102 00:10:06.360 --> 00:10:10.240 own, that your team is using day-to-day. s103 00:10:10.240 --> 00:10:16.640 And then, it will provide that information during the review for humans. s104 00:10:16.640 --> 00:10:21.200 Hey, notice that Codo used four rules. s105 00:10:21.200 --> 00:10:29.800 Uh sorry, uh used uh uh many rules and four are violated, and that includes a link to all the rules that are were being used. s106 00:10:29.800 --> 00:10:32.640 That's for human in order to trust. s107 00:10:32.640 --> 00:10:40.000 Okay, in order to trust the results that coming from your code review tool, etc., you have to build that interface for for human. s108 00:10:40.000 --> 00:10:43.560 You have to accumulate that knowledge and have an interface for human. s109 00:10:43.560 --> 00:10:48.400 But, you also want to have an interface that is dedicated for agents. s110 00:10:48.400 --> 00:10:56.560 What you're seeing here, uh for example, is a com- comment, for example, by Codo that is speaking to another agent. s111 00:10:56.560 --> 00:11:05.280 Hey, dear agent, Codo just reviewed this uh PR and has found five different issues. s112 00:11:05.280 --> 00:11:20.040 Codo already spend uh uh like uh some back background task and use cloud code, for example, harness in order to do fixes, and there is a closed PR, like you can see here in the top right, there is some closed PR with all the fixes, s113 00:11:20.040 --> 00:11:33.480 and now, when an agent is coming to review this PR once again, then it has like a cherry-picking moment uh with everything that all the code that is actually is passing your rules, s114 00:11:33.480 --> 00:11:39.360 your standard, uh and everything that we're like more architectural decision that we're going to talk about. s115 00:11:39.360 --> 00:11:52.560 So, what what I said so far is that just to to like recap so far, if you want to trust and you want to get to a point where you're trusting the code that's being shipped, s116 00:11:52.560 --> 00:11:59.520 you need to have the right context that is being gathered and being used during the code review process. s117 00:11:59.520 --> 00:12:09.560 Then, you will see that this code review process have links and information for human and have links and information for for agents, okay? s118 00:12:09.560 --> 00:12:17.720 And when that is in place, you will see that developers are writing less and less comments in the pull request. s119 00:12:17.720 --> 00:12:24.880 And then after 100 of these pull requests, there's human no more human review, you know that you're ready for for automation. s120 00:12:24.880 --> 00:12:25.520 Okay? s121 00:12:25.520 --> 00:12:28.560 Now, back to back to the context. s122 00:12:28.560 --> 00:12:31.400 So far, I talked about relatively simple context. s123 00:12:31.400 --> 00:12:43.640 I talked about uh rules and standards and skills, but actual the human knowledge that the tribal knowledge in your organization sits in understanding the system architecture. s124 00:12:43.640 --> 00:12:50.839 What are the P0s, the the bugs that actually made an outage outage for for you, unfortunately, in the last like 3 months or so? s125 00:12:50.839 --> 00:12:57.440 When a microservice uh one changed its contract and broke a microservice two, right? s126 00:12:57.440 --> 00:13:09.520 That is does not exist in most code review and and like if you try to build yourself, it's really hard to build, but it is available in some of the those code review uh solution that is dedicated for that. s127 00:13:09.520 --> 00:13:17.440 For example, what you're seeing here is the graph being built for a certain microservice and all the repos and and their connection. s128 00:13:17.440 --> 00:13:28.320 And and each node and edge, there is what is the con- if it's an edge, what is the contract between two uh piece of your software, but also links to history s129 00:13:28.320 --> 00:13:36.920 of discussions between developers that they had when they fixed an issue because a root cause analysis and now you need to codify that. s130 00:13:36.920 --> 00:13:38.160 Okay? s131 00:13:38.160 --> 00:13:49.280 And when you get to to that level of a context engine, now you're ready to start approving and blocking PRs automatically. s132 00:13:49.280 --> 00:14:05.360 And you want to do that not just by letting AI some like choose by yourself, rather giving some semantic rules that that for example, when you when do you guys approve or or or block s133 00:14:05.360 --> 00:14:06.360 a PR? s134 00:14:06.360 --> 00:14:11.320 And that knowledge also needs to be accumulated as part of your your context. s135 00:14:11.320 --> 00:14:26.400 What I'm actually saying is that software development, at least code governance, is going to change from reviewing your pull request to actually reviewing your entire software development from a graph s136 00:14:26.400 --> 00:14:40.839 abstraction, where you're seeing your PRs as bubbles with all the issues that might happen even with three different PRs are in on the fly, when they which which contract they might bridge s137 00:14:40.839 --> 00:14:43.560 they they might ruin. s138 00:14:43.800 --> 00:14:44.800 Okay? s139 00:14:44.800 --> 00:14:50.120 And that's how the software development future is going to look like. s140 00:14:50.120 --> 00:14:54.640 What you need to do in order to get there is to codify your standards. s141 00:14:54.640 --> 00:15:01.480 It needs to be built in a way that humans can trust and audit and control. s142 00:15:01.480 --> 00:15:18.160 You need to build real-time self-learning context, learning from PR history, learning from accepted and unaccepted learning from discussions between between developers, learning between learning like from those cases that broke your production. s143 00:15:18.160 --> 00:15:22.360 And that context needs to be not just like thrown into files. s144 00:15:22.360 --> 00:15:30.240 It needs to to to to sit and locate it in a place that agent understand where is that context fitting. s145 00:15:30.240 --> 00:15:35.920 And then you need the governance infrastructure that gives you that visibility of what's happening. s146 00:15:35.920 --> 00:15:39.760 Rather the graph I presented, but there's more to that. s147 00:15:39.760 --> 00:15:47.760 Okay, I'm going to show you very soon another visualization that helps you understand the overall status of your PRs and software. s148 00:15:47.760 --> 00:15:56.120 If you're already shipping AI-generated code faster than your human can review, I'm actually saying that you are in the problem. s149 00:15:56.120 --> 00:15:58.400 You're not like ahead of the problem. s150 00:15:58.400 --> 00:16:09.560 You you if you put the infrastructure, if you gather the context, if you start accumulating how code review can be automated for you, then that's where you're going to get the 10x s151 00:16:09.560 --> 00:16:16.200 velocity that you're being promising your CEO or yourself or your or your developers, because otherwise it's a bottleneck. s152 00:16:16.200 --> 00:16:23.720 You need to own your rules and standards and codify them and get analytics and statistics about each one of them. s153 00:16:23.720 --> 00:16:33.800 How many times they're being caught, which rules and standards and skill is actually being used during the review process, and it is useful or not, or does it need to get an update. s154 00:16:33.800 --> 00:16:38.080 You need to have visual full visualization of your software graph. s155 00:16:38.080 --> 00:16:38.960 What is the connection? s156 00:16:38.960 --> 00:16:40.200 What are the contracts? s157 00:16:40.200 --> 00:16:40.920 What is working? s158 00:16:40.920 --> 00:16:41.880 What is not working? s159 00:16:41.880 --> 00:16:47.920 What are two PRs are going to crash very soon because they're touching the same agent and they don't and they don't know. s160 00:16:47.920 --> 00:16:53.240 And you need to start learning how to auto approve and auto block. s161 00:16:53.240 --> 00:16:55.560 That won't happen in like immediately. s162 00:16:55.560 --> 00:17:10.040 That needs to gradually being automated for you step by step by adding more rules for blocking and more rules for for for approving over time. s163 00:17:11.079 --> 00:17:23.560 So, trustworthy automated review, your rules, your standards, in your software graph placed in the right edge and the right node of your software. s164 00:17:23.560 --> 00:17:30.840 Learning from the tribal knowledge of discussions, of history of your software, and codifying that. s165 00:17:30.840 --> 00:17:38.800 There's so much tribal knowledge what to do and not to do, and that needs to be codified as well, both for the agents and for the humans. s166 00:17:38.800 --> 00:17:44.120 And basically, you need to accumulate that experience and codify that. s167 00:17:44.120 --> 00:17:51.880 And sorry for the buzz or whatever you want to call it, that's moving from artificial intelligence to artificial wisdom. s168 00:17:51.880 --> 00:17:59.840 Because right now, you know better than I that your developer holds the judgement of what's bad and what's good. s169 00:17:59.840 --> 00:18:02.840 It's not your your software not your AI tools. s170 00:18:02.840 --> 00:18:16.800 If you want to get to a point where judgement is moving to your AI tools, that's AI wisdom where that experience need to be codified in the right way, the right place for agents and humans. s171 00:18:16.800 --> 00:18:19.280 And that's where we are here, Codota, to help you. s172 00:18:19.280 --> 00:18:24.120 And our vision and mission is not far away from now. s173 00:18:24.120 --> 00:18:33.040 We want to reach in 2027 to a place where you have zero outages, zero bugs in production, at least the critical and the high ones. s174 00:18:33.040 --> 00:18:34.560 That's why we're here. s175 00:18:34.560 --> 00:18:37.600 Thank you so much for having me.