WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=uuwDWRbxoYo NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/uuwDWRbxoYo.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:13.280 --> 00:00:14.799 Let's get started. s2 00:00:14.799 --> 00:00:16.880 Um, good afternoon everyone. s3 00:00:16.880 --> 00:00:32.880 I'm Ali and um, I'm one of the founders of guitar.ai AI and previously the CEO up until about a month ago where we were acquired by Sonar and now we are part of Sonar s4 00:00:32.880 --> 00:00:44.960 and you've heard a lot in the past few days about the importance of verification in the age of AI and so today I'm going to focus my talk on code and verifying code. s5 00:00:44.960 --> 00:00:59.600 So in in almost all software teams there is this centralized validation phase that is basically CI plus code review and as you scale your engineering this plays this phase plays a very critical part s6 00:00:59.600 --> 00:01:01.600 especially in an AI world. s7 00:01:01.600 --> 00:01:05.760 Uh first it enforces all your quality gates centrally. s8 00:01:05.760 --> 00:01:10.400 So these are security compliance code quality gates. s9 00:01:10.400 --> 00:01:14.080 It's on the critical path of getting your changes into production. s10 00:01:14.080 --> 00:01:16.400 So the faster is always the better. s11 00:01:16.400 --> 00:01:20.080 But as you scale engineering, this becomes kind of a challenge. s12 00:01:20.080 --> 00:01:32.400 As as you have longer running builds and tasks, you have asynchronous team review flows across sides and geographies and things just get slower naturally as you grow. s13 00:01:32.400 --> 00:01:38.320 Um, this is also a phase where you end up spending a lot of developer infrastructure dollars. s14 00:01:38.320 --> 00:01:50.320 So there's a lot of expensive tools here that need to be integrated and scaled up as you um scale your organization and it all runs on very expensive infrastructure. s15 00:01:50.320 --> 00:01:58.880 Uh for these reasons it's often times the main focus of your platform organization team or your platform team. s16 00:01:58.880 --> 00:02:03.840 Um but any kind of failure in this phase is very expensive. s17 00:02:03.840 --> 00:02:11.520 It introduces failures introduce an outer loop that are typically measured in hours and days. s18 00:02:11.520 --> 00:02:13.040 So failures are expensive. s19 00:02:13.040 --> 00:02:13.760 They costly. s20 00:02:13.760 --> 00:02:16.080 They really slow down your velocity. s21 00:02:16.080 --> 00:02:23.599 Uh you lose developer time and that's compounded by disruptions to your flow state. s22 00:02:23.760 --> 00:02:26.000 It's also very expensive to optimize. s23 00:02:26.000 --> 00:02:32.080 So you've got a lot of fragmented systems that have been integrated together with bespoke configuration and scripting. s24 00:02:32.080 --> 00:02:39.760 and you have a usually headcount restricted, constrained platform organization trying to keep it all up and running for you. s25 00:02:39.760 --> 00:02:55.440 And trying to address that all with AI is also kind of complex because it requires complex workflow orchestration involving various inter asynchronous interaction, team workflows, integrations across tools and so on. s26 00:02:56.720 --> 00:03:00.080 In fact, AI kind of makes the problem even worse. s27 00:03:00.080 --> 00:03:14.800 uh you've got a lot of PRs now being generated, more PRs than before uh thanks to AI and bigger PRs uh all of which can have defects and so your costs in your development life cycle are kind of shifting to the right. s28 00:03:14.800 --> 00:03:16.720 You have a lot more code to review. s29 00:03:16.720 --> 00:03:22.640 You can have potential for a lot more failures in production and and so your choices are kind of tough here. s30 00:03:22.640 --> 00:03:31.120 either you slow down and ask developers to review every PR very carefully or you rubber stamp PRs and you risk incidents in production. s31 00:03:31.120 --> 00:03:40.879 Either way, you've got less happy developers, lower sentiment, and much less productivity than than you'd expect. s32 00:03:40.879 --> 00:03:43.440 So, what's the answer here? s33 00:03:43.440 --> 00:03:44.959 Of course, it's more AI. s34 00:03:44.959 --> 00:03:56.640 It's uh an agentic approach to validation which is essentially what we built at guitar um that we're now also shipping uh as part of sonar. s35 00:03:56.640 --> 00:04:08.560 And basically what we've got here is an agent that fully automates all the validation steps and as outcomes it delivers for you green PRs that are ready to merge or merge automatically. s36 00:04:08.560 --> 00:04:10.080 Now how does this work? s37 00:04:10.080 --> 00:04:14.319 Well first when you create your PR it reviews your PR and posts issues and inline comments. s38 00:04:14.319 --> 00:04:16.079 No surprise there. s39 00:04:16.079 --> 00:04:28.639 But our goal has been to really make this experience as noisefree as possible, both in the way we handle the UX as well as posting issues that are real and really matter. s40 00:04:28.639 --> 00:04:37.520 You can customize your review with your own rules and you can add your own custom checks and you can trigger your own custom workflow automations. s41 00:04:37.520 --> 00:04:44.400 And then you can also configure the system to block merging if there are any unresolved code review findings. s42 00:04:44.880 --> 00:04:50.479 It also analyzes CI failures and posts a summary of the root causes of those failures. s43 00:04:50.479 --> 00:04:54.639 And in particular, it's really good at catching flaky tests. s44 00:04:54.639 --> 00:04:59.040 And you can even set up rules that automatically retry flaky tests if you want. s45 00:04:59.040 --> 00:05:01.840 That's a very popular feature. s46 00:05:01.840 --> 00:05:06.080 You can also automatically fix any code review issues or any CI failures. s47 00:05:06.080 --> 00:05:13.120 Any kind of uh failures that happen in CI, it can fix it for you as well as address any of the issues that it raises. s48 00:05:13.120 --> 00:05:26.479 You can either ask it to specifically fix an issue or more interestingly you can set it up to automatically loop until all the fish issues have been fixed in your PR and you have a green PR ready to merge. s49 00:05:26.479 --> 00:05:32.880 And finally, you can automatically approve and merge PRs that are green. s50 00:05:32.880 --> 00:05:45.039 you can define rules and conditions that control when uh PRs get automatically approved by the agent and merged automatically. s51 00:05:45.039 --> 00:05:49.360 So we're seeing actually a very interesting trend in our users. s52 00:05:49.360 --> 00:05:53.840 There's a clear trajectory towards fully automated PRs. s53 00:05:53.840 --> 00:06:04.800 uh as users use the system and as they build trust in the accuracy of the AI what we see is they work towards full automation first they get reviews s54 00:06:04.800 --> 00:06:17.120 and they're happy with they build trust in the accuracy of the reviews they see that these reviews are not only accurate but they're raising really important issues so then they give teeth to the agent by allowing it to start blocking s55 00:06:17.120 --> 00:06:29.039 the PRs and then they start using the autofix capabilities to automatically turn PRs green and fix all the issues that were raised by code review or any CI failures that come up. s56 00:06:29.039 --> 00:06:41.360 And then where the real interesting leap is beginning to happen now is users setting up rules that define when the agent automatically approves and merges PRs. s57 00:06:41.360 --> 00:06:54.720 So over time as they build trust in the system they unlock more levels of automation and with the additional levels of automation and with the trust they're getting more value out of the system. s58 00:06:54.720 --> 00:07:02.880 Now because the system processes every single PR using AI it's also able to uncover new types of insights. s59 00:07:02.880 --> 00:07:12.000 For example, uh one of the insights that are is very useful to platform teams is a categorization of the top CI failures. s60 00:07:12.000 --> 00:07:18.000 Where are you seeing failures in your CI system where your developer is running into failures? s61 00:07:18.000 --> 00:07:30.160 Are these flakiness issues that you can that you need to address by improving the reliability of the tests or are there infrastructure issues that you need to address or are there other kinds of failures that they're running into? s62 00:07:30.160 --> 00:07:32.720 Very very useful for platform teams. s63 00:07:32.720 --> 00:07:41.840 We also have another example of what what else we have in terms of AI enabled insights are categorization of PRs. s64 00:07:41.840 --> 00:07:46.960 What exactly is happening across your engineering team in terms of the PRs that are being landed? s65 00:07:46.960 --> 00:07:50.639 This is extremely useful for leadership teams. s66 00:07:50.639 --> 00:07:59.840 For example, what amount of PRs are going to feature development versus fixing issues, chores, refactors, and so on. s67 00:07:59.840 --> 00:08:05.599 Now under the hood, if you look at the system, there's a number of components that constitute guitar. s68 00:08:05.599 --> 00:08:07.120 There are three interesting pieces here. s69 00:08:07.120 --> 00:08:14.560 One is the control plane that acts as this orchestration layer for handling PR workflows. s70 00:08:14.560 --> 00:08:17.599 And then there's an agent runtime or or harness. s71 00:08:17.599 --> 00:08:31.039 We built our own harness here that handles multi- aent execution, context management, memory management, um tool calling, all the integrations, etc. And having this having our own agent harness s72 00:08:31.039 --> 00:08:43.599 has really allowed us to optimize specifically for the validation use case and it's allowed us to optimize for token cost, precision and coverage of the agent itself. s73 00:08:43.599 --> 00:08:52.399 So that allows us to essentially guarantee outcomes at a fixed PR price while giving really good quality results. s74 00:08:52.399 --> 00:09:00.399 We also have an NLM proxy that handles the routing across different models as well as things like failover. s75 00:09:00.640 --> 00:09:06.959 Now that we're part of Sonar, we've been integrating guitar with sonar cube. s76 00:09:06.959 --> 00:09:13.440 There's a ton of opportunity here in combining an agentic approach to validation with program analysis. s77 00:09:13.440 --> 00:09:24.080 So combining agents with things like u taint analysis, control flow analysis, data flow analysis, software composition analysis. s78 00:09:24.080 --> 00:09:27.760 So all the traditional approaches combined with agents. s79 00:09:27.760 --> 00:09:37.360 We think there's a ton of opportunity here and that together you can get much better precision, much better coverage at a really great price point. s80 00:09:37.360 --> 00:09:50.720 So the overall results uh the value that we're seeing in in all of this is much better quality, much better security, especially with AI generated code, faster delivery times through all the automation s81 00:09:50.720 --> 00:09:57.760 going from PRs straight to approved and merged PRs, much higher productivity and sentiment by developers. s82 00:09:57.760 --> 00:10:10.320 So less time spent on actually reviewing code and worrying about what's happening in production with more confidence as well as sentiment because now you much better sentiment now because you have a lot of automation in place and less grunt work. s83 00:10:10.320 --> 00:10:21.839 And finally the to the features that we've built for platform teams gives them a lot more leverage to be able to fix issues and to add customizations and see insights s84 00:10:21.839 --> 00:10:25.839 in this critical part of the software development life cycle. s85 00:10:26.399 --> 00:10:37.200 To wrap up, PR validation is really critical, even more so in AI, but AI turns validation into a friction point. s86 00:10:37.200 --> 00:10:53.760 And guitar, what we've built, allows you to automate this step, fully automate PR validation, go from created PRs all the way to merge, completely eliminating this friction and helps you keep up with the rate at which you're generating code from AI. s87 00:10:53.760 --> 00:11:07.600 And finally, the combination of using agents uh doing a gentic validation plus an algorithmic program analysis will give you something that that I think is going to be much better than either alone. s88 00:11:07.600 --> 00:11:14.079 If you want to talk more and you want to see more, if you'd like to see a demo, come on and visit us at booth number P7 right down s89 00:11:14.079 --> 00:11:15.440 right down the hall there. s90 00:11:15.440 --> 00:11:17.600 Thank you.