Your AI Agent Starts Every Session Like a New Hire - Killian Carlsen-Phelan, Sonar

AI Engineer · 17 min · 180 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12I am Killian from Sonar.
00:14And if you're unfamiliar with Sonar, we are an independent code verification platform.
00:19Um and what I'm going to be talking about today is how to move verification into the agentic coding loop.
00:25Um I mean as we know everybody has been doing this the past couple years.
00:31It's accelerated the pace and every time you start a new agentic session in your IDE, it is like starting from scratch.
00:39It is like a new hired employee into your project.
00:42It does not know your existing project conventions.
00:45It does not know uh your coding guidelines.
00:48It does not know any architecture rules that you have predefined for it.
00:52And that can be a problem.
00:55Because what it has to do then is when it's assigned a feature, it has to go look around for context, right?
01:02If you say, "Hey, build out a new API endpoint."
01:05It has to go and determine how existing API endpoints work in your code.
01:11And that process can, you know, it's it's a bit of a roll of the dice.
01:16Um it can acquire that context quickly or it can search around all over your repository.
01:23And that takes time and that takes tokens.
01:26And this is a very fixable problem.
01:28And if you have issues in your codebase, right?
01:32As I'm sorry to break it to you, every codebase does.
01:36That is uh unfortunately, it's highly likely that these issues will be replicated and these patterns repeated by whichever agent you're using.
01:44And not if, but when that agent introduces new issues, if it's not caught the ability to maintain the quality of your code is almost impossible to do.
01:59So um our solution to that is Sonar Vortex.
02:03I'm going to show you that here in a little bit.
02:06Um what it is it is a plug-in that connects to whatever coding agent you use.
02:11And uh this is so the agent knows about your project before it even begins to code.
02:17And it is able to verify itself deterministically to make sure that it's introducing no new issues into your code base.
02:25So, um I had I recorded a coding agent writing a small feature a little bit ago and it it was able to do so in under 2 minutes, but unfortunately,
02:35even though I had instructed it to build out a test suite for itself, it introduced a security issue that Sonar Vortex was was able to catch and forced the agent
02:43to continue iterating upon itself until it was fixed.
02:48So, um the issue ended up being, and we'll see this here momentarily, is that it wrote it it was to unzip a log file into a temp file, but unfortunately, the permissions that it set for that temp file
03:02um were not explicit.
03:04And that type of issue is hard to write tests for, it's hard to catch, and it's usually the type of thing that a senior reviewer would only catch if they're explicitly looking for it, if at all.
03:16And then let's say that they do catch it, now it has to be sent back for a developer or an agent to rework again and re-push and re-review and re-analyze
03:26and that burns time and money and it causes a lot of frustration for everyone involved.
03:32So, um that the frustration in that process is not just my opinion, that is uh from data that we've pulled.
03:42Sonar surveyed over 1,100 developers and found that 96% percent of them do not fully trust AI-generated code.
03:52But how many how many of those do you think actually fully verified the code before pushing it?
03:58It It was less than half.
03:59It was 48% of those.
04:01So, that that boggles my mind, personally.
04:05That terrifies me that we are just blindly setting off code into production without verifying it.
04:12Um but um like I said, the agents write quickly.
04:19Rework slows you down, and if you catch it at the PR, um you're already behind where you need to be.
04:26But, uh a quick animation to kind of demonstrate what I'm talking about here is as I mentioned, Sonar Vortex.
04:34One of the goals that we are doing is we are making your coding agents more efficient.
04:40In addition to providing guardrails for them.
04:42But, this little ball here is your coding agent, which as I said, it writes a lot of code really really fast, faster than humans can keep up with, faster than we are able to review manually.
04:54Unfortunately, as I said, when it starts out, this this grid is your code base.
05:01Every square is a file.
05:02It has to go searching around.
05:04Every time it moves, it's checking a new file.
05:07And that bar at the bottom is tokens, which is your money and time spent on just looking around.
05:14And when it gets something wrong, you get to find out at the pull request.
05:19The review fails, and the work goes back to the agent for another round, and it continues to iterate itself, and you got to keep pinging your senior engineer, and then he's mad at you, and then nobody has a good time.
05:31Before it writes anything with Sonar Vortex now, it is going to get the coding guidelines for your project and a map of how the code connects.
05:43So, it will pull in things like your architecture guidelines.
05:47It will pull in things like what the approved libraries for your applications are, what your coding standards are for uh a given project.
05:56And these are all things you get to set for yourself.
06:01So, now you can see here, it goes straight to to files.
06:03It does not burn excess tokens, it does not burn excess context.
06:08And as it works SonarQube checks its code.
06:12So, your agent, the edits that it makes with its own tools, every time it uses them it triggers a quick check.
06:19And before it ends its turn, it's told to run a deeper check um for even deeper security issues for cross-file data flow.
06:28And you see here that issue goes back to the agent and the agent fixes its own code and then Vortex checks it again.
06:36And this all works with all of the agents that people are currently using right now, be that Cloud Code, Codex, Antigravity, whatever you want to use.
06:47And if you have any custom agents designed or using any open-source ones or any of the non-major players, uh we have MCP clients that are very configurable for you to use
06:57at your own desire for that.
07:00So, essentially what I want you to take away from this is that Vortex is something that you can just plug in easily to your agent.
07:07It it isn't built on top of the CLI and MCP tools that we provide it and it brings SonarQube into the agent so that the agent knows your project
07:18before it writes.
07:19I'm going to pause it or I'm going to move back before I miss it.
07:23And what we found is that it spends 36% fewer tokens on average, especially on tasks like refactoring existing codebases.
07:31People who use it have a 44% reduction in production outages and we have a very low false positive rate as well so that we don't clutter up your coding
07:42sessions.
07:44So, um this is huge.
07:46This is huge.
07:47This the speed that this um accelerates the coding cycle to the software development cycle to is um it's we're on the new frontier being able to verify uh programming.
08:01So, to sum it all up we want to guide the agent and verify the change.
08:07And we bring SonarQube's existing intelligence into the mix.
08:11So, I'm sure at least some of you are familiar with SonarQube.
08:15We've been around for almost two decades, and we are traditionally a a pipeline step that a lot of people run that deterministically analyzes for any security issues, maintainability issues,
08:27any data flow issues, and we build out graphs, and we dive deep into the code.
08:32Now, with this knowledge already maintained within our servers, we're able to provide that context to an agent, so now it exists from the get-go.
08:43Um your agent knows your project before it writes, and it checks its own code as it goes.
08:50Your CI already has a SonarQube scan, and then Vortex reuses what that scan has learned, so it's just a quick check on your file, and it knows about the rest
09:00of your code.
09:00So, here I'm going to show it to you in a real task.
09:05This I actually recorded last night.
09:08And what you can see from the get-go, and I'll be pausing and restarting it here and there.
09:13Uh the first thing that I want you to see is that without having to do anything, we can see SonarQube has started up in the session.
09:20Right?
09:20Our CLI is detected, and the hook system is detected as well, and secrets detection is on.
09:26I'm not going to dive too much into that today, but um secrets leakage is one of the easiest things to prevent, and one of the most costly for your organization.
09:39Our plugin prevents uh secrets leaks into GitHub and into agent uh context memories.
09:47And what I want you to notice is I didn't have to paste any Sonar instructions into the prompt.
09:53It's just all automatically loaded.
09:56And so, the request that I alluded to earlier, right here, is uh it's just basically unzipping a log file into a temp file.
10:06It says, "Keep reusing that same file, and then write a test that proves it's reused."
10:11And before it writes a single line, it's going to ask Sonar two things.
10:14It's going to say, uh where does this class live?
10:18And what are the Java coding guidelines for this project?
10:21And it's right here.
10:22It's Sonar context navigation, um and we get this pattern, this single GC log file that I told it to work on.
10:28So, it gets parent and child architecture, um as well as existing patterns, um and and uh guidelines around that.
10:38As well, it says Sonar context guidelines get language is Java.
10:41So, we have um basic guidelines that we enforce agents use when building out Java projects, but if you have any customer requirements that you want your team to follow,
10:53uh that gets injected from the get-go.
10:55And guys, I cannot emphasize enough that outside of model choice, the context and steering from the get-go that agents receive is the single greatest lever that you can manipulate
11:09for agentic performance.
11:11Um and the more you do on the front end, the less rework and steering and correcting and intervention that you have to do when you're developing for yourself.
11:22So, like uh instead of searching through the files one by one like in the animation, it just gets that map.
11:28This is the guide phase uh that that Vortex does.
11:33And so, it's able to inspect the existing test, see how uh that operates, and very quickly move on into implementation.
11:43Now, that just took a few seconds, and obviously this changes from feature to feature, but running against this feature without Vortex takes longer as this has to grep around
11:55and figure out what's going on.
11:58So, it's running the implementation, and I will very quickly point out uh where it actually makes that temp file.
12:07Um it adds in its own test suite.
12:09So, here's the test suite.
12:10Uh in the recording, I scroll up.
12:12This a little bit right here is a temp file.
12:14And I hope you guys can see.
12:15It's probably really small for you in the audience.
12:18Um, but it creates that temp file right there.
12:21And, um, it's going to go ahead and check its own test result, and then it will run the sonar check on both of the files that it changed.
12:28Right here it says confirming the test result, and it says running Vortex analysis.
12:33Every time a file is edited or added, Vortex analysis runs.
12:40You do not have to enforce it.
12:42You do not even really have to think about it.
12:44It just automatically kicks in, and you can trust your code gets verified.
12:49If you have automated workflows set up, if you run a sophisticated, you know, factory design, or you're building these out, this is a layer of trust you need to add
12:59in order to make sure that no issues are being introduced.
13:02One point as well, this is not another AI checking AI, right?
13:08Um, and it's certainly not it checking itself.
13:12That is, unfortunately, we found one of the least reliable ways to ensure that issues are not being introduced.
13:20Not only do agents think orthogonally on the creation and review of themselves, um, other Asian agents are likely to miss and hallucinate when reviewing existing code.
13:34When you have a deterministic layer, and by that I mean, every time you run the analysis on the code, you get the same result back, that is how you are able to provide reliable,
13:45governable, uh, guidance and verification that is compliant and predictable.
13:52Don't get me wrong.
13:53I love AI review.
13:54We also have a, um, AI reviewing tool that I'm not going to talk about today.
13:59It's called Guitar.
14:00I ran a workshop earlier this week on that.
14:02And please come by our booth later if you'd love to hear more information.
14:05But, Sonar Vortex is a deterministic layer that stacks on top of your AI coding, and sandwiched between another AI review as well.
14:17So, we're running the Vortex analysis and it is going to catch something.
14:24It'll catch six findings that come back and Claude will say two of those findings are on lines that it just wrote and four on pre-existing code.
14:32Now, we did not engineer Vortex to be super noisy.
14:36We did not engineer Vortex to say, "Oh, you found one issue, but you got to fix three more."
14:41On new code that is created, it enforces correction and that just surfaces it to you so that you have it for your awareness and if in the future you'd like to fix it, you are able to.
14:54One of the findings um that that it has is a security problem, the one that I mentioned uh at the start of this, which is that Claude put the log
15:03in the default temp folder without saying who can read it.
15:06And on a lot of servers, that folder is shared.
15:10So, uh Claude goes ahead and it uh fixes uh the issue here.
15:16But, unfortunately, and we'll see this in a second, Sonar Vortex runs again and determines, "Hey, you actually did not fix the issue."
15:27Says, "The finding persists because the temp file is still in a shared directory."
15:32The first fix leaned on a Java default and on Mac uh and Linux, the default is already owner only, but the rule that we set in, which is a high bar that we set, is that this is the permissions must be set explicitly.
15:46We do not compromise when it comes to uh security.
15:49We do not compromise when it comes to quality.
15:53So, finally, Claude is able to implement these fixes and again, every time it runs, Vortex analysis is initiated.
16:03You do not want to let any issues into your PR because the quicker you can catch these, not only does the surface area of the issues get reduced, but the time and money and energy
16:17for remediation decreases.
16:20Now, uh that's it for the terminal session.
16:23I'm not going to make everyone stare into a tiny uh pre-recorded terminal session, but essentially, what I want everyone to take away from this is that it asked Sonar
16:34first.
16:36When Sonar flagged it's fixed, it went further and continued to remediate itself.
16:41And it will eventually hand it back at the end, and a deep analysis uh for data flow will be performed.
16:51What do you get?
16:52What do you get out of this?
16:54You get less back and forth review, and I see I have a minute left.
16:57I'll uh finish up here in a bit.
16:59You get less back and forth forth review, you get it fixed in the moment, you get more secure code, you get less time and money spent on remediation, and
17:09um you have more time freed up for your developers because of your ability to trust uh automating agentic workflows.
17:19We have a booth downstairs.
17:21I would love to talk to you guys.
17:22I'll be hanging out behind uh the room in the back uh to answer any questions.
17:27I would love to talk with you.
17:28If you're brand new to Sonar, if you're an existing Sonar customer, um you know, I'd still love to hear your input and insights.
17:34And um that's all I have for you today.
17:37Thank you very much.