AI Coding Agents Are Breaking Big Codebases — Dan Adler, Sourcegraph

AI Engineer · 12 min · 137 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Hey, welcome everybody.
00:13I'm Dan Adler, CEO at SourceCraft.
00:15It is uh great to see you all here today.
00:17Thank you so much for coming out and joining us.
00:20Um, this is a talk today about the unglamorous but loadbearing work of owning a large complex codebase and why AI and agentic coding tools today make that job more difficult and more important than ever before.
00:38Just to start, the software that runs the world is not pretty.
00:43It's not new.
00:44It's not clean.
00:45It's how everything actually works.
00:4972% of software industry employment is in companies with more than 500 people.
00:54These are the large enterprises of the world.
00:57These are the companies managing thousands of repositories and decades of codebased history.
01:02Few of your peers work at startups at the hot companies you may hear about on social media.
01:07And even fewer of them are actually like ex soloreneurs making a living off of their um social feeds.
01:14Most of us work in large longived code bases.
01:17Those code bases are how your bank account will reject a transaction in real time if you hit your limit.
01:23They're how your insurance carrier calculates the reimbursement rate for a policy holder with dual coverage.
01:28It's how a chip and a warehouse scanner will actually allow Amazon to tell you when your socks arrive.
01:33It's how your Uber estimates an arrival time.
01:35It's how an airplane uses radar to uh adjust trajectories.
01:39How your paycheck gets issued.
01:41It's how your office air conditioner ticks on and on and on and on and on.
01:46These are the things that keep the world moving.
01:49We are all completely utterly dependent on massive decades old super complex code bases that have been built over many many years by thousands of engineers and coding agents are writing more better code today
02:09faster than we have ever seen before.
02:12A title wave of code is coming for you.
02:15It's coming for your code base and for all of your engineers on your team who are trying to just keep their heads above water.
02:23Every one of us has felt this pain, this flood of AI generated code that we now have to review and make sure is healthy for our codebase and we're exhausted by it.
02:31And look, there are now code review agents.
02:33I'm sure there's a lot of them around this room right now.
02:35There are tools designed to help you measure your code base's health and so on.
02:39And look, the new the major coding agents will start to pull this stuff in.
02:43The models will get better over time.
02:45We'll keep putting sandbags down to keep this flood held back, right?
02:49We're going to keep finding these local maxima to help ourselves sort of avoid this problem.
02:56But meanwhile, what's happening is that these code bases, these massive ones that keep the world running are beginning to decay.
03:04millions of lines of code, tens of thousands of repositories, way too much to fit into a context window or even just to clone and GP in real time.
03:13That's just not possible.
03:15And this is causing big problems.
03:17We're publishing new features, new patches at record speed.
03:21There are different coding standards being applied in different places by different agents across the codebase.
03:26There's duplicated code proliferating.
03:29You you already have a library that does this, but your agent doesn't know that.
03:34There's cross-ervice dependencies that are becoming more brittle.
03:37There are slight deviations in standards that are creating more insidious hidden issues across the codebase.
03:43And maybe more important than anything else here, there are new vulnerabilities being uncovered by agents every single day.
03:49They require constant constant oversight and upkeep of all of these massive legacy code bases that nobody wants to think about or deal with.
03:59The job of owning a codebase is harder today than it has ever been before.
04:07And at the end of the day, the problem here is actually the volume of code.
04:12The call is coming from inside the house, right?
04:15The very tools that are speeding all of us up and letting us write more code faster than ever are also the tools that are creating the conditions that will cause these massive code bases that run our world to begin to fail.
04:31Every day I talk to technology executives at these large companies and they are feeling this so viscerally.
04:37The same pain that I imagine many of you are feeling too, right?
04:40I want to give my devs the best tools possible.
04:44But just today, I overheard a dev say, "I don't know what this code does.
04:47AI wrote it for me."
04:49This is a tech leader and executive at a top 10 car manufacturer.
04:52When you're writing vehicle autopilot code and you have thousands of engineers in your org that you have to manage, you want to enable, this is terrifying.
05:06All right, here we go.
05:11So what I um deeply believe is that these codebase owners deserve better.
05:17Today's agentic dev tools are not meeting the moment for this.
05:21The infrastructure to see and understand a 50,000 repo codebase is not being built or sold by OpenAI, by anthropic, by cursor, by the rest of them.
05:31This is a hard and unique problem.
05:34Their agents are so unbelievably good at smalling at solving the small-cale problems that most developers deal with every day that they are their demand is growing faster than ever before.
05:45And this bigger problem of these code bases beginning to fall apart is going unnoticed.
05:52So this session this is a call first of all off to celebrate the people the owners of these code bases people around the world who are actually keeping them alive and keeping them running.
06:01most of our software engineer peers around the globe.
06:04So, thank you all so much for what you're doing.
06:08But this session is also a call to dev tools companies out there, to the agent harness builders, to people who are building the infrastructure for these agents to help build the the infrastructure, the the ability to go see and understand
06:22these massive scale repositories and give that to their agents.
06:36This is a great quote that I heard from um a tech leader at uh one of the top 10 US banks.
06:40Tens of trillions of dollars in assets uh in custody or administration.
06:45Sure, Claude Code can uh make this change, but I have 90,000 repositories to make it in.
06:51This is actually a specific quote about a specific supply chain vulnerability, an npm thing.
06:58won't get into it, but if you're a bank and you're seeing agents uncover and create these new vulnerabilities at an unprecedented pace and you're being told to go use cloud code to solve this,
07:09that's a problem.
07:10Good luck doing that in 90,000 repositories, mapping out how these code bases all come to together, what's actually living in production across your company, across your product offerings.
07:23Because at the end of the day, right, what's really funny about all this is that, if we know anything about LLMs, is that they just love to search.
07:32It's like a new hire getting up to speed on the codebase.
07:34It's how agents build models of the world.
07:37The repo architecture that you're plugging into your agents.md files, like, let's be real, that's barely moving the needle here.
07:44This agent is going to grap gp gp gp gp its way to codebase understanding.
07:50That's simply how it works.
07:52But understanding is a context problem.
07:54You can't gp what you literally cannot see.
07:57This is tools and infrastructure that's needed.
08:00And the infrastructure to go see and search and really deeply understand a 500 or 5,000 or 50,000 or half a million repo codebase is not being built by the agent builders today.
08:17And so that's where what we're solving, what we're working on as a company.
08:20This is what source graph is built around, right?
08:23Give this a second.
08:26If you'll let me sort of take a moment and predict the future of our industry.
08:31These massive code bases are not going away.
08:35In fact, there will be more code.
08:37There will be much much much more code out there in the future.
08:43And we believe very deeply that building tools to give agents the ability to actually see and understand and evolve those large code bases is essential for the future.
08:56And where this comes from at the end of the day is having a graph that code graph in mind.
09:00The foundational graph of your codebase including search and then the actual compiler accurate output.
09:08Um, if you go and and do that analysis that is essential to give your agents the ability to go and understand how to make changes effectively.
09:17Visibility is infrastructure.
09:21And we've actually built a lot on top of this, right?
09:23We're not stopping here.
09:24Actually, just today we launched our new um our new agentic batch changes product into beta.
09:30This is a frontier agent uh and its job is to let the owner of a massive codebase actually execute code changes across hundreds or thousands of repositories at once starting with a single prompt.
09:43This is a product that will iteratively roll out changes across that that codebase.
09:47Self-heal respond to CI status PR comments.
09:50Use coding agents where it makes sense, where judgment is needed.
09:52Use deterministic scripts where it's not.
09:55and provide tracking and auditability to be sure that you've covered 100% of the places where that patch is needed, where that change is needed.
10:04Because like I keep saying here, at the end of the day, this is an infrastructure problem.
10:08When you're putting code out there into the world, you need to make sure that it's actually going to keep your codebase healthy, that it's actually going to make be able to um adapt to the places where your codebase is beginning to fall apart.
10:21Here's a great quote from an early access user um uh actually at Merkari, a global um shopping tool out of Japan.
10:27Incredible company, massive codebase, hundreds and hundreds of independent microservices running in production.
10:35This user was using um this product to go actually patch a GitHub uh code injection issue where you have to set environment variables correctly.
10:42Um he ran this on two repos where he knew this issue existed and then said yeah go explore the rest of our codebase and found 80 other potential vulnerabilities
10:50across their codebase in other places.
10:51This is like bas this is like config files.
10:53This is basically just going and doing that scan patching them all in one go with a sort of deterministic script that can go make sure it's making that change consistently
11:02effectively across 100 of the places where that risk exists.
11:06That is what confidence is all about in the agent coding era.
11:12So yeah, I'll stop here.
11:13If you're wondering right now about like how many repos in your codebase, how many forks are there out there, how many um copies of repos are there out there, how many repos have your in your developers brought into your um company and are now being plugged in as a library or an API somewhere in your codebase that's
11:28live in production that you're not aware of.
11:31That's the sort of thing that we're here to help.
11:33Right?
11:33If you're wondering what code is deployed right now and how you can go scan it all and see it all and make sure it's all updated, come talk to me.
11:42And uh tonight we're actually hosting a event here in SF called Code Cocktails and Cuttovers.
11:48Come on out, join us.
11:49Um come find us.
11:50Bring your gnarliest codebased stories.
11:52This is where we want to uh talk and hear from all of you about what you're trying to deal with and how you're dealing with the title wave of code that's coming for you.
12:01Cool.
12:01Thank you.
12:02I'll stop there.
12:02Any questions?
12:05Thank you.
12:20[music]