Generation Is Cheap, Review Is Expensive: How to Stop Shipping AI Slop — Gabriel Martinez, G2i

AI Engineer · 17 min · 200 sentences · from YouTube's caption track

Each timecode opens YouTube at the start of that sentence. Line anchors (#s42) are the cue ids in the WebVTT, and every line carries its start and end seconds. All transcripts has every talk, and the whole corpus as one file.

00:01[music]
00:12Hi everyone, my name is Gabriel Martinez.
00:14I go by Gabo.
00:16I'm an engineering manager and resident slop warden at G2I.
00:20And I'm here to talk about slop.
00:22I'm here to talk about slop because I'm rather passionate about the topic.
00:26And by passionate, I mean angry.
00:29The term slop has been beaten to a pulp, but the thing it points at has long been around.
00:34Only now it happens at warp speed.
00:39All right.
00:41Slop is what happens when judgment when generation outruns judgment.
00:45Slop is not AI generated code.
00:48I don't mean to define it as some concrete objective thing that can easily be sniffed out, although there are tells.
00:55Slop is output that looks finished before the thinking is finished.
00:59It is work product where ambigu ambiguity was cleared with guesses, where trade-offs were left to chance, where ownership was not taken.
01:08Output was produced, sure, but understanding was not.
01:13And the real danger is not simply that the machine made a bad choice.
01:16The danger is that the engineer never noticed that there was a choice to make.
01:20They never hit the ambiguity.
01:22They never felt the constraint.
01:24They never discovered the edge case that would have changed the design.
01:28The work arrived already smoothed over and with it the unknown unknowns stayed unknown.
01:35But software is a record of choices.
01:37Technical choices, product choices, operational choices, userfacing choices.
01:43If we do not see those choices, we cannot own them.
01:46And if we do not own them, we are not really engineering.
01:49We are just accepting output.
01:51We are just a code monkey hitting the merge button.
01:54I'm not the purist that's come here to tell to tell you that AI is bad.
01:58It's not bad, okay?
02:01I'm here to describe some of the patterns that have led to our drunken AI stuper.
02:06It's absolutely understandable given the advancements made since December 25, how we got to where we are.
02:12But we need to dial it in.
02:14We need to slow down.
02:15We need to sober up.
02:17Agents can seem to make magic.
02:19And the internet would make you believe it is so.
02:22They can write code, scaffold screens, and with right tuning can build whole features or systems.
02:28But they cannot replace the human work of deciding what matters, understanding what is being built, interpreting ambiguity, owning the system for the long haul, exercising judgment and restraint at the system architecture level, and arguably the most important important
02:44knowing what not to build.
02:46SLOP is not about AI as a tool.
02:49Rather, it is about the absence of judgment.
02:52So, let's talk a little bit about what got us here.
02:56In a world where anything is possible, deciding what to build is more important than ever.
03:00People have gotten overly excited and built behemoths of spaghetti product nonsense.
03:06It's almost as if they've forgotten that any line of code you write is a liability, that the best code is no code at all.
03:13The cost of creating code is falling, but the cost of understanding a software program is not.
03:18And I want to be clear here.
03:19I'm not saying every line of code needs to be personally read, blessed, and understood in isolation.
03:25That has never really been the standard.
03:27Large systems have always contained more detail than any one person can hold in their head.
03:31But the logic has to be understood.
03:34The main flows, the boundaries, the data model and product behaviors have to be understood.
03:39Good software has a kind of harmony to it.
03:42Not in some precious aesthetic sense, in a practical sense.
03:46The code should reflect the shape of the domain.
03:49AI can generate code, but it cannot magically give a messy system coherence.
03:54An agent can operate with much more effectively inside a codebase that has structure, patterns, clear boundaries, and rationale.
04:00But if the system is already a big ball of mud, the agent does not rescue you from the mud.
04:05It just builds the ball bigger, faster.
04:08Just because you can generate another component does not mean the product needs another component.
04:13Dystra had this great line.
04:15If we could count lines of code, we would not think of them as lines produced, but as lines spent.
04:20It is the cost of building.
04:22AI makes it easier to spend lines.
04:24It makes it easier to spend on files, abstractions, and complexity.
04:28But at the end of the day, you own the bill.
04:31That bill shows up as confusion, brittle flows, features nobody wants to touch because they don't fully understand.
04:38It shows up as product surfaces that technically do everything but feel like a maze.
04:43Think about that special kind of B to bloated B2B software where every task requires training where every screen has 12 escape hatches and nobody can explain whether the complexity is there because users needed it or because teams just kept adding things.
04:58Teams should be held accountable for work they are producing.
05:01Accountability is the key word here.
05:03Taking ownership over the product you're building and having a way to tie that to positive or negative reinforcement.
05:09Just in the mere fact of knowing that someone else will review your work will force engineers to pay just a little bit more attention to the finer details and be a bit more meticulous,
05:18especially if there are accountability metrics that can be made visible for each chunk of work product.
05:23We're entering the age of bloated bloated products with spaghetti UX on the order of worst kinds of B2B ERPs you may have ever been trained to use.
05:34Today, anyone can buy code an application in an hour and end up with a working clickable prototype.
05:39And that has changed expectations because once people see software appear that quickly, they start to believe software can be built that quickly.
05:47They see a few prompts turn into screens.
05:49They see a terminal session produce something that looks real.
05:52And to be fair, sometimes it is real enough to be useful.
05:56Sometimes that prototype is a great way to explore an idea.
05:59But the problem is that stakeholders do not live in the codebase of off that stakeholders that do not live in the codebase often experience that prototype as evidence that the hard part is over.
06:09They're not thinking about the parts where real battleh hardened engineers earn their keep.
06:14It's outside the happy path.
06:15Figuring out co how to cohesively integrate a new feature set.
06:19The the work involved in clarifying requirements.
06:23the work of actually manually testing a feature set to ensure that it works as a human should expect and that the UX isn't complete garbage.
06:32They're not at all thinking about the most timeconuming bits, the human parts, the last mile where the judgment lives.
06:40This is where someone has to read the code, test the behavior, own the trade-offs, notice the missing requirements, and decide whether this thing deserves to be a part of the system.
06:50Here's a tweet that emphasizes the point I'm trying to make here.
06:53It quotes the creator of SQLite who on the topic of pull request says, "You say, 'Oh, it's free.
06:59No, it's not free.
07:00What you're doing is asking me to maintain it for you, to document it for you, to test it for you, to maintain it for you for the next 25 years.
07:07That is not free."
07:09And this really struck a chord with me.
07:10I couldn't have said it better.
07:12It's easy to yeet a PR, but who has to own that thing?
07:15Who's responsible for the repercussions of what that means?
07:19So external pressures from stakeholders coupled with pressures of competing with slot misters begets mucho slopp.
07:28I'm sure most of you have seen this in the wild where the bad engineers, the ones who don't review the details, that don't refine the spec, that aren't detail oriented,
07:37end up producing more faster.
07:40The folks who pay attention to the detail and go above and beyond are the ones that might seem like they're doing less and moving slower.
07:47How this propagates is it yields performance anxiety where in team environments can pressure engineers to merge PRs faster than they can judge leading to code being merged without full understanding.
07:59Short-term thinking getting that PR in wins over long-term code health.
08:04I've personally felt this anxiety where engineers around me were merging at an incredible pace and I felt constant anxiety that I was not moving fast enough.
08:12And I can admit that it led me to merge code that I didn't necessarily review with a level of detail that I normally do or should have.
08:19I've seen horror stories of people merging code they don't fully understand or haven't even tested themselves.
08:25At the end of the day, we're all human.
08:27We are driven by incentive.
08:29And if incentives are driving people to get PRs in, then we're going to be in for a rude awakening.
08:36Rewards should be centered around producing value.
08:39elegant, well-factored, pragmatic system design and around clarity of test and evaluation harnesses.
08:46There's just something that really grinds my gears about seeing a multi,000line PR.
08:51There's no way in hell that a human can fully understand all the important bits of a change that big.
08:56It is our job as engineers to optimize for this new world, to decompose needs into its simplest chunks, which lends to understanding.
09:05understanding simpler code surfaces and controlling of our code bases.
09:12AI is supposed to have been eradicating jobs and engineers are supposed to be moving at the rate of token production.
09:19So there is a lot of pressure to be in producing more faster.
09:23What AI is really doing is producing more faster.
09:27More characters that eat more human time.
09:30I've seen this happen in the most mundane ways.
09:32You give someone 10 bullets of scope, a few user journeys, something intentionally lightweight, and they come back with a 20page document.
09:41Maybe it looks polished with fancy pros.
09:44But now I have to spend hours figuring out whether there's any real thinking inside it.
09:48That is AI generated work in the worst sense.
09:51It does not clarify the problem.
09:53It expanded it.
09:54It took ambiguity that should have been resolved by the person doing the work and handed it back as a larger artifact for other stakeholders to do it.
10:03In code, a vague task becomes a large PR.
10:07A small feature becomes a pile of files.
10:10The output arrives with the shape of completion, but the real work has just moved to the reviewer.
10:16And that is the part that we need to be honest about.
10:19Generation is cheap, but evaluation is expensive.
10:23If AI produces more than humans can possibly evaluate, it has not accelerated the team.
10:28It has created a bottleneck.
10:32One thing that I find particularly annoying is Twitter.
10:37You've got all these self-proclaimed experts and clickbaiters summarizing some talk or promoting some truths that we've always known as if there's some novel genius idea.
10:46The Ralph loop was one of those.
10:48It was made to sound like a novel agentic breakthrough when it's just a while loop.
10:53Somehow we've completely forgotten that lines of code is a garbage metric for productivity.
10:58But then we decided that it's different than deciding the engineer who burns the most tokens must be producing the most.
11:04We used to break a feature into small atomic pieces we could reason about.
11:09Small enough to hold in your head and actually review.
11:12Now we type two sentences and expect a whole not fully thought out feature set back in one shot without really understanding it what it is we're asking for.
11:22The key point I want to highlight here is that we do not forget the ground truths we've always known.
11:2860 years of software wisdom hasn't been invalidated by AI.
11:32Composable code, good patterns, and solid practices remain essential.
11:40A friend recently challenged me on why I spend so much time thinking about guard rails for agents, coding conventions, architectural styles, and and constraints around how the system should be shaped.
11:51His question was basically what's the point if the agent writes the code why should we care about what the code looks like and to an extent I agree there will eventually be a world where much implementmentation
12:05detail is completely abstracted away from humans that has been the soft the direction of software from the beginning we went from machine code to assembly which another of level of abstraction up got us to C then Java
12:18then languages like Ruby where the primary design goal was programmer happiness.
12:23Just like in languages, we built abstractions for garbage collection and manual pointers, AI is abstracting away the imperative bits.
12:30It does not remove the need to understand how a system comes together cohesively.
12:35This is why diagrams matter so much, not as decorative documentation after the fact, but as a way of making systems legible.
12:43When a agents can generate thousands of lines of implementation detail, humans need a compressed review surface.
12:50service boundaries, data flows, state transitions, and failure modes.
12:54A good diagram lets us see the shape of the system in a way code often obscures.
12:59We can leave the main process or workflow as the reviewable surface with implementation details pushed to the leaves.
13:06That is why I vouch for declarative code and architecture patterns that create layers humans can actually reason about.
13:12The goal is not just code generation.
13:15The goal is human knowledge, human understanding, and human review.
13:19We need to optimize for that human layer.
13:23Cognitive overhead is a tax every engineer has to pay before they can make a meaningful change.
13:30It is the time spent figuring out where code belongs, which pattern to follow, and what proper implementation actually means.
13:38AI does not remove this tax.
13:40Unless tamed, it exacerbates it.
13:43An agent can generate five plausible patterns in five different styles.
13:48And now the human reviewer has to reverse engineer not just the code but the intent behind it with proper conventions intent is more easily gropped.
13:58The answer to this is conventions standards documented flows small reviewable surfaces and clear ownership boundaries and definitions of done.
14:07The more obvious the path, the less room there is for slop to hide.
14:14I want to give special recognition here to Rails and its place in my heart.
14:19Rails matters here because it shows what happens when a framework aggressively reduces decision surface.
14:24When conventions reduce your overhead rather than ending up with 10 different ways to structure the same concept, you end up with one.
14:33It may not be perfect in everyone's subjective opinion, but it's understood and it works.
14:39In my career, between my first and second 4A with Rails, I had a gap of four years.
14:45It was beautiful to come back after that gap to see that the same patterns in de facto libraries were still the same patterns and de facto libraries.
14:54I could hop right back into a codebase and immediately understand intent without needing to parse through architecture.
15:01It allows engineers to easily move between projects laterally with minimal ramp up.
15:07The same can't be said about other language ecosystems.
15:11JavaScript environments with a single correct way of doing things are superior for agentic development because agents aren't confused by dozens of competing approaches.
15:23This is well supported by the official Rails doctrine.
15:26It explicitly names optimize for programmer happiness and convention over configuration as core pillars and describes convention as a way to eliminate recurring decisions and lowering the barrier to getting started.
15:39It is the reduction of cognitive overhead.
15:44Everything I've talked about today is what led us to build ORC.
15:47Not only because we think the answer, not only because we think the answer is more agents and definitely not because we think engineering judgment can be automated away.
15:57We built ORC because we kept seeing the same failure pattern.
16:00AI could generate code but the surrounding discipline was still missing.
16:04The planning, the review, the adversarial checks, the context management.
16:09Over time, we found ourselves repeating the same human processes to fight slop.
16:14break down the work, manage context, weigh out your options, and make sure the output enters the codebase in the way the team can still reason about.
16:24ORC is taking those lessons and baking them into the workflow.
16:27It is not a magic box that makes correctness disappear as a human responsibility.
16:32It is a system for preserving that responsibility while letting agents do more of the lift.
16:37The goal is not to ship more generated code.
16:40The goal is to ship software that is still coherent, tested, maintainable, and owned after the generation happens.
16:47Because the future is not vibe coding better, faster.
16:51The future is orchestration with judgment.
16:54Thank you all for your time and remember, don't get sloppy.
17:14[music]