WEBVTT

NOTE Sentence-level transcript of https://www.youtube.com/watch?v=WLXxTaPagA8

NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/WLXxTaPagA8.html. A cue ends where the next begins, or 2 s after its last word.

s1
00:00:00.000 --> 00:00:04.840
Here's something nobody warns you about when you start building agents alone.

s2
00:00:04.840 --> 00:00:06.840
You think you're building prompts.

s3
00:00:06.840 --> 00:00:09.960
You think your building skills are worthless.

s4
00:00:09.960 --> 00:00:17.040
If you build long enough, specially alone, you will start building something completely different.

s5
00:00:17.040 --> 00:00:20.640
Something that looks suspiciously like CICD.

s6
00:00:20.640 --> 00:00:26.880
Except worse, because you're building it from scratch, one failure at a time.

s7
00:00:26.880 --> 00:00:27.680
I'm Sumaiya.

s8
00:00:27.680 --> 00:00:31.520
I run a 19 skill cloud code agent system.

s9
00:00:31.520 --> 00:00:39.000
Writing, research, vault sync, analytics sync, hook, transcript, and many more.

s10
00:00:39.000 --> 00:00:45.000
And the most useful thing I learned from building this was not how to build better prompts.

s11
00:00:45.000 --> 00:00:51.600
It was recognizing the five controls I was rebuilding badly and what you can do instead.

s12
00:00:51.600 --> 00:00:56.600
Before I show you the problem, let me show you the system that taught me the problem.

s13
00:00:56.600 --> 00:00:59.080
This is the agent content system.

s14
00:00:59.080 --> 00:01:00.600
It is open source.

s15
00:01:00.600 --> 00:01:02.360
Link in the description.

s16
00:01:02.360 --> 00:01:04.600
It runs every other Saturday.

s17
00:01:04.600 --> 00:01:19.920
It reads from a knowledge vault, creates a research brief, builds content plan, produces 12 content pieces, then runs verify passes, reviewer gates, deduplication, and finally saves the output as markdown files.

s18
00:01:19.920 --> 00:01:23.600
But here's the thing that matters for this talk, not the content.

s19
00:01:23.600 --> 00:01:27.440
What matters is that this system has seven handoffs.

s20
00:01:27.440 --> 00:01:40.160
Scheduler to command, command to research, research to content content plan to production production skill to verifier, verifier to reviewer, reviewer to the output folder.

s21
00:01:40.160 --> 00:01:45.360
Every single handoff is the place where the system can lie to you.

s22
00:01:45.360 --> 00:01:52.800
And if you're building the system alone, nobody catches the lies except you, usually after the damage is done.

s23
00:01:52.800 --> 00:01:56.640
Here's the pattern I want you to watch for in your own systems.

s24
00:01:56.640 --> 00:02:04.360
If you build agents independently, you will rebuild these five things roughly in this order.

s25
00:02:04.360 --> 00:02:09.280
You change a prompt or a skill, and something downstream breaks.

s26
00:02:09.280 --> 00:02:14.800
So, you build a way to test whether the output still matches the expected shape.

s27
00:02:14.800 --> 00:02:19.240
Congratulations, you have reinvented regression testing.

s28
00:02:19.240 --> 00:02:22.480
So, you set up a cron job or scheduled task.

s29
00:02:22.480 --> 00:02:27.160
One day, it silently fails, but you haven't noticed for a week.

s30
00:02:27.160 --> 00:02:29.400
So, you build alerts.

s31
00:02:29.400 --> 00:02:31.920
You just reinvented CI monitoring.

s32
00:02:31.920 --> 00:02:37.800
One skill changes its output schema, so three skills downstream break.

s33
00:02:37.800 --> 00:02:42.800
You decided to add a validation at the boundary because of it.

s34
00:02:42.800 --> 00:02:45.360
You just reinvented contract testing.

s35
00:02:45.360 --> 00:02:48.800
An artifact looks done, but it shouldn't ship.

s36
00:02:48.800 --> 00:02:52.959
So, you add a checkpoint before it goes to the ready folder.

s37
00:02:52.959 --> 00:02:55.640
You just reinvented staging environments.

s38
00:02:55.640 --> 00:03:04.360
Something goes wrong, but you cannot find out which prompt, which skill, or which handoff had the bad output.

s39
00:03:04.360 --> 00:03:07.000
So, you start logging everything.

s40
00:03:07.000 --> 00:03:09.800
You just reinvented audit trails.

s41
00:03:09.800 --> 00:03:20.560
The reason the title says worst version isn't because agents are software builds, it's because you end up needing the exact same operational guarantees.

s42
00:03:20.560 --> 00:03:25.720
However, the agent systems give you none of it by default.

s43
00:03:25.720 --> 00:03:28.760
So, you build them independently.

s44
00:03:28.760 --> 00:03:30.440
The worst version.

s45
00:03:30.440 --> 00:03:33.400
Without even realizing that you're building it.

s46
00:03:33.400 --> 00:03:39.400
The dangerous failure in an agent system is never a bad output.

s47
00:03:39.400 --> 00:03:42.080
A bad output is very easy to fix.

s48
00:03:42.080 --> 00:03:46.640
You glance at it, and immediately you can understand it's a bad output.

s49
00:03:46.640 --> 00:03:53.240
The dangerous failure is a polished artifact that looks great at a glance.

s50
00:03:53.240 --> 00:03:57.440
However, it will never pass your exit gates.

s51
00:03:57.440 --> 00:04:00.000
It uses the wrong voice pattern.

s52
00:04:00.000 --> 00:04:02.240
It makes an unverified claim.

s53
00:04:02.240 --> 00:04:04.320
It repeats an old angle.

s54
00:04:04.320 --> 00:04:09.280
It's missing required sections, and it gets leveled ready to publish anyway.

s55
00:04:09.280 --> 00:04:15.800
That is the agent equivalent of shipping because the code compiled, but the tests never run.

s56
00:04:15.800 --> 00:04:17.760
This is what I'm going to demo for you.

s57
00:04:17.760 --> 00:04:24.280
Not the happy path, but the three ways the agent can lie to you and the gates to catch it.

s58
00:04:24.280 --> 00:04:29.760
Let me start with the happy path because this is what all agent demos show you.

s59
00:04:29.760 --> 00:04:34.840
I'm going to run a small privacy-safe version of my content engine pipeline.

s60
00:04:34.840 --> 00:04:36.480
This isn't the full repo.

s61
00:04:36.480 --> 00:04:39.600
It's a distilled version of the production problem.

s62
00:04:39.600 --> 00:04:41.120
The pipeline is simple.

s63
00:04:41.120 --> 00:04:46.640
Generate a content artifact, run gates, and either allow or block the output.

s64
00:04:46.640 --> 00:04:48.320
Look at the output.

s65
00:04:48.320 --> 00:04:59.720
The markdown has a caption, pinned comment, visual brief, verification log, vault assets, production notes, and a ready status.

s66
00:04:59.720 --> 00:05:03.520
If I demo only this, the system looks done.

s67
00:05:03.520 --> 00:05:06.080
The artifact looks professional.

s68
00:05:06.080 --> 00:05:08.160
The content reads well.

s69
00:05:08.160 --> 00:05:10.720
This is why agent demos are misleading.

s70
00:05:10.720 --> 00:05:13.280
They always show you the happy path.

s71
00:05:13.280 --> 00:05:18.440
But what happens if the path is not happy, but the output still looks ready?

s72
00:05:18.440 --> 00:05:22.240
Now, let's look at the failure number one, voice drift.

s73
00:05:22.240 --> 00:05:23.960
Look at the content.

s74
00:05:23.960 --> 00:05:26.280
Unlock the power of AI adoption.

s75
00:05:26.280 --> 00:05:33.919
This game-changer will transform how teams operate in today's fast-paced enterprise landscape.

s76
00:05:33.919 --> 00:05:41.919
If you have spent enough time on LinkedIn, you have seen this exact same sentence thousands of times.

s77
00:05:41.919 --> 00:05:45.040
It's generic AI marketing language.

s78
00:05:45.040 --> 00:05:46.800
It's not my voice.

s79
00:05:46.800 --> 00:05:48.280
It's not your voice.

s80
00:05:48.280 --> 00:05:50.160
It's nobody's voice.

s81
00:05:50.160 --> 00:05:53.480
But, knife mode saved it anyway.

s82
00:05:53.480 --> 00:05:56.960
Because the artifact has all the required sections.

s83
00:05:56.960 --> 00:06:00.800
It has a ready status, and it looks complete.

s84
00:06:00.800 --> 00:06:04.120
Now, what what happens when I add one boundary?

s85
00:06:04.120 --> 00:06:07.440
Guarded mode blocks it at the voice contract.

s86
00:06:07.440 --> 00:06:12.560
The pipeline stops before this artifact enters the publish ready folder.

s87
00:06:12.560 --> 00:06:13.640
This is the point.

s88
00:06:13.640 --> 00:06:16.440
The gate doesn't make the content better.

s89
00:06:16.440 --> 00:06:20.800
If you're building a content system, this is the first gate I would recommend.

s90
00:06:20.800 --> 00:06:28.880
If you're building any other agent system, the equivalent question is, what does wrong voice looks like in your domain?

s91
00:06:28.880 --> 00:06:31.760
Failure number two is missing verification.

s92
00:06:31.760 --> 00:06:40.240
This piece says, "Teams with a clear semantic ownership model reduce AI rollout rework by 37%."

s93
00:06:40.240 --> 00:06:42.040
37%.

s94
00:06:42.040 --> 00:06:44.600
This is a very specific claim.

s95
00:06:44.600 --> 00:06:46.400
Where did it come from?

s96
00:06:46.400 --> 00:06:48.160
Check the verification log.

s97
00:06:48.160 --> 00:06:49.440
It's empty.

s98
00:06:49.440 --> 00:06:51.480
The prose is usable.

s99
00:06:51.480 --> 00:06:54.040
The number sounds plausible.

s100
00:06:54.040 --> 00:06:56.960
That's what makes the failure dangerous.

s101
00:06:56.960 --> 00:07:02.000
A confident sounding claim without any verification or reference.

s102
00:07:02.000 --> 00:07:04.120
And knife mode saved it.

s103
00:07:04.120 --> 00:07:05.919
Guarded mode blocks it.

s104
00:07:05.919 --> 00:07:10.760
Claim bearing content cannot ship without a verification trail.

s105
00:07:10.760 --> 00:07:12.800
"Trust me" is not a verifier.

s106
00:07:12.800 --> 00:07:25.520
If your agent system makes claims about data, about users, about anything, and you don't have a validation chain, you're shipping unverified assertions with a professional looking wrapper.

s107
00:07:25.520 --> 00:07:27.360
That's not an agent problem.

s108
00:07:27.360 --> 00:07:29.240
That's a credibility problem.

s109
00:07:29.240 --> 00:07:31.919
Failure three, duplication hook.

s110
00:07:31.919 --> 00:07:37.040
This is my favorite failure, because this is the most realistic one for solo builders.

s111
00:07:37.040 --> 00:07:38.200
The output is new.

s112
00:07:38.200 --> 00:07:41.320
The content looks technically coherent.

s113
00:07:41.320 --> 00:07:47.600
But, the hook, the opening angle, is a near duplicate of something from your vault history.

s114
00:07:47.600 --> 00:07:52.720
Yeah, adoption fails when the dashboards looks right, but the workflow is wrong.

s115
00:07:52.720 --> 00:07:55.080
That angle has already been used.

s116
00:07:55.080 --> 00:08:04.120
If your system keeps generating near duplicates, your system looks automated, even if every individual piece is technically fine.

s117
00:08:04.120 --> 00:08:07.040
Your audience notices before you do.

s118
00:08:07.040 --> 00:08:10.600
Garden mode blocks it at the data contract.

s119
00:08:10.600 --> 00:08:12.120
I noticed this.

s120
00:08:12.120 --> 00:08:15.400
It also wrote an audit record.

s121
00:08:15.400 --> 00:08:23.440
That audit record is boring, but when a scheduled run fails at 2:00 a.m., the final artifact alone is not enough.

s122
00:08:23.440 --> 00:08:25.919
You need to know which gate failed.

s123
00:08:25.919 --> 00:08:28.880
Which contract was violated and why?

s124
00:08:28.880 --> 00:08:31.880
That's the audit trail, the fifth reinvention.

s125
00:08:31.880 --> 00:08:38.840
And that's the one most solid builders had lost after the damage has already been done.

s126
00:08:38.840 --> 00:08:41.080
So, what's the pattern here?

s127
00:08:41.080 --> 00:08:43.000
You don't need a platform.

s128
00:08:43.000 --> 00:08:44.839
You don't need a framework.

s129
00:08:44.839 --> 00:08:47.680
You don't need the ecosystem to catch up.

s130
00:08:47.680 --> 00:08:50.400
What you need are a few boring gates.

s131
00:08:50.400 --> 00:08:52.640
A pre-save output contract.

s132
00:08:52.640 --> 00:08:56.440
Does the artifact have the required shape before it's saved?

s133
00:08:56.440 --> 00:08:58.680
A voice or domain contract.

s134
00:08:58.680 --> 00:09:03.760
Does the output match the rules your system was designed around?

s135
00:09:08.000 --> 00:09:09.120
designed around?

s136
00:09:09.120 --> 00:09:10.920
A verification contract.

s137
00:09:10.920 --> 00:09:15.560
If the output makes claims, can those claims be traced to a source?

s138
00:09:15.560 --> 00:09:17.640
A deduplication check.

s139
00:09:17.640 --> 00:09:19.480
Is this genuinely new?

s140
00:09:19.480 --> 00:09:22.080
Or is the system recycling itself?

s141
00:09:22.080 --> 00:09:23.360
An audit trail.

s142
00:09:23.360 --> 00:09:29.040
When something fails, can you reconstruct what happened without rewriting the entire pipeline?

s143
00:09:29.040 --> 00:09:34.839
In software, we learned not to deploy only because code exists.

s144
00:09:34.839 --> 00:09:41.640
In agent systems, we need to learn not to ship just because the artifacts look complete.

s145
00:09:41.640 --> 00:09:44.520
The problem is not your agent will fail.

s146
00:09:44.520 --> 00:09:46.120
Your agent will fail.

s147
00:09:46.120 --> 00:09:52.560
The problem is when your system farmers that failure nicely and ships it downstream.

s148
00:09:52.560 --> 00:09:54.720
Here's what I want you to take away.

s149
00:09:54.720 --> 00:09:59.040
Map your handoffs from input to the final output.

s150
00:09:59.040 --> 00:10:04.480
Every arrow between two steps can be a place where the output gets corrupted.

s151
00:10:04.480 --> 00:10:06.600
You don't have to fix all of them.

s152
00:10:06.600 --> 00:10:08.800
You just have to know where they are.

s153
00:10:08.800 --> 00:10:11.200
Pick the most expensive handoff.

s154
00:10:11.200 --> 00:10:14.280
Not the most complex, most expensive.

s155
00:10:14.280 --> 00:10:18.200
The one where bad data can cost you the most.

s156
00:10:18.200 --> 00:10:28.400
A wrong claim published publicly, a broken schema that cascades to three downstream skills, a duplicate that errors your audience's trust.

s157
00:10:28.400 --> 00:10:30.560
That's where your first gate goes.

s158
00:10:30.560 --> 00:10:36.840
Make it say no. A gate which logs only warnings is not a gate.

s159
00:10:36.840 --> 00:10:37.920
It's a suggestion.

s160
00:10:37.920 --> 00:10:41.720
The gate needs to block the artifact from moving forward.

s161
00:10:41.720 --> 00:10:46.040
That's the difference between an impressive demo and an operable system.

s162
00:10:46.040 --> 00:10:50.000
Before you add another agent, add one boundary.

s163
00:10:50.000 --> 00:10:52.480
Thank you for watching.