Stop Writing Tone Instructions. Layer Them. - Isadora Martin-Dye, Isadora & Co
https://www.youtube.com/watch?v=ij-AU9dpJjc
AI Engineer · 20 min · 316 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:00] Hi, I'm Isadora.
[00:01] I own and run a 225-year-old wedding venue in Virginia.
[00:05] I also built an AI agent that talks to my couples.
[00:08] And then I built it for other venues, a personal AI companion app, and a public utility for families and missing people.
[00:15] I want to be clear up front about how I think about this work because it does change everything that follows.
[00:22] I'm not programming a robot, I'm managing a brilliant intern with an incredibly high IQ and a terrible EQ.
[00:30] They have photographic memory for whatever I've told them on the first morning and absolutely no instinct for when to read the room.
[00:37] They will say something technically perfect and socially catastrophic in the same confident sentence.
[00:43] That framing matters because it changes what you build.
[00:46] If you're programming a robot, you write rules and walk away.
[00:49] If you're managing an intern, you build structure and you check their work before it goes out the door.
[00:54] This talk is about that structure.
[00:56] The standard advice is write a detailed system prompt.
[01:01] Describe your brand's voice, give examples, and that does work for a while.
[01:05] It works for what I call the happy path.
[01:08] The happy path is every question that you have anticipated.
[01:12] You've given it examples, but turn 21 is the first one that the example didn't work.
[01:18] So, on turn 21, the model does something technically correct that your brand would just never say.
[01:25] It's not wrong exactly, but it's not you.
[01:28] This matters most where the voice is the product.
[01:31] Not for a product search on a retail site, but for a luxury hotel that spent 30 years building a specific relationship.
[01:38] A high-end real estate firm or in my case, a wedding venue.
[01:42] Places where a single wrong sentence can cost more than a refund.
[01:47] And the users are exactly the kind of people who notice.
[01:50] They're paying for a relationship and treating them like they won't is always going to backfire.
[01:55] Write in our brand's voice is a comment that says, "Just make it work."
[02:00] It does nothing that the model wasn't already going to try and do.
[02:04] And the reason it keeps failing isn't that the examples are bad, it's that you're asking one prompt to do four completely different jobs.
[02:12] It's really hard for one lad to do all four.
[02:15] The architecture I landed on after watching my brand fail, and the voice deliver inaccurate and not brand specific answers is four lad.
[02:27] Lad one is the immutable identity.
[02:30] The brand structurally cannot say these things.
[02:32] These are hard rules.
[02:34] They cannot be overwritten by anything below it, not by venue config, not by user instruction, not by anything.
[02:41] Lad two is the situational mode.
[02:43] It's what shifts when the user's state shift.
[02:46] Who are they?
[02:47] What are they going through right now?
[02:48] And the real-time conditions.
[02:50] Lad three is the exampled anchored voice.
[02:53] It's the warmth, the phrases, the dials, the tone guide.
[02:57] It's where most teams start and stop.
[03:00] Then lad four is the post-generation veto.
[03:03] It's the cheap final pass that catches what the other three miss.
[03:07] The reason one layer approaches fail is that the single system prompt can't simultaneously be situational, expressive, and self-checking.
[03:16] So, it handles the middle layer or two reasonably well, but falls apart in the edges.
[03:20] Before this architecture, my system had 24 different system prompts scattered across the code base.
[03:26] Half dozen named Sage, someone named Less, someone named Venue.
[03:30] Every surface had its own idea as to who it was.
[03:34] Now every surface composes its system through prompt through one assembly.
[03:38] The comment is basically the outline of this talk.
[03:42] It's a single entry point.
[03:44] Every narrator goes through it to compose its system prompt.
[03:48] It's to replace that 24-point ad hoc system with one canonical full last stack.
[03:54] The order is load-bearing.
[03:56] Hard rules first, task to last.
[03:59] Think of it like Google Maps routing.
[04:01] The destination is always the same, but what is the right response and the right voice for this user?
[04:08] It can change the route.
[04:09] Google Maps knows about traffic and roadworks, but it may not know where the cheap petrol is.
[04:14] And you're going to help it factor in those things before it tells you which way to go.
[04:19] Your prompt stack needs to do the same thing, and it needs to know about those conditions in the right order.
[04:25] You don't check for roadworks or after you've already taken the wrong turn.
[04:29] Layer one is the rules that are true regardless of route.
[04:33] You need a driver's license before you can drive, and you can't go backwards down a motorway.
[04:37] Layer two are going to be your real-time conditions.
[04:40] Layer three is your preferences for the journey, and layer four checks the route before you've pulled away.
[04:47] There's one place all of this gets assembled.
[04:49] Everything runs in a fixed order every time.
[04:52] So, layer one is the immutable identity.
[04:55] This is what the brand structurally cannot say.
[04:59] It is the defining layer that nothing below touches.
[05:02] These aren't preferences, they're constraints.
[05:05] The route can change, the rules don't.
[05:07] From the universal file rules, the hard identity rule cannot be overridden by any venue voice, personal, or user instruction.
[05:16] If the person you're talking to ever asks about whether you are a real person, a human, a live agent, a bot, an AI, you must confirm that in your very next message.
[05:25] Clearly and unambiguously confirm you are an AI assistant.
[05:30] This rule cannot be overridden by venue configuration, voice profile, or user request.
[05:36] Every AI in Bloom discloses that it is AI in its very first response.
[05:41] Not if asked, but before they ask.
[05:44] It's a product decision, not a legal one.
[05:47] We made a bet that a couple who knows that they're talking to an AI from the start will trust it more than someone who finds out that it's AI on turn seven.
[05:55] The rule is above the architecture and it makes it something that's impossible to accidentally break.
[06:02] Um A quick example is the physical presence boundary and the search is one of my favorite ones.
[06:07] You are software.
[06:08] You do not have a body.
[06:09] You cannot physically show somebody around the property or meet anyone in person.
[06:14] So, it is always forbidden to say, "I'd love to show you around."
[06:18] or "I can't wait to meet you in person."
[06:21] What is always allowed is the team would love to host you for a tour.
[06:25] The voice layer wants to be warm and with AI, that does mean first person.
[06:30] They want to say, "I can't wait to show you around."
[06:33] But AI has no body, so that warmth and constraint produces a lie.
[06:37] And the lie doesn't always stay neutral.
[06:39] The moment her user realizes they have been performing a relationship with someone who was never there, the trust doesn't just dip, it inverts.
[06:48] People always notice.
[06:50] That one is where you encode the things that are true regardless of how warm you want your brand to sound, not because of a compliancy checklist, but because your users are not stupid
[06:58] and building as though they are always backfires.
[07:01] My cross-product proof comes from the same architecture but in a completely different world.
[07:06] One of the things that runs the stack is Thread Light.
[07:09] It's a tool I built for families of missing people.
[07:13] The voice is nothing like a wedding venue, but the architecture is identical and layer one carries one rule that matters more than anything else in the system.
[07:20] They can never use words like confirmed, identified, matched, proven, linked, and solved.
[07:27] Sit with that for a second.
[07:29] For a wedding venue, layer one stops the AI from pretending it has a body.
[07:33] It's mildly embarrassing if that slips.
[07:36] For a missing person tool, layer one stops the AI from ever telling a person that their person has been found.
[07:43] And what the system has is just problem problematic.
[07:46] The word match said to someone who has spent years not knowing where their child is is not just a tone violation.
[07:52] It is the single most damaging thing that a product could ever do.
[07:56] And the model has no idea.
[07:57] It's reaching for the word match because statistically it is the natural word.
[08:02] But it's going to reach for it with the same level of confidence.
[08:05] And it cannot bring that level of confidence to someone who is grieving.
[08:09] It's the same architecture but wildly different stakes.
[08:13] The point was never specific rules.
[08:15] The point is that things your brand can say have to live in a layer that the voice, however warm, well trained, however confident, physically cannot use.
[08:23] Layer two is your situational mode.
[08:25] Real time conditions and what changes the route.
[08:29] This is the layer that most teams never build at all.
[08:31] They write one system prompt and send it to everyone regardless of who that person is or what they're going through.
[08:37] Google Maps doesn't do that.
[08:39] It's going to know if there's accident on your route.
[08:42] It might not know if you're low on fuel.
[08:45] It might learn that you prefer the scenic route.
[08:47] But it's going to factor all these different things in before the route's not after once you tell it.
[08:52] Layer two is those real time signals built into the prompt before it runs.
[08:58] So, condition one is going to be to adjust to who you're talking to.
[09:03] The same AI that talks to couples also briefs the venue staff.
[09:07] Same destination.
[09:09] It's going to give the right answer but it's two completely different routes.
[09:12] For the corn walk from the coordinator rules, it is going to talk to them like a colleague, not a customer.
[09:19] You are in the same character that the that the couple interacts with.
[09:23] So, you mustn't fall into a generic intelligence analysis framing.
[09:28] But it flips depending on the audience.
[09:31] So, a coordinator might ask, will inquiries be up in June?
[09:36] And it should hear, I can't forecast that confidently.
[09:38] Here is the trend.
[09:39] This is what you might see.
[09:41] A couple should never be refused like that.
[09:43] Same identity, same voice, but the route changes based on who's in the car.
[09:48] Condition two is what they're going through.
[09:51] The second real-time signal is to know about this specific person situation.
[09:56] Not their role, but their life.
[09:58] And it comes from universal rules.
[10:00] Your soft context notes policy.
[10:03] Use these notes for tone, empathy, and what not to say.
[10:07] Never quote them verbatim.
[10:09] That's really important, too.
[10:10] A couple munching in grief should hear gentleness, not a quote about loss.
[10:15] A couple navigating a sick parent should get patience and slack on timing, never a sentence that's just talking about the illness.
[10:23] The assembler deliberately renders this before the numbers.
[10:26] Tone is set by human context first, then numeric constraints.
[10:30] The code comment explains the reasoning.
[10:32] Couple note block render before the numbers guard block.
[10:36] So, the LLM uh sets tone first from the soft context, then satisfies numeric content constraints.
[10:43] Reversing the order makes the prose feel mechanically slotted because the model is already committed to the numeric framing before it reads the qualitative tone fuel.
[10:52] The best single example of why this matters is a real comment in the actual code base.
[10:57] Um I have a heat map about how often we are hearing from couples.
[11:03] If a couple drops down that heat map, it will change how it reacts to that based on what it knows.
[11:11] So, if it knows a client has a mom in chemo for 3 weeks, it is going to narrate that very differently than a heat drop with soft context or no context at all.
[11:20] This is the opposite of the lie problem.
[11:22] Layer one is what the AI must never pretend.
[11:26] Layer two is about what the AI already knows and letting that shape its behavior honestly rather than driving past the roadworks if they're not there.
[11:34] The voice doesn't change.
[11:35] What changes is the read.
[11:36] When their engagement drops and you know that chemo, that reads as a family under strain, not a cold lead or a problematic couple to chase.
[11:45] Layer three is the example that good voice.
[11:48] It is the tone guide and it is why most teams start and stop.
[11:52] For most engineering teams, the work ends here because it feels like a brand problem, not a technical problem.
[11:58] Someone in marketing owns the tone guide.
[12:00] It's given to them.
[12:01] The engineer wires it in and job done.
[12:04] It's the dials, the phrase list.
[12:06] If we're keeping the intern analogy, it's the induction pack.
[12:09] The folder of good examples you hand them on day one and say sound like this.
[12:13] The induction pack is fixed.
[12:15] It's Although it's before the intern has ever met anyone, it doesn't know who walked in the door this morning, it has no context.
[12:22] It has a lot of really positive things about it.
[12:25] Writing in our brand voice, it is a really good training exercise.
[12:29] But cannot enforce a rule that came out.
[12:31] That's layer one.
[12:32] It cannot respond to who is this person and what they're going through.
[12:36] That's layer two.
[12:37] And it cannot catch the model producing something that it shouldn't have done.
[12:41] That would be layer four.
[12:43] Examples teach the model what good looks like on the happy path.
[12:47] On turn 21 where the user is asked the things that examples never cover, the phrase list has nothing to say.
[12:54] It's not a failure of the examples, it's a catch-all error.
[12:57] Examples are not the right tool for guarantees, they were never designed to be.
[13:01] In Bloom, we brought in a voice training exercise that the person dialed in the brand voice.
[13:08] And very importantly, it can actually be trained from the coordinator writing actual AI responses and the AI learns from the edits that the coordinators have made.
[13:17] Layer four is the post generation veto.
[13:20] The only layer that actually reads what came out.
[13:23] You wouldn't let the new intern send a client email blind.
[13:27] Someone's going to read it first.
[13:28] And layer four is that read.
[13:30] It's automated, it's cheap, and it's the only part of the whole architecture that isn't a prompt.
[13:35] The first three layers are all instructions.
[13:37] The ins- The instructions are a request.
[13:39] This layer is the only one that looks at what was actually produced and has the power to say no. There are two types of vetoes.
[13:47] The soft flag.
[13:48] The honesty inspector runs after generation and flags a response that slipped through a rail.
[13:53] One of the most important examples of that is did it actually answer the question?
[13:58] It should not respond.
[13:59] It should not hedge.
[14:00] A false positive means someone double-checked a phone response.
[14:04] A false negative remains a hallucinated number or a privacy violation that ships to a client.
[14:09] This asymmetry is obvious once you say it out aloud, but it's often not written in as its own layer.
[14:15] The hard reject.
[14:17] The numbers guide us for the expensive failures.
[14:19] The model confidently stating a figure that was never given.
[14:22] The prompt asking it not to invert numbers.
[14:25] The guard rejects the output anyway.
[14:28] Let me tell you that this layer exists, but it wasn't in the original design.
[14:31] I added it in because of a specific failure that kept happening and it's one of the most ordinary failures in the world.
[14:37] The AI would keep offering dates to my clients that did not exist.
[14:41] A couple writes in excited about a Saturday in October.
[14:45] The model wants to be warm.
[14:46] Layer three is doing its job.
[14:48] So, it says something lovely about how beautiful the property is at the time of year and how it would love to hold the date for them.
[14:54] Except that that date is booked.
[14:56] The model didn't know that.
[14:57] It was never given the calendar.
[14:59] It was researched for an encouraging specific confident answer because it knew that what good service sounded like.
[15:06] And it was confident that these models produce something, but it didn't actually know the thing.
[15:12] And here is the thing.
[15:13] Every layer above did its job.
[15:15] The identity rules held.
[15:17] The mode was right.
[15:19] The voice was perfect.
[15:21] But, the voice was also the problem.
[15:23] A warm, confident voice offering something that isn't real is worse than a cold one because the couple now believes they have a date.
[15:30] You haven't given them good service, you've given them a disappointment with a 48-hour delay on it.
[15:35] This is where I understood that the whole talk is really what the whole talk is really about.
[15:40] The first three layers are all probabilistic.
[15:43] There are instructions to a system that usually follows instructions.
[15:47] Usually is completely fine when the cost of being wrong is a slightly off-brand sentence.
[15:52] It is not fine when the model is confidently inventing a fact that a real person is about to act on.
[15:58] A date, a price, a policy, a promise.
[16:00] So the veto is actually the cheapest layer to build and it's the only one that's deterministic.
[16:05] It reads what the model actually wrote, not just what you asked it to write.
[16:09] And it checks the specifics against what is real.
[16:12] A date the model offered that isn't in the allow list doesn't ship.
[16:16] Prevention is the prompt.
[16:18] If the veto is the check, you need both because the prompt will eventually lose and you don't want to find out that it lost by reading a couple's reply.
[16:27] It's multi-tenant.
[16:28] It's one architecture with completely different voices.
[16:31] The seam is calling one function.
[16:34] Layer one is identical for every tenant.
[16:36] Layer two and three are per venue.
[16:38] This is how code base serves venues with completely different personalities or different things like the ground and thread line without forking.
[16:46] They're the same root logic, different preferences and conditions loaded per driver.
[16:51] There's a specific failure in multi-tenant voice worth naming because it's subtle, but it really hurts when it happens.
[16:57] From the personality builder, important brand identity fields are not defaulted here.
[17:03] Letting them default silently caused every venue to ship as sage@hawthornemanner.com, which was a critical white label leak.
[17:11] The brand identity must come from the venue AI config.
[17:15] If it is missing, call is thrown.
[17:18] Every venue that shipped a sage emailing from another venue was addressed.
[17:22] In Google Maps terms, every driver was getting the same saved home address regardless of where they actually lived.
[17:29] The fix is a principle.
[17:31] In a multi-tenant system, identity must never have a default.
[17:35] A missing brand identity is a crash, not a fallback.
[17:37] It must fail loud because the quiet failure is a venue speaking in a stranger's voice, and the user on the receiving end has no idea why something feels off.
[17:47] They just know it does, and the trust erodes before anyone on your team even knows there's a problem.
[17:52] If you take one thing from this, take this.
[17:54] The first three layers are all instructions.
[17:57] Identity, conditions, and voice.
[17:59] They're all things you tell the model, and the model usually listens.
[18:02] Usually, they are a request.
[18:05] The fourth layer is not a request.
[18:07] It is reading what actually came out and decides whether to allow it to leave your business.
[18:12] The first three layers are instruction, the fourth is permission.
[18:16] And that's the whole distinction.
[18:17] Instructions are probabilistic.
[18:20] Permission is deterministic.
[18:23] Everything before layer four is prompt engineering.
[18:25] You're asking nicely and hoping.
[18:28] Layer four is systems engineering.
[18:30] You're checking, and you are sure.
[18:32] It was never really about brand voice.
[18:34] It's what happens when you ask one mechanism to do four fundamentally different jobs.
[18:39] Be situational, be inviolable, be expressive, and check itself.
[18:44] And then act surprised when it can't.
[18:47] Pull those jobs apart, give each one a layer you built for it, and the thing that it the thing that you used to break on on turn 21 doesn't.
[18:56] So, users aren't testing your brand voice, they are trusting it.
[18:59] The moment you treat that trust as a prompt engineering problem, something you solve once and ship, you've already lost the thing you were trying to protect.
[19:07] The four-layer pattern isn't a framework I'm trying to sell.
[19:10] It is what happens when a system prompt fails enough times and it will fail.
[19:15] A prompt will eventually lose.
[19:17] The only question is is whether you in that about that infra testing or in front of a customer.
[19:22] I have learned a few things that I would consider building differently.
[19:25] And we'll probably go back and edit.
[19:28] Um the veto should be its own service not wired individually into each surface.
[19:33] Right now the numbers God lives inside a heat narration path and the honesty inspector is a separate function.
[19:40] A new service remembers to wire in the veto manually.
[19:44] Uh that's a checklist I waiting to be forgotten and making it a shared gate that everything passes through by default means you can't opt out by accident.
[19:53] Latent mode detection is still partly manual.
[19:57] Right now each surface decides which conditions apply.
[20:00] The heat narration surface knows to load couples notes.
[20:04] The briefing surface knows not to.
[20:06] That knowledge is scattered and a proper condition resolver would make that decision explicit and central.
[20:12] Um one place to look at the context and says, "This is who the user is.
[20:16] This is what they're going through.
[20:17] This is the route."
[20:19] And the soft flag is rejects not a model.
[20:22] Um rejects is fast and cheap and deterministic.
[20:25] It either matches or it doesn't.
[20:27] Um and it never gets wrong on the patterns it covers.
[20:31] A small classifier might catch more edge cases including the ones I haven't thought to write a pattern for yet, but it is a classifier that is probabilistic.
[20:39] Sometimes it gets it wrong.
[20:41] And right now I'm choosing determinism over coverage.
[20:44] I'd make that choice again as it stands, but it is a real trade-off and not an obvious win.
[20:49] That is something you might need to determine for yourself.
[20:52] Um please let me know if there's any other questions.
[20:54] Um and I hopefully it's been helpful.