HTML is All You Need (for Agents to Make Graphics) - Amol Kapoor, Nori
https://www.youtube.com/watch?v=JRTAtZ5iBkU
AI Engineer · 7 min · 115 sentences · sentence-level transcript from YouTube's caption track; each line begins at the time shown

[00:07] [music]
[00:10] Hi, I'm Amol, CEO of Nori Atentic.
[00:14] [music]
[00:14] We deploy an AI employee that understands your company, your code, docs, Slack,
[00:20] [music]
[00:21] and other kinds of data.
[00:23] We spend a lot of time thinking about how coding agents really work.
[00:28] Most people think coding agents only write code.
[00:31] But if you ask me, that's just bad marketing.
[00:33] [music]
[00:33] Forget the name for a second.
[00:35] Coding agents can do almost anything.
[00:38] There's just one trick.
[00:39] You have to be able to think like an agent to get it to do what you want it to do.
[00:43] [music]
[00:44] Today, we're going to talk about how we use coding agents to do something most people think agents are terrible at, make visual artifacts, like slides, docs,
[00:53] [music]
[00:53] and yeah, even video.
[00:58] [music]
[00:59] Every day, the world pours something like 34,000 human years into making slide decks.
[01:05] Most of that time isn't the thinking, it's the fiddling.
[01:08] [music]
[01:09] A deck that takes 10 hours should really take about 25 minutes once you remove all the formatting and the branding and the moving things around.
[01:18] Say you need to make a slide.
[01:20] What do you do?
[01:21] You open a tool, PowerPoint, Slides, Figma, Canva, and then you start manipulating a canvas.
[01:29] Every one of these tools is built for human hands and human eyes.
[01:32] Click, drag, drop, resize, snap to grid, all motions and patterns that make sense for our geospatial view of the world.
[01:41] There is a data structure underneath, but it's in a format that only the application can read.
[01:47] What happens when you hand these tools to an agent?
[01:49] Well, the output comes out all wrong.
[01:52] Things overlap in weird ways, you can't see the text, there's no alignment, It's just garbage.
[01:58] AI skeptics say that it's not just the tools.
[02:01] Agents fundamentally can't reason about space.
[02:04] And there are whole benchmarks like Arc AGI that are built exactly around that premise.
[02:10] There's a famous little test for this from developer Simon Willison.
[02:14] He asks every new model the same thing.
[02:17] Can you draw a pelican riding a bicycle?
[02:20] But there's a trick.
[02:21] The agent is only allowed to use SVG.
[02:25] It's a quick gut check for whether a model can reason about space at all.
[02:29] Here's some examples of what the models actually give you on this test.
[02:32] And
[02:32] [music]
[02:33] yeah, these are pretty bad.
[02:35] Like genuinely deeply really bad.
[02:38] [music]
[02:39] So, does that mean it's hopeless?
[02:41] Agents are just doomed to be bad at graphics?
[02:43] No, I don't think so.
[02:45] If you ask me, it's not the model, it's the medium.
[02:49] If I asked you, someone who is presumably human, to handwrite an SVG of a pelican, you wouldn't be able to do that either.
[02:57] [music]
[02:57] SVGs are just a wall of numbers.
[03:00] You can't go from a wall of numbers to a pelican, you just can't see that way.
[03:05] That's just not how people think.
[03:07] We think graphically.
[03:09] So, we build tools that let us draw on a canvas.
[03:12] Figma MCPs, PowerPoint CLIs, screenshot and replace loops, what do all of these agent tools have in common?
[03:19] They all approach the problem like a human.
[03:23] But an AI is not a human.
[03:25] Asking an AI to use a canvas is like asking a human to write SVG by hand.
[03:29] It doesn't really make sense.
[03:31] You need to give the AI tools based on how it thinks, not in pixels, in language.
[03:37] Words, tokens, structure, that is its native medium.
[03:42] Imagine a language that's incredible at describing layout.
[03:46] That models have seen and trained on billions of examples of.
[03:50] That they understand intuitively.
[03:52] that renders to pixels and can run everywhere.
[03:57] Oh, right.
[03:59] HTML lets the model think in structure.
[04:02] HTML tags have meanings built into the language.
[04:05] A heading, a chart, a grid, and the browser turns it all into pixels.
[04:09] So, the model never actually places a coordinate, and you can get all sorts of visual effects, charts and layouts, fonts and motion, all of it for free.
[04:19] Remember that pelican from earlier?
[04:21] Now, ask it to do the same exact task, but in HTML.
[04:25] Same bird, but now it's in a structure that the model can reason about.
[04:29] And you can read and theme and edit every single line of it.
[04:34] I spent my whole life building slide decks with PowerPoint.
[04:37] So, I always thought that those two things, slide decks and PowerPoint, were synonyms.
[04:41] [music]
[04:42] But that's just not really true, is it?
[04:44] PowerPoint is a tool that you use to make slide decks.
[04:49] The deck itself, that's just the presentation mode.
[04:52] And as it turns out, no one in your audience is going to care how you got to the presentation mode.
[04:57] The editing format is totally arbitrary.
[05:00] [music]
[05:00] So, you can just pick the editing format that the agents are already good at, HTML, and if you need to render to a different format like PDF later on.
[05:11] We use this HTML trick to build all of our slide decks, our board decks, and our sales decks.
[05:16] [music]
[05:16] These are real things that we actually present and send out constantly.
[05:20] We use it for our docs, too.
[05:22] It gives our docs color and vibrancy, all while following our brand.
[05:26] And of course, we also use it to make videos like this one.
[05:31] What you're watching is just HTML and CSS.
[05:34] It's literally just divs all the way down.
[05:40] Almost everything is better with a little structure and a little bit of color.
[05:44] Plain text is a choice, generally a choice of convenience, but it's usually the wrong one if you're actually trying to create something of use.
[05:53] Now, I do want to take a quick beat here and point out that a beautiful deck on its own is generally not worth anything.
[06:00] You still have to go and get all of that content, all of the things that actually populate that deck, right?
[06:05] Well, again, we can think like the model.
[06:09] If you just give the model access to your data, say your call transcripts or your emails, you can have the model build the deck end to end.
[06:17] Let your agents do all the grunt work while you focus on vision and story.
[06:22] That's what Nori Sessions lets you do.
[06:24] I've built entire board decks for my phone on the subway during my commute.
[06:29] Why?
[06:30] Because our Nori bot lives in the fabric of our company.
[06:33] Of course, Nori ships with everything you need to make this all work, so don't bother reinventing the wheel.
[06:39] That's my little spiel.
[06:40] Thanks for listening.
[06:42] If you have just one takeaway, it's this.
[06:45] Stop thinking like a user.
[06:47] Think like the model.
[06:49] Give it the right language, and for graphics, all you need is HTML.
[06:54] [music]