WEBVTT NOTE Sentence-level transcript of https://www.youtube.com/watch?v=Cz4v1WHVyZc NOTE One cue per sentence. Cue ids are the line anchors on /transcripts/Cz4v1WHVyZc.html. A cue ends where the next begins, or 2 s after its last word. s1 00:00:01.309 --> 00:00:03.309 [music] s2 00:00:36.640 --> 00:00:39.920 Since we're talking about generative media, I thought starting with a video might be good. s3 00:00:39.920 --> 00:00:45.920 That video was made completely by an agent in a single shot, all utilizing HTML. s4 00:00:46.360 --> 00:00:47.600 My name is James Russo. s5 00:00:47.600 --> 00:00:51.040 I am the co-creator and tech lead of hyperframes at HeyGen. s6 00:00:51.040 --> 00:00:57.680 Today, we're going to be talking about why we think HTML is all your agents need to create great videos. s7 00:00:58.800 --> 00:01:06.480 Show of hands here who finds creating that launch post or that launch video harder than actually building now with coding agents. s8 00:01:06.560 --> 00:01:07.360 All right. s9 00:01:07.360 --> 00:01:08.480 Some hands up there. s10 00:01:08.480 --> 00:01:15.960 Um Coding agents have democratized building, made it incredibly easy for anyone to create anything that comes to their minds. s11 00:01:15.960 --> 00:01:22.040 Um however, we think that launching your product or your feature, getting it out into the world, is still quite hard. s12 00:01:22.040 --> 00:01:26.680 And we at HeyGen have been trying to close that gap. s13 00:01:27.240 --> 00:01:31.600 If you're not familiar with HeyGen, our mission is to solve communication through video. s14 00:01:31.600 --> 00:01:35.400 We started by creating the best AI avatar on the market. s15 00:01:35.400 --> 00:01:40.240 You can think of this as the A-roll, the footage you see here on the left side of the screen. s16 00:01:40.240 --> 00:01:43.400 Um it's the narrator, the character, the main subject of the video. s17 00:01:43.400 --> 00:01:48.920 However, it's kind of plain, and great videos have a lot more pieces to them. s18 00:01:48.920 --> 00:01:53.480 They have B-roll, which is the images and the the other pieces of media and assets. s19 00:01:53.480 --> 00:01:56.760 They have animations, they have captions, they have music. s20 00:01:56.760 --> 00:02:00.280 All of this is needed to create a great video. s21 00:02:00.280 --> 00:02:11.440 And for us, it's important that we nail every layer, not just the avatar, to solve communication through video and give agents the right canvas to create great videos. s22 00:02:13.440 --> 00:02:20.920 So, how do you give agents the ability to generate all of these layers and build up this composition needed to create great video? s23 00:02:20.920 --> 00:02:23.640 Our bet is on HTML. s24 00:02:23.640 --> 00:02:27.560 HTML, CSS, and JavaScript are the native languages of LLMs. s25 00:02:27.560 --> 00:02:34.480 Most of their training data, every webpage that gets scraped at the end of day is essentially just HTML, CSS, and JavaScript under the hood. s26 00:02:34.480 --> 00:02:42.640 Uh when you try to teach a model a new DSL or even your own custom JSON structure, it's forcing it to speak another language. s27 00:02:42.640 --> 00:02:48.840 Uh we like to think of this as trying to ask Shakespeare to write a poem in Japanese or Chinese. s28 00:02:48.840 --> 00:02:57.560 Even if you give them a bunch of examples um and teach them a bunch of things, you will not get the best output from them because it's not their native tongue. s29 00:02:57.560 --> 00:03:03.880 So, why not let the LLMs and agents talk in their native tongue when creating videos? s30 00:03:04.680 --> 00:03:06.200 We're not the only ones saying this. s31 00:03:06.200 --> 00:03:12.200 Over the last few months, Tarik, Andrej Karpathy have been talking a lot about how HTML is a new markdown. s32 00:03:12.200 --> 00:03:16.800 It's a great output um for LLMs to give you visual representation. s33 00:03:16.800 --> 00:03:20.440 For what's it's worth, I submitted this talk before both of these tweets came out. s34 00:03:20.440 --> 00:03:33.600 Uh I can show you proof later if you're curious, but we all kind of came to the same conclusion here, which is that uh let LLMs talk in their native language and you can get better output from them. s35 00:03:34.600 --> 00:03:36.480 It wasn't necessarily an easy thing. s36 00:03:36.480 --> 00:03:41.680 We've been working on this for over a year now and tried a bunch of things along the way. s37 00:03:41.680 --> 00:03:47.360 Uh there's a bunch of different uh frameworks and tools to help create great videos. s38 00:03:47.360 --> 00:03:53.200 And we like to think about them uh on this axis of quality of results and agent-friendly. s39 00:03:53.200 --> 00:03:58.760 Um for us, After Effects, Premiere Pro, these are kind of like the gold standards of creatives. s40 00:03:58.760 --> 00:04:02.280 They create great output, but they're not very agent-friendly. s41 00:04:02.280 --> 00:04:10.400 And even with the more recent connectors into it, it's more of a a co-pilot or something that can help you facilitate things you already know how to do. s42 00:04:10.400 --> 00:04:12.880 It doesn't give it creative output. s43 00:04:12.880 --> 00:04:20.200 We tried things like Lottie and Rive, which are coding uh languages in JSON or custom XML formats. s44 00:04:20.200 --> 00:04:36.760 Um and they can get you pretty good output, but agent-friendly isn't necessarily true for those because they're not um their native tongue and they aren't as editable or controllable, which is a big thing for us is that controllability layer. s45 00:04:36.760 --> 00:04:44.560 We played around with Remotion quite a bit and honestly thought it was a great example of what LLMs and agents could do with coding. s46 00:04:44.560 --> 00:04:48.520 Um however, we noticed that we had to teach them the framework. s47 00:04:48.520 --> 00:04:57.760 We had to teach them the language and give them a bunch of examples on how to write code properly, which ultimately took away a lot of the creativity of it. s48 00:04:57.760 --> 00:05:07.520 Um then we have HTML, which around November of last year when Gemini 3 and the latest models came out, we saw a step function improvement in what LLMs could do. s49 00:05:07.520 --> 00:05:16.720 When we just gave them examples of what the output we wanted was, they naturally gravitated toward HTML, CSS, and JavaScript and gave us great output. s50 00:05:16.720 --> 00:05:23.720 Um and we decided, let's not fight the model, but find a way where we can let them talk in their native tongue. s51 00:05:24.200 --> 00:05:31.800 So, how we did this was starting with a very small model, Gemini 3 Flash, as our design partner. s52 00:05:31.800 --> 00:05:42.400 We knew that if the smaller models could author workable code in a framework, then the larger models and these coding agents could 100% do it as well. s53 00:05:42.400 --> 00:05:52.440 We tried with a bunch of different wrappers around HTML, CSS, and JavaScript, adding in a lot of context, and making our system prompts bigger, adding in skills. s54 00:05:52.440 --> 00:06:05.920 Um but to our surprise, the thinnest wrapper ultimately won, which is essentially just HTML at the end of the day with a few data attributes as metadata to let the agent know, and to let us know uh about timing and things like that. s55 00:06:05.920 --> 00:06:08.840 Um our Yeah, as I already mentioned, our thinking was very simple. s56 00:06:08.840 --> 00:06:13.320 If this smaller model could get it, as the models got better, it would continue to improve. s57 00:06:13.320 --> 00:06:17.640 They would naturally understand it more as well as training data came into the picture. s58 00:06:17.640 --> 00:06:23.440 Um and this is when we knew the format of what we were building was right. s59 00:06:24.040 --> 00:06:29.720 Here's a little bit more detailed example of what a video looks like uh in hyperframe. s60 00:06:29.720 --> 00:06:37.640 So, here is on the left just HTML um with a few attributes, and on the right, the web page that it renders. s61 00:06:37.640 --> 00:06:40.680 Um the preview and the render are all done in the browser. s62 00:06:40.680 --> 00:06:45.200 The same pixels that the browser sees is ultimately what your video is going to see. s63 00:06:45.200 --> 00:06:53.200 Um and anything any web page that your LLM or agent knows how to write, it can now write into a video. s64 00:06:54.200 --> 00:07:01.000 This gave us hyperframes, our open-source framework that turns your agent's HTML into video. s65 00:07:03.440 --> 00:07:14.240 So, now that we had the HTML and the language part of it, um the next hard part was how do we actually turn this into a deterministic MP4 video that anyone can post or utilize. s66 00:07:14.240 --> 00:07:18.240 And this was a lot harder because browsers are async on purpose. s67 00:07:18.240 --> 00:07:22.280 Um they have a different set of requirements and concerns to video rendering. s68 00:07:22.280 --> 00:07:28.600 They need to work across a bunch of different networks and a bunch of different uh constraints. s69 00:07:28.600 --> 00:07:30.760 But, that's not true for video. s70 00:07:30.760 --> 00:07:34.560 So, on the left you can kind of see here what a a browser render might do. s71 00:07:34.560 --> 00:07:37.680 It's okay to loading things asynchronously like fonts. s72 00:07:37.680 --> 00:07:42.080 So, as the page loads in, the font and style of the text might change. s73 00:07:42.080 --> 00:07:46.960 Images and videos and other assets can load in asynchronously as well. s74 00:07:46.960 --> 00:07:50.920 Um so you might not see them initially on the page, but they'll come in at some point. s75 00:07:50.920 --> 00:08:00.400 Whereas for video, we need everything on the page 100% of the time so that we can show you exactly what you expect in the the video. s76 00:08:01.640 --> 00:08:08.040 How we do this in hyperframes is um essentially freezing the clock and seeking frame by frame. s77 00:08:08.040 --> 00:08:18.040 For those who aren't super familiar with video, uh basically a video is a series of frames or images strung together to create motion across an entire video. s78 00:08:18.040 --> 00:08:25.040 So we took this insight of ours and basically applied it to our rendering of HTML as well. s79 00:08:25.040 --> 00:08:39.039 We freeze the clock in the browser and then we seek deterministically to every single moment in time or every single frame, uh wait for everything to load on the page, ensure that it's loaded and ready to go, and then we take a screenshot s80 00:08:39.039 --> 00:08:46.360 and move on to the next frame, and do that over and over again uh until we get all of the necessary frames to encode that into a video. s81 00:08:46.360 --> 00:08:52.080 So the same input that is previewed in the browser is also rendered into the video. s82 00:08:52.080 --> 00:08:54.760 Hence the name hyperframes. s83 00:08:56.160 --> 00:09:01.160 The power of this is that anything you can render in a browser essentially can now be in your video. s84 00:09:01.160 --> 00:09:13.600 Things like 3.js, charts, SVGs, shaders, WebGL, WebGPU, Lottie, all of these are renderable in the browser, and therefore all of them are renderable in hyperframes. s85 00:09:13.600 --> 00:09:24.160 This is a big part of how we create a lot of our videos is finding inspiration on the internet, taking these as examples, tweaking them to our needs, and then putting them into our videos. s86 00:09:25.720 --> 00:09:33.640 The next part outside of the actual framework and rendering is how do you get great or good output out of the agents. s87 00:09:33.640 --> 00:09:37.360 And a big part of this is the skills that we couple with our framework. s88 00:09:37.360 --> 00:09:49.920 Our skill is focused on taste and video aspects because the LLMs and agents already know how to write HTML and CSS and JavaScript, we don't have to teach them the language, we just teach them how to create good videos. s89 00:09:49.920 --> 00:09:57.840 This is a big difference between other frameworks where if you look at certain skills, it's really just like how do you write something in that framework? s90 00:09:57.840 --> 00:10:07.200 Um this allows us to focus on the important parts of what makes a great video and a big part of this is constantly evaling and using agents to improve them. s91 00:10:07.200 --> 00:10:17.960 Uh this allows us to raise the floor of videos and ensure that the base output of a single shot prompt gets you pretty good results. s92 00:10:18.440 --> 00:10:24.440 Here's an example I'm going to show of a single shot utilizing our website to video skill. s93 00:10:24.440 --> 00:10:36.480 All you have to do is give it a website a We tell it how to get to that website, retrieve all the assets and information it needs uh to create that design and theming and branding using DesignMD s94 00:10:36.480 --> 00:10:38.560 or FrameMD in our case. s95 00:10:38.560 --> 00:10:50.680 Um and then from there we just teach the LLM basic motion examples and things like that of great videos that you can then go ahead and get good output like this. s96 00:10:54.520 --> 00:10:57.680 However, great output takes craft. s97 00:10:57.680 --> 00:11:13.600 Similar to AI coding, you can get decent output by just giving a single prompt and having something that works for your needs, but getting great output from agents requires craft, taste, the same principles of any uh software engineer before s98 00:11:13.600 --> 00:11:14.520 AI coding as well. s99 00:11:14.520 --> 00:11:18.280 Breaking the problem up into individual pieces and working iteratively. s100 00:11:18.280 --> 00:11:22.520 The same is true for getting great output for HyperFrames videos. s101 00:11:22.520 --> 00:11:31.160 Um we want to make sure our power users have full control over this and the same way we created videos before HyperFrames is how we create them with HyperFrames. s102 00:11:31.160 --> 00:11:34.880 We think about the narrative and the vision and the mission of this video. s103 00:11:34.880 --> 00:11:39.200 We storyboard it frame by frame and think about what each frame needs to do. s104 00:11:39.200 --> 00:11:53.800 We then move that into adding motion frame by frame utilizing HTML, CSS, and JavaScript, merging it into one cohesive video, um and then utilizing our studio that we released with Hyperframes for that last mile editing. s105 00:11:53.800 --> 00:12:04.040 Ensuring that humans are always in the loop and have access to do anything that they would do in their normal video editor within the open-source framework's editor so that you can manually drag, tweak, s106 00:12:04.040 --> 00:12:09.560 uh etc. And that gets you output that looks like this. s107 00:12:09.560 --> 00:12:19.520 Uh we actually just released this today, keyframes in Hyperframes, uh which is a big aspect of what makes great videos for motion designers, and it allows you to basically coordinate s108 00:12:19.520 --> 00:12:31.680 all of this different motion uh frame by frame or keyframe by keyframe um in our studio and make sure that you can do anything that a professional motion designer might be able to do in After Effects. s109 00:12:35.720 --> 00:12:37.440 So, it's not just a demo. s110 00:12:37.440 --> 00:12:40.360 We've been uh we released Hyperframes a couple months ago. s111 00:12:40.360 --> 00:12:42.280 It's been running at scale in our opinion. s112 00:12:42.280 --> 00:12:47.360 Over 1.3 million videos have been rendered by open-source users of Hyperframes in the last 90 days. s113 00:12:47.360 --> 00:12:50.040 267,000 creators have tried it. s114 00:12:50.040 --> 00:12:56.800 We have about 15,000 videos every single day being rendered utilizing the open-source framework and 32,000 GitHub stars. s115 00:12:56.800 --> 00:13:00.480 Um but this is only just the start for us. s116 00:13:00.800 --> 00:13:03.280 As I mentioned already, it's open-source, free forever. s117 00:13:03.280 --> 00:13:05.960 Anyone can go ahead and use Hyperframes right now. s118 00:13:05.960 --> 00:13:13.480 It works with any coding agent that you have, Cloud Code, Codex, Cursor, any number of other ones up here, any ones I'm not aware of even. s119 00:13:13.480 --> 00:13:19.160 If your agent knows how to write HTML, CSS, and JavaScript, it knows how to create a Hyperframes video. s120 00:13:19.160 --> 00:13:26.560 Now, with this, the same agent that is helping you create your product can also help you make your launch video. s121 00:13:28.040 --> 00:13:33.120 The one honest thing that we're going to say here is that the models still aren't good at creative work. s122 00:13:33.120 --> 00:13:38.480 We spend a lot of time evaling and trying to improve this in our skills and push it even further. s123 00:13:38.480 --> 00:13:49.120 But we think there's something at a higher level that needs to change here, which is why we started to work on a code to video benchmark where we are trying to work with the LLM labs, s124 00:13:49.120 --> 00:13:56.320 any creators who are working on video agents to ensure that we can raise the floor of videos for everyone. s125 00:13:56.320 --> 00:14:01.160 If anyone is interested in this space or working on this, we'd be happy to be collaborators, talk more about this. s126 00:14:01.160 --> 00:14:05.920 Feel free to find me or any member wearing a Hyperframes t-shirt in the crowd afterwards. s127 00:14:05.920 --> 00:14:08.760 Love to chat to you guys more. s128 00:14:09.320 --> 00:14:14.760 And one more thing, some of you may have seen the AI engineer Warfare showcase video. s129 00:14:14.760 --> 00:14:22.040 We collaborated heavily with the team to create this with them utilizing Hyperframes as a little surprise of what it can do. s130 00:14:22.040 --> 00:14:28.480 Here's a great example of the motion graphics and designing that you can do with HTML, CSS, and JavaScript. s131 00:14:31.080 --> 00:14:34.040 So yeah, agents are made building incredibly easy. s132 00:14:34.040 --> 00:14:35.880 Launching is still quite hard. s133 00:14:35.880 --> 00:14:42.360 We think HTML is all your agents need in order to make great videos and launch your product into the world. s134 00:14:42.360 --> 00:14:45.240 Here is a link to Hyperframes, the project, the open source repo. s135 00:14:45.240 --> 00:14:47.680 Feel free to check it out, star it, download the skills. s136 00:14:47.680 --> 00:14:52.040 You can reach me on X at Reems_Juso, and thank you for attending. s137 00:14:52.040 --> 00:14:55.320 I'll be outside if anyone wants to chat.