{ "video": { "id": "9WiBJRO84yY", "title": "Building on the Codex Harness — Dominik Kundel, OpenAI", "duration": 1088, "upload_date": null, "channel": "AI Engineer", "source": "AI Engineer", "event_format": "" }, "analysis": { "video_id": "9WiBJRO84yY", "title": "Building on the Codex Harness — Dominik Kundel, OpenAI", "one_liner": "The Codex app server is an open-source (Apache 2) JSON-RPC protocol around the Codex harness, and the same one powers OpenAI's own Codex app and IDE extensions. You can build any client on it: bundle your own binary, append developer instructions instead of rewriting the system prompt, and mark your custom tools as deferred.", "summary": "Dominik Kundel, who works on developer experience for Codex at OpenAI, argues that you don't need to build your own agent harness. You can build your own interface on top of the Codex harness through the app server, a JSON-RPC style protocol similar to ACP but with deeper control over the harness. He used a custom inspector to show the real traffic between the Codex app and the app server. He explained how to customise the agent with developer instructions, dynamic and deferred tools, and per-thread config. He then demoed Codex running inside Doom, calling game tools and shell commands. He closed with when to use the app server instead of `codex exec`, and three tips.", "key_points": [ "The app server is the protocol that wraps the Codex harness. Both are open source under Apache 2, and they can connect to any model provider with a Responses-API-compatible API. There are convenience settings for LM Studio and Ollama.", "The same JSON-RPC protocol powers OpenAI's first-party interfaces (Codex app, IDE/VS Code extension) and third-party ones: Xcode, JetBrains IDEs, T3 Code, Remote X, and a plugin that puts Codex inside Claude Code for code review and handing off tasks.", "It has over 120 client messages, and more keep being added. They cover initialization, threads and turns, setting a goal (for your own goal mode), listing plugins, file search and file system changes. Unlike the more generic ACP, it gives finer control over individual parts of the harness.", "Clients can support Sign in with ChatGPT, so users bring their own ChatGPT subscription and usage counts against their Codex limits.", "Ways to customise: developer instructions (appended to the Codex system prompt), dynamic tools (which can be marked deferred so the agent finds them through tool search instead of loading them into context), and per-thread overrides for model provider, model, and sandbox network and file permissions.", "The app server ships in the same binary as the Codex CLI. The Codex app bundles its own copy, and on Windows it ships both a Linux build (for WSL mode) and a native Windows build. Codex can generate TypeScript bindings or a JSON schema for a given version. Transports are STDIO or WebSockets.", "There are two ways to integrate: the newly released Python SDK, which handles lifecycle management and Sign in with ChatGPT, or just asking Codex to implement the protocol. Dom says Codex built a Visual Basic Windows-native client largely in one shot.", "Demo: an Electron app using Cloudflare's Doom Wasm renderer and FreeDoom renders Codex natively inside the game and talks to the app server over Electron IPC. Running on 5.5, it used dynamic tools to grant armor, health, weapons and ammo, and ran zsh to get the working directory.", "Use `codex exec` for non-interactive scripting and CI/CD, such as auto-fixing a PR. Use the app server for full clients, or for running many threads in parallel, such as eval harnesses or /goal, without having to parallelise exec calls yourself." ], "takeaways": [ "Bundle your own version of the Codex binary instead of relying on the user's installed CLI. That avoids breaking changes, experimental-feature drift, and users who haven't updated.", "Expose your app's own functions as dynamic tools, but mark them as deferred so they don't fill or confuse the context window.", "Customise behaviour with developer instructions before writing your own system prompt. The base prompt is tuned to stay in distribution for each model, and replacing it throws that away.", "Generate TypeScript bindings or a JSON schema from the exact app server version you ship, because the protocol changes between versions.", "Pick the app server over `codex exec` when you need an interactive UI or many parallel threads. Keep exec for scripted CI tasks." ], "topics": [ "agent harness", "coding agents", "protocols", "json-rpc", "tool calling", "context management", "developer tools", "open source" ], "tools": [ "OpenAI", "Codex", "Codex app server", "Codex CLI", "codex exec", "Codex app", "Codex IDE extension", "VS Code", "ChatGPT", "Agent Client Protocol (ACP)", "MCP", "Responses API", "LM Studio", "Ollama", "Xcode", "JetBrains", "T3 Code", "Remote X", "Claude Code (transcribed as \"Cloud Code\")", "Codex Python SDK", "TypeScript", "JSON Schema", "WebSockets", "WSL", "Electron", "Cloudflare Doom Wasm", "FreeDoom", "Doom", "Visual Basic", "zsh", "5.5" ], "quotes": [ { "text": "So unsurprisingly, Codex is incredibly good at implementing this protocol", "at": "10:54", "url": "https://www.youtube.com/watch?v=9WiBJRO84yY&t=654s" }, { "text": "thought about how can I actually run Codex inside of Doom rather than running Doom inside of Codex.", "at": "12:09", "url": "https://www.youtube.com/watch?v=9WiBJRO84yY&t=729s" }, { "text": "So ship your own version of the harness.", "at": "16:35", "url": "https://www.youtube.com/watch?v=9WiBJRO84yY&t=995s" }, { "text": "And you're losing out on all of that if you're actually jumping in and like writing your own system prompt.", "at": "17:27", "url": "https://www.youtube.com/watch?v=9WiBJRO84yY&t=1047s" } ], "words": 3652 }, "summary_url": "/#9WiBJRO84yY", "transcript": { "html": "/transcripts/9WiBJRO84yY.html", "txt": "/transcripts/9WiBJRO84yY.txt", "vtt": "/transcripts/9WiBJRO84yY.vtt" } }