{
  "video": {
    "id": "xfPVBzDYjH4",
    "title": "Can Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff",
    "duration": 1214,
    "upload_date": null,
    "channel": "AI Engineer",
    "source": "AI Engineer",
    "event_format": ""
  },
  "analysis": {
    "video_id": "xfPVBzDYjH4",
    "title": "Can Your Agent Hear You Now? Building Live Voice Agents with Gemini — Thor Schaeff",
    "one_liner": "Gemini 3.1 Flash Live is a native-audio model, not a cascading STT→LLM→TTS pipeline. You reach it through the stateful WebSocket Live API, sending text, audio and video frames in real time and getting streamed audio chunks plus transcripts back. It supports 90 languages, barge-in, tool calls and Google Search grounding.",
    "summary": "Thor Schaeff (developer experience, Google DeepMind) previews a workshop on real-time multimodal conversational agents built with the Gemini API, running from one live demo to the next. He shows Gemini Live calling Nano Banana Light through the stateful Interactions API to illustrate a story with consistent images, Gemini 3.5 Live Translate translating the talk live to audience phones, and a Gemini Live 'radio DJ' that makes music on demand with Lyria 3 through a tool call. He then explains how the Live API works and demos it in AI Studio with vision, voice selection and Google Search grounding.",
    "key_points": [
      "Gemini TTS is pitched for expressive speech. You can steer it with inline audio tags, stream it, and it has much lower latency than it used to.",
      "Google shipped Omni Flash (video generation and editing) and Nano Banana Light the morning of the talk. Nano Banana Light cuts image generation and editing latency significantly, though an image still takes about 2-3 seconds.",
      "Story canvas demo: Gemini Live gets a tool call and uses the Interactions API to generate images on the fly. Passing the previous interaction ID chains the context, so visual style and characters stay consistent across images.",
      "The Interactions API is stateful and is described as Google DeepMind's unified API for all of its models and agents.",
      "Gemini 3.5 Live Translate does near-real-time translation. It detects the speaker's language automatically, and audience members scanned a QR code to hear the talk in German, Chinese and other languages. It is available on the Gemini API.",
      "Gemini 3.1 Flash Live is the same model you talk to in the Gemini app. The live jukebox demo was vibe-coded in Google AI Studio and calls Lyria 3 through a function call to make reggaeton on request. It can be forked in AI Studio.",
      "The Live API is a stateful WebSocket API: you create a session, stream in text, audio and video frames, and get back audio chunks plus a transcript. Because the model is native audio, its intelligence is in the audio model itself rather than going through text. That is what lets it switch fluidly between the 90 languages supported in preview, and it also supports barge-in/interruption and tools.",
      "AI Studio demo (ai.studio/live, ai.dev/live): pick Gemini 3.1 Flash Live, a voice (e.g. Puck) and Google Search grounding, which works like a built-in function call for real-time information. The model correctly described the speaker's shirt, hat and background and read 'Gemini' off his sweatshirt, then switched to Mandarin."
    ],
    "takeaways": [
      "For real-time voice agents, look at a native-audio model through the Live API instead of a cascading STT→LLM→TTS pipeline. You get lower latency and fluid multilingual switching.",
      "Give your live agent tool calls to real backends, such as pulling customer information from a CRM or creating a ticket, the same way the demo called Lyria 3.",
      "Use the Interactions API's previous interaction ID to chain generations and keep images consistent across a session.",
      "Real-time media is hard to scale, so consider partners like LiveKit and Pipecat to get running quickly with the Gemini Live API.",
      "Prototype first in Google AI Studio (ai.studio/live): try voices, grounding with Google Search and the translate model, then fork the demo apps."
    ],
    "topics": [
      "voice agents",
      "real-time",
      "multimodal",
      "speech translation",
      "tool calling",
      "image generation",
      "gemini api",
      "websockets"
    ],
    "tools": [
      "Gemini TTS",
      "Gemini 3.1 Flash Live",
      "Gemini Live API",
      "Gemini 3.5 Live Translate",
      "Gemini Interactions API",
      "Nano Banana Light",
      "Omni Flash",
      "Lyria 3",
      "Google AI Studio",
      "Google Search grounding",
      "Gemini app",
      "Google DeepMind",
      "LiveKit",
      "Pipecat"
    ],
    "quotes": [
      {
        "text": "the interactions API is exciting because it is a stateful API which means we can chain our interactions very seamlessly",
        "at": "05:45",
        "url": "https://www.youtube.com/watch?v=xfPVBzDYjH4&t=345s"
      },
      {
        "text": "this is a native audio model. So the model itself does not go through text, but rather the intelligence is baked into the audio model.",
        "at": "15:07",
        "url": "https://www.youtube.com/watch?v=xfPVBzDYjH4&t=907s"
      },
      {
        "text": "We spend a lot of money on this domain, AI.studio. So I do appreciate if you use that one.",
        "at": "16:37",
        "url": "https://www.youtube.com/watch?v=xfPVBzDYjH4&t=997s"
      },
      {
        "text": "Gemini made up this title.",
        "at": "10:11",
        "url": "https://www.youtube.com/watch?v=xfPVBzDYjH4&t=611s"
      }
    ],
    "words": 3326
  },
  "summary_url": "/#xfPVBzDYjH4",
  "transcript": {
    "html": "/transcripts/xfPVBzDYjH4.html",
    "txt": "/transcripts/xfPVBzDYjH4.txt",
    "vtt": "/transcripts/xfPVBzDYjH4.vtt"
  }
}