Seriously, It’s AI Seriously, It’s AIAI News • Tools • Trends Subscribe
Latest
AI News

Kyutai AURA Turns Your Voice Into a Live Talking Avatar

November 22, 2026 By Seriously, It’s AI 0 Comments


I read Kyutai's AURA paper the morning it hit arXiv (19 November 2026) and checked the headline numbers against the paper itself before writing this. My method was simple: the paper first, then the Hugging Face release notes and the live demo page — no second-hand summaries. Here is what I found, and where the numbers need a footnote.

What AURA actually does

Give AURA one portrait photo and a stream of speech, and it draws that face talking — lips moving in sync with the words, video generated as the audio arrives. It is a speech-to-video model, and it comes in two sizes: AURA 1.3B and AURA 5B. The small one is the news: it streams 576p video at 20.7 frames per second on a single GPU, faster than its 16 Hz playback rate, which is what "real time" means here. The bigger one renders 720p at 9.7 fps — sharper, but below real-time speed.



AURA's two sizes against the reference photo, and the quality-vs-speed trade-off. Figure: Kyutai, AURA paper (arXiv:2611.26540).

The trick is what it doesn't wait for

Most talking-head models are bidirectional: before they can draw a chunk of video, they need several seconds of audio from the future. That makes them fine for pre-rendered clips and useless for a live conversation. AURA is causal. It generates video chunk by chunk, and to start a chunk it needs only the audio of that chunk — about 0.64 seconds. A rolling KV cache carries the state forward: the first chunk stays as an identity anchor, the most recent chunk supplies context, and memory use stays constant no matter how long the stream runs. The paper reports streams of 60 seconds and longer.

Under the hood, AURA is a distilled student of TalkVerse, a bidirectional teacher that needs 50 denoising steps per chunk. AURA does it in four. Two training stages — first matching the teacher's trajectories, then Distribution Matching Distillation with backward simulation — close the quality gap. The result, per the paper: the 1.3B model runs 23 times faster than its own teacher at comparable quality, and the 5B model matches the teacher at 720p while running 17 times faster.

What you get for free

Kyutai released the weights for both models, the inference code, the paper, and a dataset of 30,000 synthetic talking-head videos — 38.7 hours of footage, with the teacher's trajectories included so other researchers can reproduce the distillation. There is a live demo in the browser at aura.kyutai.org, and the 1.3B model is wired into Unmute, Kyutai's voice AI stack, so you can speak to a lip-synced avatar in the browser.

One catch, and it matters: the release is non-commercial. The small model carries a CC-BY-NC-4.0 licence, and the 5B weights carry Snap's non-commercial terms because they descend from TalkVerse. You can research with AURA and build demos. You cannot ship it in a commercial product — a sharper limit than the Apache 2.0 terms on Thinking Machines' Inkling 2 Mini, the open-weights model that shipped the same week.

Why this matters

Real-time changes what a talking head is for. A pre-rendered avatar is a video-production tool. A live one is an interface: the face of a voice assistant, a tutor that looks at you while it explains, a receptionist that never sleeps. That is the use case Kyutai is aiming at, and the open release means anyone can test it, break it and fine-tune it. It lands in a week when mid-tier models are closing on flagships fast — Google's Gemini 4 Neon, for one, codes like a flagship at a mid-tier price.

Creative AI is also moving, frame by frame, from stills to motion: earlier this month OpenAI's Images 2.6 added animated GIFs to image generation. AURA is the audio-driven half of that same shift — sound in, moving face out.

The honest limits

First, lip-sync quality still trails the best offline models. On the HALLO3 benchmark, AURA-5B scores a Sync-C of 4.30, against 5.06 for Hallo3 and 5.42 for Wan-S2V — the paper's own numbers, and the gap is real.

Second, the speed figures come from one Nvidia H100, a data-centre GPU. Kyutai reports no consumer-GPU numbers, so "runs on a single GPU" does not mean "runs on your GPU".

Third, only the 1.3B model is truly real-time; the sharper 5B streams at 9.7 fps, below playback speed.

Fourth, the models were trained and evaluated on English speech only — other languages, non-speech audio and non-portrait framings are untested.

And the obvious one: a convincing live face from a single photo is a deepfake kit as much as a product feature. Open weights plus a public demo means that risk is live now, not theoretical.

My verdict: AURA is not the best-looking talking head you can generate — the offline models still win that. It is the first open one that can hold a conversation, and that changes the category more than a prettier render would. Try the demo, watch the lips, and judge the latency for yourself.

Would you talk to a face that isn't real, if it answers in real time? Tell me in the comments.

Sources:

- AURA: AUdio-dRiven streaming Avatar, arXiv:2611.26540 (19 Nov 2026) — https://arxiv.org/abs/2611.26540
- AURA live demo — https://aura.kyutai.org
- Kyutai on Hugging Face — https://huggingface.co/kyutai

Comments

Browse by Category

Stay Ahead with Seriously, It’s AI

Get the latest AI news, trends and expert insights — explained simply, every day.

Follow Us →

Stay updated with AI

One short email a week. No spam, unsubscribe anytime.