Connect with us

NEWS

Google Genie 4 Puts Gemini in Charge of Live Worlds

Genie 4 adds live audio and a Gemini planner in the loop, while AMD, Rivian, and Amazon move to buy standalone world-model labs.

Published

on

Google DeepMind released Genie 4 on October 27, 2026, a world model that builds interactive scenes with live sound and mid-run text steering.

The model is a research preview for trusted testers. Google said it will follow into Project Genie for Google AI Ultra subscribers and to early partners on the Gemini API and Google AI Studio. Gemini now sits in the loop as a planner that breaks goals into steps and checks frames while Genie draws the world.

Google Puts Gemini Inside the Simulation Loop

Raia Hadsell, Diego Rivas, and Hang Qi, the DeepMind researchers who posted the announcement, wrote that Genie 4 is built on the Gemini family. Gemini Omni anchors how things look and behave. Other Gemini models supply the reasoning that keeps a scene from falling apart as it runs.

The lab’s own follow-up on X put the pitch in five lines: lifelike detail from Gemini Omni, better aerodynamics and collisions, live prompt control, native audio, and Gemini planning multi-step events so a scene can keep going.

That last piece is the change from a demo you watch to a sim you can task. DeepMind said Gemini 3.8 Flash acts as a guiding agent in the loop. As Genie 4 generates a world, Gemini watches the scene, splits a goal into steps, and reviews incoming frames to check that each step landed.

What we’ll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction.

Tejas Kulkarni, research director at Google, press briefing

DeepMind has also been pairing Genie 4 with Gemini 4 Argon on Gemini Robotics work. In one clip, Argon breaks a user task into steps and feeds those steps to Genie as text captions while the scene is still rendering. The lab showed Gemini 4 Argon and Genie 4 simulating Apptronik’s Apollo humanoid robot playing checkers, with the task split into live steps.

That is a rehearsal room for robots, not a finished factory floor. The clip is a research experiment. DeepMind still lists complex physics, spatial accuracy, and long-term consistency as open problems, so a checkers demo does not mean a humanoid can be signed off in this sim before it walks a real room.

Wind, Waves, and Characters Who Answer

DeepMind calls Genie 4 its first real-time world model to generate audio natively. Wind in trees, waves on a beach, music, and spoken lines are produced with the picture, not laid on after the frames exist.

One official recording starts with a man in a loose brown suit playing a saxophone on a floating concrete platform in a foggy lake. The camera is free to glide over the water. The horn keeps sounding, with lake ambience and wind, while the player’s fingers and breath keep moving. That is the product Google wants testers to feel: a place you can walk, not a clip you can replay.

Users steer with a keyboard for movement and camera, then type to change weather, shift the view, or drop in a person or object. Between those inputs, Gemini rewrites the prompt to fit the scene and keeps the world moving so it does not freeze when you stop typing.

Speech is the harder layer. DeepMind said characters can talk and react as events happen, and that human speech is the frontier it is chasing. It also said real-time audio accuracy is still under study, so a line that matches the picture on one take can slip on the next.

Chipmakers Are Buying the Other World Labs

While Google keeps Genie next to Gemini, buyers who do not own a frontier model stack are purchasing the remaining labs. The pattern around this preview is physical AI: chips, cars, and robots, not a consumer game store.

AMD said on September 28, 2026, that it had agreed to an all-stock deal valued at $8.2 billion for World Labs, the San Francisco spatial-intelligence lab led by Fei-Fei Li. The deal is expected to close by the end of 2026, subject to regulatory approvals. Li will join AMD as executive vice president and chief scientist, reporting to chair and CEO Lisa Su. World Labs builds models that generate, rebuild, and simulate interactive 3D spaces from text, images, and video, plus tools for robot learning.

Rivian said on October 22, 2026, that it had acquired Third Dimension AI, whose platform turns real-world data into controllable 3D environments for driver assistance and self-driving work. Terms were not disclosed. James Philbin, Rivian’s senior vice president for autonomy and AI, said simulation is central to how the carmaker trains and checks those systems. Amazon has been in talks to buy Decart, a startup working on real-time video and world models; those talks have not been announced as a signed deal.

THE BUYOUT WINDOW AROUND GENIE 4

Company Move Date What they said they want
Google DeepMind Genie 4 research preview, built in-house October 27, 2026 Robot and agent training worlds, plus new interactive media
AMD World Labs, about $8.2 billion in stock September 28, 2026 Spatial intelligence and robot simulation to shape future chips
Rivian Third Dimension AI, terms undisclosed October 22, 2026 Editable 3D driving scenes from real footage

Lisa Su said building compute for the next wave of AI takes a deep read on how models are changing. Li said joining AMD gives her team the hardware depth to move faster. Those lines match the Genie 4 robotics clip: the scarce asset is a world that a planner can task, not a prettier trailer.

A Playable Rival With No Session Cap

Runway, the AI video company, posted GWM Worlds 2 on September 3, 2026, as a research preview of a playable world model. It generates continuous 720p video at 24 fps with audio at 48,000 Hz, and Runway said sessions have no preset length because the model is autoregressive.

You define the environment, subjects, style, physical rules, and ambience, then steer with text actions aimed at a character or at the scene, plus continuous camera motion. Older frames drop out of a sliding cache, which is Runway’s own account of why long-term memory is imperfect even without a timer. DeepMind has not published resolution or frame rate for Genie 4.

GENIE 4 AND RUNWAY WORLDS 2

Item Genie 4 GWM Worlds 2
Announced October 27, 2026 September 3, 2026
Audio Native ambient sound, music, dialogue 48,000 Hz with the video
Published picture spec Not published 720p at 24 fps
Session length Short sessions; details can drift No preset length; older frames evicted
Planner in the loop Gemini 3.8 Flash, plus Argon tests Optional LLM authoring; agents can drive actions
Access Trusted testers; Ultra and API next Research preview

Google has already parked Unity’s engine inside prompt-built games. Genie 4 is a different bet: skip the engine and generate the place frame by frame. Game studios still have a physics engine that does not forget a crate when you turn around. World models still do, which is why the near-term money is in sketching a scene, not shipping a level.

What Still Breaks After a Short Session?

DeepMind was blunt about the gaps. Genie 4 can hold a coherent environment over short sessions, and then visual details and object permanence start to shift, especially after a long run, after several hard prompt edits, or after you wander the scene. In-world text is often unreadable unless you wrote it into the prompt. Complex collisions still fail. The lab is still working on audio that stays accurate as the picture moves.

LIMITS GOOGLE LISTED IN THE PREVIEW

  • Object drift: Details and permanence slip as a session runs longer or after stacked edits.
  • Hard physics: Complex collisions and interactions remain an active research problem.
  • Live audio: Native sound is in the same pass, but accuracy is not solved.
  • Readable type: Legible text inside a world is still unreliable.
  • Space and time: Spatial accuracy and long-term consistency are open work.
  • Watermarks: All video and audio carry SynthID, DeepMind’s hidden mark for AI media.

Those limits cut against the uses Google is selling. A storyboard can live with a two-minute sketch that smears. A robot policy that trains on a crate that vanishes when the camera pans will learn the wrong world. Testers who expected a game they could play for an afternoon will hit the same wall.

Film Crews Are Using It to Sketch Scenes

DeepMind’s public list of uses is familiar: games, interactive lessons, and robotics. One speculative lesson is walking into a painting of a historical scene and talking to the people in it. Trusted testers have been pointing somewhere else: film prototyping.

The real time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful.

Diego Rivas, Google DeepMind engineer, press briefing

Rivas pointed to ideation and storyboarding, when a crew needs to try ideas fast. Google already sells Veo, a video model that makes high-definition clips with synced audio from text, images, or video. Rivas said users could later lean on Gemini Omni to turn Genie sketches into finished assets, and that those tools are still a long way off. He also said it is early, and that the end-to-end product is not set.

That matches how people who watch these launches have been sorting the demo. A live world you can rain on and re-light is a cheap pre-vis bay. It is a weak stand-in for a shipped game, and a weaker stand-in for a robot that has to pick up a real cup. The robotics clip is Google showing where it wants the stack to go. The testers asking for storyboards are showing where it works today.

Trusted Testers First, Then Ultra and the API

Genie 4 is in a research preview now. DeepMind said it ran safety checks before this round and is collecting feedback before a wider release. Project Genie, the web app that already lets Ultra subscribers walk generated worlds, is the consumer door Google named first.

THE PATH FROM GENIE 3 TO GENIE 4

  1. August 2025: DeepMind shows Genie 3, its first real-time interactive world model.
  2. January 29, 2026: Project Genie opens to Google AI Ultra subscribers in the United States, ages 18 and over.
  3. May 20, 2026: Google Labs says Project Genie is available to all eligible Ultra subscribers worldwide.
  4. September 3, 2026: Runway posts GWM Worlds 2 with native audio and no preset session length.
  5. September 28, 2026: AMD agrees to buy World Labs for about $8.2 billion in stock.
  6. October 22, 2026: Rivian acquires Third Dimension AI for driving simulation.
  7. October 27, 2026: DeepMind opens the Genie 4 research preview and a Project Genie waitlist.

Google has not published a day when Ultra users will see Genie 4 inside Project Genie, and it has not published API pricing. The waitlist form on the DeepMind post is the only public queue. Until that wider door opens, the hybrid stack lives with testers: Gemini writing the next beat, Genie drawing and voicing it, objects still liable to drift if you stay too long.

Harry is the editor of BUDGY APP, an independent title he owns and runs after ten years in journalism that began on a reporter's desk and ended up at the editor's. Numbers get particular attention here. A percentage in a business story is recomputed from the underlying figures before it goes live, a benchmark in a technology or gaming review is quoted with the conditions it was measured under, and a transfer fee or a lap time in the sports and auto pages is traced back to the club, the league or the timing sheet that published it. The same rule covers news, science, entertainment, lifestyle and travel: if a figure cannot be tied to a filing, a dataset, a transcript or a test Harry ran himself, it does not appear. Readers around the world see prices in the original currency with a conversion alongside. Errors are corrected in the open under a published corrections policy, with the change noted on the article. Questions about any figure reach him at support@budgyapp.com.

Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Trending