Genie 4 Generates Sound, Speech and Video Together in Real Time
Google has released a research preview of Genie 4, a world model that generates video, ambient sound and spoken dialogue at the same time while a user moves through the scene with keyboard controls and steers it with text prompts. In the announcement, Genie 4: A new generation of world simulation, Raia Hadsell, Diego Rivas and Hang Qi describe it as the company’s first real-time world model to produce audio natively. The post contains demonstration clips, the prompts behind them and feedback from trusted testers. It does not contain benchmarks, resolution figures, latency measurements or session lengths, which limits what can be said about performance.
A world model built on Gemini components
Genie 4 incorporates capabilities from the Gemini family. According to Google, Gemini Omni anchors the simulations in real-world appearance and behavior, while other Gemini models supply the reasoning that keeps a simulation coherent and continuous. Users set an event in motion, and Genie 4 simulates what follows: how the world responds, how it sounds, and how the people in it move, react and speak.
Google ties the work to artificial general intelligence, arguing that understanding and simulating the world is a requirement for building it. The nearer-term uses it names are training environments for robots and agents and new interactive experiences. The model is currently a research preview for trusted testers. Google AI Ultra subscribers in Project Genie and early access partners of the Gemini API and Google AI Studio are listed as next, and a waitlist for Project Genie is open.
Steering a simulation with text
The first capability Google highlights is real-time modification through natural language. A user can change the weather, move the camera, or introduce a new person or object, and Gemini adapts the prompt to fit the surrounding scene. Between user inputs, Gemini also monitors the scene and steers it so the world keeps progressing cohesively. A model other than Genie is therefore making decisions while the user is idle.
The published prompts show how much specification goes into a scene. One begins with a reference image used as the exact first frame, describes a brass saxophonist on a floating concrete platform in a misty lake, and asks for 16mm film grain, muted color and a camera that moves independently of both the man and the platform. Another runs a river of bubblegum-pink satin through a wildflower meadow. A third places the viewer in the basket of a hot air balloon over Antarctica, with the wicker edge, rigging chains and canopy framing the view.
Several prompts devote as much text to exclusions as to content. The dragon-racing prompt specifies a first-person view and then lists what the camera must never show, including the rider’s face or body, third-person angles, drone shots and any CGI or video-game look. The prompt for a crowded 1980s Japanese train station ends with a run of prohibitions covering empty platforms, a distant train, sparse crowds and symmetrical composition. The announcement does not say who wrote the prompts or how many attempts each clip required.
Physics, people and sound
Google says Genie 4 is built on an advanced physical understanding of the world and simulates aerodynamics, collisions and similar scenarios. It calls the largest improvement the simulation of people, who move naturally, respond to their environments and can be steered in real time. The demonstration prompts include a watermelon on a stone plinth with airborne droplets backlit in slow motion, a macro shot of a maple seed falling into grass, and a platform packed with passengers beside a deep-red train.
Audio is the other new element. Google says every world carries the ambient sound of its setting, from wind in the trees to waves at the beach, and that characters can hold dialogue that adapts as the user acts or events unfold. The prompts include audio instructions. The dragon prompt asks for wingbeats, wind gusts, volcanic rumble and crackling embers, and the saxophone prompt asks for lake ambience and instrument sound with no background music beyond what is physically in the scene. The written announcement includes no dialogue transcript and does not describe how dialogue quality was assessed. Google also lists real-time audio accuracy among its open research problems.
Gemini as a planner inside the loop
For multi-step events, Gemini 3.8 Flash acts as a guiding agent. As Genie 4 generates a world, Gemini observes the scene, breaks a goal into steps, and reviews incoming frames to verify that each step succeeded. When the world shifts or the user intervenes, it revises the plan. Google says this produces worlds with reliable causality, procedural behavior and multi-step reasoning.
Two examples illustrate the approach. In the first, a Universal Robots UR5 arm holds a pipette and dispenses liquid into a lab plate, triggering tiny electric sparks. In the second, Gemini 4 Argon breaks a user-instructed task into steps and feeds them to Genie 4 as text captions while it renders. The demonstration shows Apptronik’s Apollo humanoid playing checkers. Google presents this as a way to simulate a breadth of scenarios in imagined worlds and to test how AI systems navigate environments before they reach the real world, and it describes the work as a step toward combining its frontier research with Gemini Robotics. The post reports no completion rates, so it is unknown how often Gemini’s step verification catches a failed step.
What testers report
Google worked with cohorts of trusted testers in creative industries, journalism, education, gaming and engineering. Creative teams are using Genie 4 as a draft mode for pre-visualization. Filmmakers can step into a live simulation of a scene and run multiple takes, adjusting camera angles, lighting, dialogue and action as they go.
Photojournalists are testing whether a single archival 2D photograph can become a grounded simulation of that environment in seconds, and whether populated worlds can help people experience historical spaces or visualize short-term events such as a flash flood or wildfire. Game developers and creative technologists are going from an initial idea to an explorable prototype in seconds, trying visual styles, materials and mechanics through prompts before writing engine code. Enterprise partners are evaluating Genie 4 for rapid scenario generation from a blank starting point, and Google says they are identifying gaps it still needs to close.
The testers are not named and no counts are given. These findings are Google’s summary of feedback, and “in seconds” is not tied to a measurement.
Limitations and safeguards
Google calls Genie 4 an early, experimental model. It can maintain coherent environments over short sessions, but visual details and object permanence can shift or drift, especially as a simulation runs longer, after multiple complex prompt edits, or while the user navigates through the scene. Complex physical interactions, real-time audio accuracy, legible text, spatial accuracy and long-term consistency are listed as active research areas. Repeated filmmaking takes involve many prompt edits, and a reconstruction of a historical space depends on spatial accuracy, so two of the four tester uses touch areas Google describes as unfinished.
On safety, Google says Genie 4 includes guardrails consistent with its policies for other models and went through internal safety evaluations before the preview launched. It was deployed to trusted testers first to gather feedback before broader release. All video and audio the model generates carries SynthID, an imperceptible watermark covering both visual and audio output. Because the watermark cannot be seen or heard, checking a clip requires a detection tool.
Looking ahead, Google points to hybrid world models that pair the structured reasoning of code and language models with pixel-level simulation. In the current arrangement, Gemini maps the structure and plans the narrative timeline while Genie generates the immediate physical reality, including how the world looks, sounds and responds.
Next steps for teams evaluating it
The Project Genie waitlist is the stated route to access. Teams considering Genie 4 for pre-visualization, prototyping or robotics scenarios can test it against the conditions Google flags as weak points. That means running sessions at the length they would need, applying the number of prompt edits their workflow involves, and checking whether on-screen text is legible if signage or labels matter. They should also ask Google for latency, resolution and session-length figures, since the announcement does not supply them.
