WonderWorld

Preview：

<br />

Introduce：

WonderWorld is an innovative 3D scene extension framework that allows users to explore and shape virtual environments based on a single input image and user-specified text. It significantly reduces computation time through a fast Gaussian voxel and guided diffusion depth estimation method, generating geometrically consistent extensions that allow 3D scenes to be generated in less than 10 seconds, enabling real-time user interaction and exploration. This opens up the possibility of quickly generating and navigating immersive virtual worlds in fields such as virtual reality, gaming and creative design.

Stakeholders:
The target audience includes virtual reality developers, game designers, creative designers, and others who need to quickly generate and explore 3D virtual environments. WonderWorld’s real-time interaction and fast generation capabilities are especially suitable for professional users who need immediate feedback and creative implementation.
Usage Scenario Examples:

Users generate a virtual marketplace in Marienlilatz through text descriptions and camera movements.
Generate a historical scene in Forbidden City and explore it from different perspectives.
Create a virtual tour experience in the Taj Mahal that allows users to appreciate the beauty of the architecture from all angles.

The features of the tool:

Real-time rendering and fast scene generation, allowing users to navigate existing content and specify where and what to build.
The user can specify the new scene content and style through text, and specify the generation position of the new scene through camera movement.
Generate scenes based on a single view instead of multiple views for each scene, reducing large discontinuous holes.
Generate connected and diverse 3D scenes in less than 10 seconds on a single A6000 GPU.
Enable users to quickly generate and navigate immersive virtual worlds in areas such as virtual reality, gaming and creative design.
The technical context offers new possibilities for user-driven content creation and exploration in virtual environments.
Commitment to release full code and software for replication.

Steps for Use:

Load the WonderWorld model and select an input image.
Use text descriptions to specify the content and style of the scene you want to generate.
Move the camera to determine where the new scene will be generated.
Observe the 3D scene rendered in real time and make adjustments as needed.
Use the ‘W/A/S/D’ key to move the scene and use the ‘I/J/K/L’ key to look around.
Tap the canvas to activate the controls and start exploring and interacting.
Further customize and extend the virtual environment according to individual needs.

Tool’s Tabs: 3D scene generation, virtual reality