HY-World 2.0 — 3D World Model Skill
Skill by
ara.so — Daily 2026 Skills collection.
HY-World 2.0 is a multi-modal world model by Tencent Hunyuan that reconstructs, generates, and simulates 3D worlds. It accepts text, single-view images, multi-view images, and videos as input and produces 3D representations (meshes, 3D Gaussian Splattings, point clouds). Two core capabilities:
- World Reconstruction (multi-view images / video → 3D): Powered by WorldMirror 2.0, a ~1.2B feed-forward model predicting depth, surface normals, camera parameters, 3D point clouds, and 3DGS attributes in a single forward pass.
- World Generation (text / single image → 3D world): Four-stage pipeline — Panorama Generation (HY-Pano 2.0) → Trajectory Planning (WorldNav) → World Expansion (WorldStereo 2.0) → World Composition (WorldMirror 2.0 + 3DGS).
Installation
Requirements
- Python 3.10
- CUDA 12.4 (recommended)
- PyTorch 2.4.0
bash
# 1. Clone repository
git clone https://github.com/Tencent-Hunyuan/HY-World-2.0
cd HY-World-2.0
# 2. Create conda environment
conda create -n hyworld2 python=3.10
conda activate hyworld2
# 3. Install PyTorch with CUDA 12.4
pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
# 4. Install project dependencies
pip install -r requirements.txt
# 5a. Install FlashAttention-3 (recommended for performance)
git clone https://github.com/Dao-AILab/flash-attention.git
cd flash-attention/hopper
python setup.py install
cd ../../
rm -rf flash-attention
# 5b. OR install FlashAttention-2 (simpler)
pip install flash-attn --no-build-isolation
Model Weights
Model weights are automatically downloaded from Hugging Face on first run. Alternatively, download manually:
| Model | HuggingFace | |
|---|
| WorldMirror 2.0 | tencent/HY-World-2.0 → HY-WorldMirror-2.0 | |
| WorldMirror 1.0 (legacy) | tencent/HunyuanWorld-Mirror | |
To pre-download:
bash
# Set HuggingFace cache directory if needed
export HF_HOME=/path/to/cache
pip install huggingface_hub
python -c "from huggingface_hub import snapshot_download; snapshot_download('tencent/HY-World-2.0')"
Core API — WorldMirror 2.0 (World Reconstruction)
Basic Usage
python
from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
# Load pipeline — weights auto-downloaded on first run
pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
# Run reconstruction from a folder of images
result = pipeline('path/to/images')
With Prior Injection (Camera & Depth)
Provide known camera parameters or depth priors to improve accuracy:
python
from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
result = pipeline(
'path/to/images',
prior_cam_path='path/to/prior_camera.json',
prior_depth_path='path/to/prior_depth.npy', # optional
)
Camera JSON Format
The prior_camera.json format expected by the pipeline:
json
[
{
"image": "frame_001.jpg",
"fx": 800.0,
"fy": 800.0,
"cx": 640.0,
"cy": 360.0,
"width": 1280,
"height": 720,
"c2w": [
[1.0, 0.0, 0.0, 0.0],
[0.0, 1.0, 0.0, 0.0],
[0.0, 0.0, 1.0, 0.0],
[0.0, 0.0, 0.0, 1.0]
]
}
]
Result Object
The pipeline returns a result object with the following attributes:
python
result = pipeline('path/to/images')
# Access outputs
point_cloud = result.point_cloud # 3D point cloud (numpy or torch)
depth_maps = result.depth_maps # Per-image depth maps
normals = result.normals # Surface normal maps
cameras = result.cameras # Predicted camera parameters
gaussians = result.gaussians # 3DGS attributes
# Save outputs
result.save('output_dir/') # Saves all outputs to directory
Gradio App — WorldMirror 2.0
Launch an interactive web UI for 3D reconstruction:
bash
# From project root
python -m hyworld2.worldrecon.app
# Or if a dedicated script exists
python app.py --model tencent/HY-World-2.0
Access at http://localhost:7860 by default.
Common Patterns
Pattern 1: Reconstruct from a Video
Extract frames from a video, then run reconstruction:
python
import cv2
import os
from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
def extract_frames(video_path, output_dir, fps=2):
os.makedirs(output_dir, exist_ok=True)
cap = cv2.VideoCapture(video_path)
video_fps = cap.get(cv2.CAP_PROP_FPS)
frame_interval = int(video_fps / fps)
frame_idx = 0
saved = 0
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
if frame_idx % frame_interval == 0:
cv2.imwrite(f"{output_dir}/frame_{saved:04d}.jpg", frame)
saved += 1
frame_idx += 1
cap.release()
return output_dir
# Extract frames at 2 fps
frames_dir = extract_frames("scene.mp4", "frames/", fps=2)
# Run reconstruction
pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
result = pipeline(frames_dir)
result.save("output_3d/")
Pattern 2: Flexible Resolution Inference
WorldMirror 2.0 supports 50K–500K pixel resolution. Control via resize parameters:
python
from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
# Low resolution (fast, lower memory)
result_fast = pipeline(
'path/to/images',
resolution=512, # resize shorter edge to 512
)
# High resolution (slower, more detail)
result_hq = pipeline(
'path/to/images',
resolution=1024,
)
Pattern 3: Batch Processing Multiple Scenes
python
import os
from pathlib import Path
from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
scenes_root = Path("scenes/")
output_root = Path("outputs/")
for scene_dir in sorted(scenes_root.iterdir()):
if not scene_dir.is_dir():
continue
out_dir = output_root / scene_dir.name
out_dir.mkdir(parents=True, exist_ok=True)
print(f"Processing: {scene_dir.name}")
try:
result = pipeline(str(scene_dir))
result.save(str(out_dir))
print(f" Saved to {out_dir}")
except Exception as e:
print(f" Failed: {e}")
Pattern 4: Export to Common 3D Formats
After reconstruction, export to formats compatible with Blender / Unity / Unreal:
python
from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
pipeline = WorldMirrorPipeline.from_pretrained('tencent/HY-World-2.0')
result = pipeline('path/to/images')
# Save 3DGS (.ply format for tools like 3D Gaussian Splatting viewer)
result.save_gaussians("scene.ply")
# Save mesh (if mesh export is supported)
result.save_mesh("scene.obj") # or scene.glb
# Save point cloud
result.save_pointcloud("scene_pointcloud.ply")
Pattern 5: GPU Memory Management
For large scenes or limited VRAM:
python
import torch
from hyworld2.worldrecon.pipeline import WorldMirrorPipeline
# Load in fp16 to reduce memory
pipeline = WorldMirrorPipeline.from_pretrained(
'tencent/HY-World-2.0',
torch_dtype=torch.float16,
)
pipeline = pipeline.to('cuda')
# Run with lower resolution to fit in memory
result = pipeline('path/to/images', resolution=768)
# Free memory after use
del result
torch.cuda.empty_cache()
Project Structure
HY-World-2.0/
├── hyworld2/
│ ├── worldrecon/ # WorldMirror 2.0 reconstruction
│ │ ├── pipeline.py # Main WorldMirrorPipeline class
│ │ ├── app.py # Gradio web app
│ │ └── ...
│ ├── worldgen/ # World generation (coming soon)
│ │ ├── panorama/ # HY-Pano 2.0
│ │ ├── nav/ # WorldNav trajectory planning
│ │ └── stereo/ # WorldStereo 2.0
│ └── utils/
├── assets/ # Demo assets
├── requirements.txt
└── README.md
Environment Variables
bash
# HuggingFace model cache location
export HF_HOME=/path/to/hf/cache
# HuggingFace token (if accessing private/gated models)
export HUGGING_FACE_HUB_TOKEN=your_token_here
# CUDA device selection
export CUDA_VISIBLE_DEVICES=0
# For multi-GPU setups
export CUDA_VISIBLE_DEVICES=0,1
Troubleshooting
FlashAttention installation fails
bash
# Use FlashAttention-2 as fallback
pip install flash-attn --no-build-isolation
# If that fails, disable flash attention (slower but works)
# Set environment variable before running
export USE_FLASH_ATTENTION=0
CUDA out of memory
python
# 1. Reduce resolution
result = pipeline('path/to/images', resolution=512)
# 2. Use fp16
pipeline = WorldMirrorPipeline.from_pretrained(
'tencent/HY-World-2.0',
torch_dtype=torch.float16
)
# 3. Process fewer images at once — use a subset
import os
images = sorted(os.listdir('path/to/images'))[:10] # limit to 10 frames
Model download issues
bash
# Use HF mirror if huggingface.co is blocked
export HF_ENDPOINT=https://hf-mirror.com
# Or manually download and point to local path
pipeline = WorldMirrorPipeline.from_pretrained('/local/path/to/model')
Wrong PyTorch/CUDA version
bash
# Verify versions match
python -c "import torch; print(torch.__version__, torch.version.cuda)"
# Should output: 2.4.0 12.4
# Reinstall if mismatch
pip install torch==2.4.0 torchvision==0.19.0 --index-url https://download.pytorch.org/whl/cu124
Images not loading
python
# Ensure images are valid and in supported formats (.jpg, .png)
from PIL import Image
import os
img_dir = 'path/to/images'
for f in os.listdir(img_dir):
try:
img = Image.open(os.path.join(img_dir, f))
img.verify()
except Exception as e:
print(f"Bad image {f}: {e}")
Related Projects
| Project | Use Case | Link | |
|---|
| WorldStereo | Panorama → 3DGS (open-source preview of WorldStereo-2) | GitHub | |
| HunyuanWorld 1.0 | Panorama generation (interim for HY-Pano 2.0) | GitHub | |
| WorldMirror 1.0 | Legacy reconstruction model | HuggingFace | |
Key Limitations (Current Release)
- World Generation pipeline (WorldNav, WorldStereo-2, HY-Pano-2) is not yet open-sourced — only WorldMirror 2.0 reconstruction is available.
- Panorama generation: Use HunyuanWorld 1.0 as interim.
- World Expansion: Use WorldStereo as interim.
- Requires CUDA GPU — CPU inference not officially supported.
- Minimum ~8GB VRAM recommended; 16GB+ for full-resolution inference.