---
license: mit
library_name: pewm
pipeline_tag: image-to-video
tags:
  - robotics
  - world-model
  - video-generation
  - embodied-ai
  - dynamicrafter
  - qwen2.5-vl
---

# Primitive Embodied World Models

This is the Hugging Face card template for **Learning Primitive Embodied World
Models: Towards Scalable Robotic Learning** (arXiv:2508.20840).

## Intended artifacts

Upload each component separately or as a collection:

- `pewm-dynamicrafter-1.4b`: three-stage PEWM world-model checkpoint;
- `pewm-causal-student`: four-step/four-frame Self-Forcing student;
- `pewm-qwen2.5-vl-7b-lora-p`: primitive planning adapter;
- `pewm-qwen2.5-vl-7b-lora-g`: start/goal grounding adapter;
- calibration/reference assets needed by the Gen6D execution example.

Replace this section with exact filenames, hashes, base model revisions, licenses,
and reproduction hyperparameters before publishing weights.

The DynamiCrafter repository must contain `pewm_manifest.json`; the Qwen adapter
repositories must contain `pewm_adapter_manifest.json`. Do not upload a learned
artifact without these provenance files.

## Inputs

- RGB initial observation;
- primitive instruction;
- signed Gaussian heatmap `H_goal - H_start` at image resolution.

This reproduction implements SGG through an explicitly documented ninth latent
input channel. That injection location is a reproduction choice, not a paper-
disclosed official detail.

## Output

32 RGB future frames representing one semantically atomic action. The downstream
Gen6D pipeline estimates an executable 6-DoF end-effector trajectory.

## Training data

The paper reports 7,326 simulated and 11,465 real primitives. Real trajectories use
five synchronized camera views. Simulation sources include RLBench and LIBERO.
Raw data is not included in this placeholder model repository.

## Evaluation

Paper-reported world-model results on 32-frame sequences:

| SSIM | PSNR | LPIPS | VIF | TVD | FVD | EPiCS |
|---:|---:|---:|---:|---:|---:|---:|
| 0.8126 | 21.0644 | 0.1647 | 0.3188 | 0.0018 | 0.0002 | 11.45 / 13 |

These values must remain labeled paper-reported until evaluated from the uploaded
checkpoint with a published evaluation script and held-out split.

## Limitations

Performance depends on camera placement, calibration, visible gripper scale, and
coverage of primitive motions. The paper observes failure for very distant end
effectors and depth displacement below roughly 15 pixels. Rotation-intensive tasks
need additional targeted data. Use collision checking and robot safety controls;
generated video is not a safety guarantee.

## Citation

```bibtex
@article{sun2025pewm,
  title={Learning Primitive Embodied World Models: Towards Scalable Robotic Learning},
  author={Sun, Qiao and Yang, Liujia and Tang, Wei and Huang, Wei and Xu, Kaixin and Chen, Yongchao and Liu, Mingyu and Yang, Jiange and Zhu, Haoyi and Wang, Yating and He, Tong and Chen, Yilun and Dai, Xili and Ye, Nanyang and Gu, Qinying},
  journal={arXiv preprint arXiv:2508.20840},
  year={2025}
}
```