Learning Action Priors for Cross-embodiment Robot Manipulation
Dong Jing, Tianqi Zhang, Jiaqi Liu, Jinman Zhao, Zelong Sun, Li Erran Li, Zhiwu Lu, Mingyu Ding
Read on arXiv →
Key claim
Pretraining action modules improves VLA model performance significantly.
In plain English
Imagine you're trying to build a system that can understand both visual and language inputs while also performing actions based on that understanding. The challenge is that existing models often struggle with learning how to move or act because they start from scratch, relying solely on the data they see during training. This can lead to issues where the model doesn't effectively learn the timing and coordination needed for actions, especially when it has to adapt to different physical embodiments — this is what's called a lack of motion prior.
To address this, the authors propose a clever solution: instead of jumping straight into training the model to align visual and language features with actions, they first pretrain the action module using motion data alone. This two-stage approach allows the model to learn the necessary motion dynamics before it even starts to deal with the complexities of visual and language inputs. In the first stage, the action module learns from unconditioned action trajectories, which helps it understand how to move without any distractions from visual or language data. Then, in the second stage, this learned knowledge is transferred to the full Vision-Language-Action training, allowing for better alignment and refinement.
The results are promising. By using this method, the model not only converges faster but also performs significantly better on tasks where data is limited. This means that for builders working on similar systems, adopting this two-stage training could lead to more efficient and effective models, especially in real-world applications where data scarcity is a concern.
The approach introduces a two-stage training framework that significantly enhances action learning in VLA models.
The claims are supported by extensive experiments across diverse tasks, demonstrating clear improvements over prior methods.
Deep reliability assessment
The methodology supports the narrower claim that action-only pretraining can improve convergence and success under their shared cross-embodiment training setup across LIBERO, RoboCasa GR1, and a small Franka real-world suite. The broader claim that this learns a generally reusable motion prior is less fully established without stronger comparisons, held-out embodiments, and clearer quantitative reporting for scaling and real-world transfer.
Reproducibility
Partial. The paper evaluates on public simulated benchmarks LIBERO and RoboCasa, but the provided text does not mention released code, trained checkpoints, or the real-world Franka dataset, so full reproduction of the headline cross-embodiment results would likely be difficult.
Key figure
Figure 1 illustrates the two-stage pipeline: first learning an action prior from action trajectories alone using a flow-matching encoder-decoder, then reusing that prior during VLA training so visual-language features align to a structured action embedding space.