Cortex: A Bidirectionally Aligned Embodied Agent Framework for Long-horizon Manipulation

Imagine you're trying to build a robot that can perform complex tasks, like cooking or assembling furniture. These tasks often require planning several steps ahead, but many current models only react to what they see right now, which limits their effectiveness in long-term scenarios. This is a problem because when a robot can't plan ahead, it can get stuck or make mistakes when transitioning between different subtasks — this is what's called planning ambiguity.

To tackle this, Cortex introduces a new way to connect high-level plans with the actual actions the robot needs to take. Instead of just reacting to the current situation, Cortex breaks down tasks into smaller, manageable pieces, called skill primitives, and ensures that these pieces are easy to execute. It also improves how the robot learns from data by using a smart sampling strategy that helps it better understand when to switch between tasks. This means that Cortex can handle complex tasks more effectively than previous models, which often struggled with the same issues.

In practical terms, Cortex allows robots to complete tasks they've never seen before, like conducting multi-stage chemistry experiments, just by combining its planning capabilities with its execution skills. This is a big step forward for anyone building robots that need to perform a variety of tasks without extensive retraining.

Cortex introduces a new framework that effectively bridges high-level planning and low-level execution in VLA models.

The claims are supported by strong evaluations against monolithic baselines and extensive data generation.