Transformer Transformer: A Unified Model for Motion-Conditioned Robot Co-Design

Not each robots are created adjacent — but what if you could creation 1 for a circumstantial task? We propose Transformer Transformer, a unified exemplary that does precisely this: manus it a manipulation demonstration, and it generates a complete robot — each link, joint, motor, and inertial spot — optimized for that motion. We fabricated 1 specified creation for cloth flinging connected an ALOHA2 bimanual platform; it reduced search correction by 73% and max associated velocity by 30% versus the original.

Behind that consequence is simply a diffusion transformer trained connected RoboTokens, a unified tokenization of robot embodiments, states, and actions. The aforesaid architecture spans embodiment spaces (wheeled bimanual, quadrupeds, humanoids) and usage cases (embodiment generation, cross-embodiment control). Rather than overfitting to 1 reward function, it is simply a dynamics exemplary whose reward-agnostic predictions are converted into reward-specific worth predictions astatine conclusion time, past utilized to steer embodiment diffusion done a process we telephone Dynamics Self-Guidance. Experiments crossed 3 creation spaces show zero-shot optimization of unseen rewards and trajectories, improving capacity and runtime complete an evolutionary baseline.

A robot's embodiment decides what tasks it tin do well. We move effort into making robots smarter — bigger datasets, amended algorithms, much tin argumentation architectures — but a bad embodiment tin hamper moreover a awesome policy. You tin cod each the tossing information successful the world, but if your robot's style is far from optimal, it mightiness flip itself alternatively of the ball.

So let's framework the problem concretely. Given arsenic input a target end-effector mobility and a reward function, we want to make a complete robot embodiment for the task — by complete we mean each link, joint, motor, and inertial property, positive a controller that drives the full thing. We telephone this motion-conditioned robot co-design.

If we want 1 exemplary to co-design any robot, the web needs a shared vocabulary that spans each embodiment. RoboTokens are that vocabulary: typed tokens for each link, joint, motor, state, and action — organized truthful a azygous series describes some a robot's embodiment (time-invariant) and its dynamics (time-varying).

RoboTokens has the pursuing cardinal creation goals:

• Complete. A RoboToken series captures everything astir a rigid articulated robot: 5 embodiment token types (links, fixed joints, rotating/sliding joints, shot joints, motors) positive state and action tokens.
• Flexible. Tokens reference each different by ID — a associated token knows the 2 nexus tokens it connects — truthful the aforesaid format handles a 6-DoF limb aliases a 35-DoF biped.
• Consistent. The tokenizer canonicalizes redundant spatial offsets and stores inertial / toggle shape information successful a azygous frame, reducing the variance the exemplary has to study through.
• Extensible. New token types slot successful without changing the architecture — we add target end-effector airs tokens for trajectory search by pointing them astatine the applicable end-effector and timestep IDs, nary retokenization of the remainder of the robot required.
• Optimizable. Unlike autoregressive matter formats for illustration MJCF, diffusing continuous-valued RoboTokens gives some global controllability (later tokens power earlier ones across denoising steps) and differentiability — the 2 ingredients needed for gradient-based reward optimization astatine conclusion time.

The exemplary is simply a diffusion transformer (DiT1) trained on RoboTokens pinch a DDIM4 sound schedule. Noise is added to each input token except the ones we information connected — and by changing which tokens are masked, the aforesaid web wears different hats. Mask everything to get an unconditional robot generator. Mask the actions and you get a cross-embodiment controller5,6. Mask the embodiment and you get a motion-conditioned robot designer. Same weights, jointly trained connected dynamics.

(And astir the pun: the first “Transformer” refers to robots whose embodiments can change. The 2nd refers to the self-attention architecture 🥸)

Now the existent co-design problem: really do we make a high-reward robot for a reward usability the exemplary has ne'er seen during training?

Here is the trick. The aforesaid exemplary predicts some the embodiment and its dynamics, truthful we tin feed those predictions into the user's reward usability and get a predicted reward. With predicted rewards successful hand, we tin tally the exemplary galore times successful parallel, foretell the reward for each candidate, and return the champion one. That is best-of-N sampling, but astatine GPU speed.

We tin do better. Because the predicted reward is simply a differentiable usability of the embodiment tokens, we tin return its gradient and inject it backmost into the diffusion process — pushing samples toward higher reward astatine each denoising step, pursuing classifier-guided DDIM11. The exemplary is, successful effect, asking itself: “how should this robot change to summation the reward?” We telephone this Dynamics Self-Guidance.

Take a randomly-generated quadruped that falls during a move toss. Ask our exemplary to make a robot for the aforesaid trajectory pinch only a search reward and it lowers the halfway of wide and widens the support polygon — unchangeable creation choices it discovered arsenic a side-effect of trying to maximize tracking reward it has ne'er seen.

Now adhd a torque-penalty word and the optimization scenery shifts. The exemplary returns a smaller, lighter robot — still search the motion, but utilizing 30% little mean torque. Add a size punishment and the scenery shifts again. This is what group mightiness telephone zero-shotting rewards: train connected dynamics modeling, optimize astatine conclusion for immoderate reward you tin constitute down.

Manipulation isn't 1 motion; it's a distribution of motions. We want to optimize a robot for an full task — say, crockery washing — not a azygous trajectory. Diffusion composition14,15 lets america do precisely this: tally the exemplary successful parallel pinch different trajectories arsenic conditioning, mean the predicted sound at each diffusion step, and you get a generator optimized for each of them simultaneously.

We tested this connected the publicly-available UMI bimanual dish-washing dataset3, holding retired 26 validation trajectories. Transformer Transformer generated robots optimized complete the full held-out distribution — navigating some the discrete abstraction (spine designs: fixed, sliding, bending) and the continuous abstraction (arm dimensions, mounting offsets) — and the aforesaid model past served arsenic the cross-embodiment controller to validate each generated design.

We comparison against 2 baselines: Random (sample a robot from the procedural design space) and CMA-ES16, a black-box optimizer wide utilized successful co-design. CMA-ES rolls retired campaigner robots successful MuJoCo17 pinch the creation space's controller (Mink18 for fixed-base / bimanual; an RL expert7 for legged robots), past biases the adjacent procreation toward higher-reward samples.

We springiness each methods the aforesaid task: make a robot for a target trajectory and reward, power it, study the achieved reward.

To stress-test the full strategy connected hardware, we picked move cloth unfolding connected ALOHA2. Flinging20 punishes embodiment mistakes done some kinematics (the limb has to scope complete the cloth astatine precocious speed) and unmodeled dynamics (aerodynamic drag, cloth weight, friction). The original ALOHA creation struggled to moreover way the trajectory.

We asked our exemplary to optimize the ALOHA creation for the “Tracking Velocity” reward, then fabricated the resulting design. Two things changed compared to the original:

• Longer nexus lengths. Just capable to kinematically scope crossed the flinging trajectory, but ray capable to enactment wrong the Dynamixel centrifugal torque envelope.
• Inverted mounting. The arms bent upside-down down the workspace, enabling an energy-efficient underarm plaything alternatively of an overhead fling.

Geometry and scene. RoboTokens presently screen primitive-based geometry, not arbitrary meshes aliases deformables, and don't yet encode the surrounding segment aliases interaction targets. Extending to analyzable geometry and segment discourse is simply a clear adjacent measurement toward afloat task-conditioned co-design.

Controller transfer. Our cross-embodiment controller correlates pinch the RL oracle at Pearson r = 0.53 connected diffused embodiments — promising but pinch headroom. Some outlier embodiments expose wherever the controller fails to generalize.

Test-time scaling plateau. Like different guided-diffusion methods, our test-time compute gains plateau aft astir a minute. Unlike LLMs19, more conclusion clip stops helping past a point. Improving reward-prediction accuracy and alternative conclusion schemes are unfastened directions.

Data procreation cost. RL-based whole-body controller information procreation is costly (one RL master per discrete creation choice, 16 hours of A100 per policy). Scaling Transformer Transformer to caller creation spaces hinges connected reducing this cost.