Imitation learning when there is more than one right way
In a master thesis at TOS Robotics, Lore Verstraete measured how an ACT imitation policy on a FANUC robot switches between two correct ways of doing a task when both appear in its demonstrations. Giving ACT its own planned actions as extra input cut the deviation of the first joint by 5 to 22%, but the number of failed tasks did not go down consistently.
Problem
When a task has more than one correct way, the demonstrations show different answers to the same situation, called action multimodality. ACT (Action Chunking with Transformers) can learn both answers and then switch between them from one prediction to the next. ACT also averages overlapping predictions, so “go to the left ball” and “go to the right ball” can become “go between the balls”, a movement that does not appear in the demonstrations. The result is hesitation, odd gripper poses or a livelock, where the robot stays stuck above a product and does not finish.
Setup
- a FANUC LR Mate 200iD/7L with a suction cup, driven at 250 commands per second through the TOS path planner
- the Teachbot for teleoperation, with a trigger that releases the vacuum
- four Intel RealSense D405 cameras, two on the gripper, one above and one at the side
- a belt that brings two identical 90 mm balls to two fixed positions, where the robot picks both and chooses which one first
300 demonstrations were recorded with the Teachbot, half starting left and half right, 240 for training and 60 for validation. Each working model ran the task 100 times in a row.
Method
In this setup ACT predicts a 1 s chunk of movement every 0.15 s and gets no input about what it planned before. The thesis added the next f planned actions as extra input, so the model knows which ball it was heading for. Three of eight settings did not work: the robot did not release the ball or hovered above it.
Consistency was measured as the mean deviation between the predictions for the first joint, which decides left or right, and the position that joint actually reached. Lower means less switching.
Results
| Look-ahead given as input | Mean deviation, first joint | Failed tasks out of 100 |
|---|---|---|
| None, standard ACT | 0.53 degrees | 7 |
| 0.04 s | 0.47 degrees | 1, but it always chose the same side |
| 0.2 s, two settings | 0.44 and 0.46 degrees | 11 and 7 |
| 0.4 s | 0.50 degrees | 13 |
| 0.9 s | 0.41 degrees | 8 |
Each working setting deviated 5 to 22% less than standard ACT. Failed tasks did not fall consistently, and livelocks still occurred. Results varied between test sessions, which the author attributes to uncontrolled disturbances in the setup.
The task was simplified on purpose and only ACT was evaluated. No deterministic criterion was found that predicts whether a switch recovers or ends in a livelock. The production cells of TOS Robotics now use a diffusion policy, see diffusion policy versus ACT.
Full title: Improving Imitation Learning from Visual Observation for Tasks with Action Multimodality. Carried out at TOS Robotics.
Questions and answers
Why does this switching matter on a packing line?
A robot that hesitates between two products loses cycle time, and one that gets stuck between them needs an operator. Every new prediction is another chance to switch, so a robot that predicts more often to move faster needs this consistency more.
How does this relate to diffusion policies?
The thesis names diffusion policies as an approach that captures several valid behaviours better than ACT, with slower inference, but did not test them. The production cells of TOS Robotics now use a point cloud diffusion policy with a separate path planner.
What does the thesis propose as next steps?
Systematic hyperparameter tuning, a comparison with real-time action chunking, and choosing the latent variable of ACT at run time instead of fixing it at zero, so the policy commits to one way of doing the task.