How do you make ACT move a robot arm smoothly?
TOS Robotics made ACT move more smoothly by predicting more often than a chunk lasts, blending each new chunk into the current plan, and leaving the targets the robot already holds untouched. On our FANUC arm, which takes a new joint target every 4 ms, the handover is sized for the slowest prediction, not the average. Smoothing has limits, and they are part of why our production cells moved on to a diffusion policy.
First setup
ACT, short for Action Chunking with Transformers, was our first learned controller, trained on demonstrations that our engineers record with the Teachbot. From camera images and the current joint angles it predicts a chunk of future joint positions. In June 2025 a chunk held 75 targets at 62.5 per second, 1.2 seconds of motion.
The FANUC controller takes a joint target every 4 ms. A planner called Ruckig limited speed, acceleration and jerk, the rate at which acceleration changes. The loop paused 0.1 seconds between predictions, and each new chunk replaced the old one.
What went wrong
Each chunk comes from a slightly different estimate of where the arm should be, so replacing one with the next made the target jump at the handover. Prediction time also varies. Over 1,000 episodes on our robot, in the master thesis of Quint Bakens, ACT took 17.1 ms on average and 97.9 ms at worst. The robot does not wait for the model, so the old plan has to stay valid until a late plan arrives.
What we changed
Between July and October 2025 we rebuilt the handover. With a new chunk every 0.1 seconds and chunks of 1.0 to 1.5 seconds, several chunks overlap at each moment.
Where a new chunk overlaps the current plan, each target is the half and half average of both. Only the last merged plan is kept, so older chunks fade by half at each step, a light version of the temporal ensemble proposed with ACT. The handover then runs in four zones.
| Zone | What happens |
|---|---|
| Protected | Targets already in the robot’s buffer, 3 by default, stay as they are |
| Extended freeze | Old plan kept for as many 4 ms ticks as the slowest prediction takes |
| Blend | Over the same number of ticks, old to new along a fifth-order curve, so speed and acceleration stay continuous |
| New plan | Only the new chunk |
The slowest prediction time is tracked live with a 10% margin. With a floor of 0.1 seconds the extended freeze is 22 ticks, 88 ms.
Results and limits
The handovers became smoother in our tests, but we recorded no smoothness number before and after.
Long chunks and heavy blending are smooth but react late, short chunks react fast but jerk. A master thesis on a moving conveyor tested one object at 0.15 m/s with 30 trials per setting. Changing only the merge weight moved ACT’s success from 30% at a weight of 0.95 on the newest chunk to 100% at 0.3 or less.
Blending cannot fix a model that changes its mind. In a lab task with two balls, a master thesis on action multimodality found that ACT could switch between the two equally good choices from one prediction to the next. Blending then averaged a left and a right prediction into a pick between the balls, which is in no demonstration, and sometimes the robot stopped above a ball and stayed there.
Questions and answers
What is action chunking?
Instead of predicting one next position, the model predicts a short series of future positions at once, called a chunk. The robot executes that chunk while the next prediction is computed.
Do TOS Robotics production cells still use ACT?
No. The production cells use a diffusion policy on point clouds, the 3D points measured by depth cameras, to predict where to pick and where to place. ACT remains a base for research by master students at TOS Robotics.
Keep reading
- Why we moved from ACT to a diffusion policy with point cloudsWe sample 50 candidate picks instead of one average. In nearly all cycles at least one of them was a working pick.
- What is imitation learning for robots, and when does it fit?Teaching a robot by showing it the task, used by TOS Robotics on a bell pepper line at 40 picks per minute per robot.