Skip to main content
Roviral

A 16-Joint Robot Hand Learns In-Hand Object Manipulation in Simulation

Simulation-trained control lets a LEAP robot hand reposition objects between its fingers; a motion safeguard reduces drops while slightly lowering task success.

A 16-Joint Robot Hand Learns In-Hand Object Manipulation in Simulation

A robot hand that can reposition an object between its fingers has to solve two problems at once: reach the requested pose and keep the object from slipping. Researchers at the University of California, Berkeley and Davis trained a 16-joint LEAP robotic hand to perform these in-hand movements using simulation alone, then added a software safeguard that trades some task success for fewer drops.

Nidhi Agarwal

The distinction matters for manipulation. A hand that can only grasp and hold an object still has limited ability to adjust it for a later operation. Moving an object within the grasp requires coordinated finger motion and control that accounts for how each object’s shape affects contact. The researchers’ system was tested on four objects, with a separate controller for each.

Training the hand in simulation

The training process began with hundreds of thousands of computer-generated grasp examples. Early in training, the simulated fingers had high friction, which made objects easier to retain. As the model improved, that assistance was gradually reduced so the hand had to cope with conditions closer to those expected in reality.

Robot hand reaches diverse in-hand object poses at any wrist orientation. Credit: arXiv (2026). DOI: 10.48550/arxiv.2610.37652

The team also measured the motor behaviour of a real hand and incorporated those measurements into the simulation. That gives the training process a physical reference for how the hand’s motors respond, alongside the simulated examples. The reported approach did not use human demonstrations: the hand learned to move and rotate objects through its fingers from simulation-based training.

Simulation can expose a controller to many grasp and movement situations without requiring a person to demonstrate each one on hardware. But a simulated hand still has to transfer its learned actions to a physical system. Incorporating measured motor behaviour addresses one part of that gap; the tests on real objects provide an initial check of whether the resulting controls can carry out the requested movements.

Object-specific control and a motion safeguard

The researchers tested the hand with a real light bulb and three 3D-printed plastic objects: a knife, a wine glass and a duck. Their differing shapes and handling requirements meant that each object needed its own controller. The reported results therefore describe a set of object-specific demonstrations, rather than one controller shown to generalize across all four objects.

Nidhi Agarwal

A second software system checked each controller’s planned motion and made small adjustments when it judged that a change could help prevent a drop. Its design goal was to preserve the requested movement as much as possible while making the trajectory safer. That means the safeguard can affect whether the hand completes a target: an adjusted path may retain the object but fail to reach the pose the controller originally planned.

This is a practical control trade-off. For tasks where losing the object carries a high cost, a slightly less direct movement may be preferable. Where exact pose completion matters more, the added caution can reduce the rate of successful targets. The reported evaluation shows both effects rather than treating drop prevention as a free improvement.

Results across 4,800 movement requests

The evaluation covered 4,800 requests involving movements of the four objects in six wrist orientations. If an object fell or became stuck, the hand was reset. Without the safeguard, the system reached 95.2% of its targets. With it enabled, target completion fell to 91.5%.

A separate set of 1,200 movement sequences measured drops: the hand dropped objects 16 times without the safeguard and four times with it. These figures show a substantial reduction in drops in that test, alongside lower target completion in the larger evaluation. The two reported counts refer to different evaluations, so they should be read as separate measures rather than combined into a single success rate.

The results suggest that a controller for dexterous manipulation benefits from being judged on more than whether it reaches a target. Retaining the object, avoiding a stuck state and completing the intended pose all matter, and the safeguard shifts the balance among them. The reported tests don’t establish how the approach performs on untested objects or in longer sequences of handling.

What the results leave open

For teams developing robotic manipulation, the work points to a useful division of control: a task controller plans the desired object movement, while a separate layer can modify that plan when grip security is at risk. The measured results make the cost visible—fewer drops, but also a lower percentage of reached targets. Whether that balance suits a particular application will depend on the task’s tolerance for dropped objects and incomplete poses.

The next engineering challenge is reliability across a wider range of object shapes and handling conditions. The four test objects provide varied examples, but each had its own controller, and the reported evaluation does not show general-purpose control across arbitrary objects. For now, the strongest takeaway is that simulation-trained in-hand manipulation can be paired with a software layer that reduces drops, while the remaining gap is dependable performance across more objects and movements.

Source: Nidhi Agarwal · www.electronicsforu.com

Filed alongside