Robot Learning · Nov 2026 · In simulation

Robot Learning: what the numbers show, and what they don't

We study planner-based manipulation with a small amount of learning in simulation, on tasks from the KinDER benchmark family (arXiv:2604.25788). This page reports our own internal evaluation panel. It is not an official benchmark result.

Conceptual illustration generated with AI. Not experimental data or a depiction of the actual equipment.

Method

Each variant runs 250 episodes: environment seeds 0 to 249, one episode per seed. Planning uses a sampling-based planner (RRT) whose seed is derived from the environment seed and the step number. With unchanged code and the same seed, the simulation is deterministic on the seeds we rechecked. Episodes that run close to the 95 second cap depend on machine load and can flip, and a small code change can flip single episodes. We added a 95 second wall-clock cap per episode. Episodes over the cap count as failures, so for the same protocol without the cap the results would be equal or higher. This says nothing about other protocols or other seeds.

Results

On the Cluttered family, success falls as the number of obstacles grows: 249 of 250 episodes with 1 obstacle, 233 of 250 with 10, and 152 of 250 with 25.

Cluttered family, internal panel, 250 episodes per variant, final version (v18). Source: raw per-episode results from our evaluation run, 9 November 2026.

The hardest case, 25 obstacles, improved from 138 to 152 successes out of 250 after we let the policy retry a failed placement up to 5 times instead of 2. The comparison is paired on the same seeds.

Cluttered, 25 obstacles: same 250 seeds before and after the placement-retry change. A gain of 14 episodes. One configuration, one run.

Where the remaining failures come from

Of the 98 failed episodes at 25 obstacles, 47 were failed grasps, 26 hit the wall-time cap, 15 had no valid drop spot and 10 were imprecise placements.

Failure classes at 25 obstacles (v18), as a share of all 250 episodes.

Limits

Simulation only. No real-robot result.
Not comparable one-to-one with the official benchmark: the paper uses 5 seeds of 50 episodes, we use 250 different seeds. For the Cluttered, Obstruction and Storage families the paper publishes no numbers, so we have no official reference and make no state-of-the-art claim.
Development and evaluation used the same 250 environment seeds (0 to 249). A held-out set was never tested, so the numbers may be optimistic for fresh seeds. We did not measure the size of any such effect.
The wall-time cap is our addition and can only lower the reported success under this protocol.
Whether a clearing sequence exists for every hard 25-obstacle scene is open. We have no proof either way.

Code, raw results and a full write-up are not publicly available.