What robots need to learn from

Video of skilled people working is abundant and valuable. It shows what scenes look like, how tasks are structured, and how real work goes wrong. But a policy that controls a robot must output actions in that robot's own command space, and video records none.

Recovering actions from video means inferring poses, contacts and forces through occlusion, then retargeting a human body onto different kinematics. That can be a useful research direction; it is not a property of the footage. We therefore sell authentic footage and robot demonstrations as separate products, and never imply one substitutes for the other.

Imitation-learning policies learn a mapping from observations (images, joint state) to actions (often short chunks of future joint targets). Each training example asserts: given what the robot saw and felt at time t, this is what it was commanded to do next. If cameras and joint states are misaligned in time, or the calibration drifted, that assertion is false, and the model learns it anyway.

• Commanded vs. measured. Leader commands and follower state differ by latency and compliance. We record both rather than choosing one silently.
• Pinned embodiment. Firmware, calibration and gripper geometry are versioned with every episode; a changed gripper is a different dataset.
• Measured, not imagined, physics. Force/torque is delivered only when a sensor measured it.
• Two arms are not a humanoid. Bimanual data is valuable for support, slack and packaging coordination; it does not establish humanoid-hand or whole-body transfer.

Once a policy handles the nominal case, more nominal demonstrations add little. What it lacks is coverage of the places it fails: unusual poses, clutter, partial seating, lost slack, recoveries after a slip. Those are rare in uncurated collection and expensive to stumble upon.

Data efficiency is therefore an acquisition problem as much as a modeling one. Our working hypothesis is simple and testable: for a fixed collection budget, demonstrations selected from a frozen policy's categorized failures improve held-out performance more than the same budget of untargeted collection. It may be wrong for a given task and budget. The protocol below is designed so that we find out either way.

Labor arbitrage only matters after acceptance. A cheaper hour can lose to a costlier site with better active time, reset discipline and fewer rejected episodes, so we measure cost per accepted episode, per task.

In software, models increasingly help generate the data that trains their successors. The physical analogue is harder: every new example must be performed in the world, reset, and checked. Our long-term thesis is that infrastructure which makes physical data acquisition specifiable, verifiable and model-directed is the precondition for that loop to close.

We have not demonstrated recursive self-improvement, and we do not describe behavior-cloning updates as reinforcement learning. The ladder below is how we think about the path; and where we actually are.

The experiment tests one narrow hypothesis. It does not test autonomous science or cross-embodiment transfer.

1. Define a versioned SOP, object IDs, fixture geometry, sensor schema, permitted recovery policy, attempt limits and an objective terminal check.
2. Freeze baseline checkpoint M0 with its training manifest, software, firmware and calibration versions, and seal diagnostic and final holdout splits.
3. Diagnose failures by mode (grasp, orientation, placement, coordination, perception, recovery), counting aborted and interrupted trials.
4. Allocate equal operator-minutes and training compute to a targeted arm and a random, coverage-matched control.
5. Train M1-target and M1-control from the same M0 with identical optimizer, steps, augmentations and seeds.
6. Evaluate with randomized initial conditions and order, the same scenario strata and independent outcome checks.
7. Iterate by selecting the next batch from M1 errors on a separate diagnostic set. Never train on final-test failures.

A proposed minimum is 100 randomized attempts per policy per task, across two tasks, with at least three training seeds per update policy where budget allows. This is a planning design, not completed work or a guarantee of statistical power.

We hold ourselves, and our partners, to the following:

• Illustrative figures are labeled as illustrative, everywhere they appear.
• We report cash, accepted delivery, contracted minimums and pipeline as different things.
• A negative result is published, and the acquisition rule revised. We never manufacture a rising curve.
• We do not call a system RL unless a policy is actually updated from rewards or value estimates.
• Humanoid or cross-embodiment transfer requires a separate experiment on the target embodiment.
• Introductions are not authorized sites, and desk research is not a supply network.