Every humanoid robot company faces the same bottleneck covered in the first two entries of this series — training data — and Tesla, Figure AI, and NVIDIA's broader partner ecosystem have converged on three genuinely different ways to generate it: teleoperation, where a human directly pilots the robot; video imitation learning, where the robot watches humans perform tasks; and synthetic data, where a world model generates entirely computer-created training scenarios. The honest 2026 picture is that no serious production system relies on just one. Each method solves a problem the other two can't, and the real engineering decision is how to layer them.
Teleoperation: Highest Fidelity, Doesn't Scale
Teleoperation — a human directly controlling a physical robot in real time, often through motion-capture suits or exoskeleton rigs — produces the highest-fidelity training data available, with what researchers describe as zero embodiment gap, since the action-state correspondence between the human operator's movement and the robot's actual physical response is perfect by construction. Open platforms like ALOHA and UMI, along with exoskeleton-based rigs used by companies like AgiBot and Fourier, have all built around this approach specifically for dexterous manipulation tasks where precision matters most. The problem is scale: a skilled teleoperator produces only 5 to 50 training episodes per hour depending on task complexity, and quality degrades as the operator fatigues. One robot, one operator is the practical ceiling, which is exactly why teleoperation-focused companies have spent their engineering effort on cost reduction and throughput rather than trying to scale the fundamental approach itself.
Video Imitation Learning: Tesla's Big Strategic Pivot
Tesla made a significant, publicly discussed strategy shift in mid-2025 when Ashok Elluswamy, who previously led Tesla's Autopilot program, took over Optimus and moved the company away from teleoperation toward learning directly from first-person human video instead. Workers now wear a custom camera rig — a helmet and backpack combination with five in-house cameras — and simply perform ordinary tasks naturally, like folding laundry or using tools, without any robot involved at all during data capture. Tesla engineer Mihir Dalal summarized the reasoning bluntly: "We can now do bi-manual, dexterous manipulation across a wide range of tasks with barely any data on these skills coming from teleoperation. As we know, teleop does not scale! But turns out human video does!" The longer-term goal is even more ambitious: Tesla has discussed eventually learning from ordinary third-person internet video, including YouTube how-to content, which would represent a genuinely different order of data scale than any teleoperation program could match. Figure AI has pursued a parallel path with its Helix vision-language-action model, which learns household tasks like bed-making by watching human demonstration video rather than direct robot teleoperation.
Synthetic Data: The Most Scalable, But Needs Grounding
Synthetic data generation, covered in more technical depth in the previous entry of this series through NVIDIA's Cosmos and GR00T-Dreams pipeline, is the most scalable approach by a wide margin, since it doesn't require any human — operating a robot or performing a task on camera — at all once the underlying world model is built. Tesla revealed its own version of this approach in November 2025, when Elluswamy described a "neural world simulator" at the ICCV conference that runs Optimus inside the same virtual environment used to train Tesla's Full Self-Driving system, built on the same underlying Niagara infrastructure. NVIDIA's own DreamGen research reports a striking result from this approach: robots trained partly on synthetic data achieved over 40% success on entirely novel tasks, starting from 0% success, without a single additional real-world demonstration. The catch is that synthetic data still needs grounding in physical reality to avoid what researchers call the sim-to-real gap — the performance drop when a policy trained in simulation meets an actual physical robot. Closing that gap typically requires domain randomization (varying lighting, textures, and friction during training), photorealistic rendering platforms like Cosmos, 3D Gaussian Splatting for scene reconstruction, and system identification to match a simulation's physical parameters to the real robot's actual hardware.
The Honest Caveat: Teleoperation Controversy in Public Demos
It's worth being direct about a real credibility issue in this space: several high-profile Tesla Optimus demonstrations used humans remotely controlling the robot without the company clearly disclosing that fact at the time, a pattern that drew sustained criticism. Tesla has stated that specific movements shown in other demonstrations, including a widely shared kung fu sequence, were genuinely AI-driven rather than teleoperated. As of Tesla's January 2026 earnings call, Musk himself acknowledged that despite earlier claims of 1,000-plus deployed units, no Optimus robots were doing verifiable "useful work" in factories yet, describing the program as "still very much at the early stages" — an important, unusually candid reality check on how much of any company's public robot footage reflects finished autonomous capability versus training-stage demonstration.
Why Nobody Uses Just One Method
The consensus among robotics data teams by 2026 is a layered strategy rather than a single winning approach: simulation and synthetic data for locomotion and whole-body control, where errors are cheap and volume matters most; exoskeleton-based teleoperation specifically for fine dexterous manipulation, where precision still beats synthetic data's current fidelity; and large-scale egocentric video for natural scene priors and broad task variety. Open cross-embodiment datasets like AgiBot World and Open X-Embodiment have emerged specifically to support this blended approach, letting companies pre-train on data collected across many different robot platforms before fine-tuning with targeted teleoperation on their own specific hardware's kinematics.
Training Methods at a Glance
| Method | Fidelity | Scalability | Best used for |
|---|---|---|---|
| Teleoperation | Highest — zero embodiment gap | Low — 5-50 episodes/hour per operator | Fine dexterous manipulation requiring precision |
| Video imitation learning | High — captures natural human movement | High — no robot needed during data capture | Broad task variety, bi-manual manipulation at scale |
| Synthetic data | Variable — depends on sim-to-real gap closure | Highest — no humans required once model is built | Locomotion, whole-body control, rare/novel scenarios |
Frequently Asked Questions
Why did Tesla switch from teleoperation to video imitation learning for Optimus?
Tesla shifted strategy in mid-2025 under Ashok Elluswamy because teleoperation doesn't scale — a single operator can only produce a limited number of training episodes per hour. Learning from first-person human video removes the robot from the data collection process entirely, enabling far greater data volume.
Is synthetic training data as reliable as real-world data?
It depends on how well the sim-to-real gap is closed. NVIDIA's DreamGen research found robots trained partly on synthetic data achieved over 40% success on novel tasks with zero additional real-world demonstrations, but techniques like domain randomization and photorealistic rendering remain necessary to make synthetic data reliably transfer to physical robots.
Have Tesla's Optimus demos actually used hidden human teleoperation?
Yes, at points. Several high-profile Tesla Optimus demonstrations used remote human control without the company clearly disclosing it at the time, though Tesla has stated other specific demonstrated movements were genuinely AI-driven rather than teleoperated.
Do robotics companies use only one training method?
No. The 2026 consensus is a layered approach combining simulation and synthetic data for locomotion, teleoperation for fine dexterous manipulation, and large-scale video for broad task variety, often built on shared cross-embodiment datasets like AgiBot World and Open X-Embodiment.