Sim-to-real: mixing simulated and real-world demonstrations

The short version

• Simulation gives you unlimited, cheap, perfectly-labeled rollouts, invaluable for reinforcement learning and rare cases.
• The sim-to-real gap is where simulated physics and rendering diverge from reality, especially contact and friction.
• The winning recipe is rarely sim-only or real-only; it’s a deliberate mix, with real data anchoring what simulation gets wrong.

Real robot data is expensive and slow; simulated data is cheap and instant. That trade is why simulation is everywhere in reinforcement learning, and why “just train in sim” so often disappoints. The useful question isn’t whether to use simulation, but how to combine it with real demonstrations so the policy gets the volume simulation offers without inheriting the ways simulation lies.

What simulation is genuinely good for

In a simulator you can run thousands of rollouts in parallel, reset instantly, and get perfect labels, exact object poses, contact points, and success flags no real setup can hand you. That makes simulation the natural home for reinforcement learning, where a policy needs enormous numbers of trials to learn from reward. It also lets you manufacture coverage cheaply: rare configurations, adversarial clutter, and dangerous states you’d never want to reproduce on real hardware. For anything bottlenecked on trial count or edge-case coverage, simulation earns its place.

Where the gap opens up

The sim-to-real gap is the difference between simulated and real observations and dynamics. Two flavors dominate manipulation. The visual gap: rendered images don’t match real camera noise, lighting, and materials, so a vision policy trained on clean renders can be baffled by a real frame. The dynamics gap: contact, friction, deformation, and cable stiffness are exactly what manipulation depends on and exactly what simulators approximate worst. A policy that looks flawless in sim can fail on contact-rich tasks, insertion, cloth, anything where grams of force matter, because the physics it learned isn’t the physics it meets.

Closing it: randomization and real anchors

The classic tool is domain randomization: vary textures, lighting, masses, and friction across simulated episodes so reality becomes just one more variation the policy already handles. It works well for the visual gap and helps with dynamics, but it can’t invent physics the simulator doesn’t model. That’s where real demonstrations come in, not as a separate dataset, but mixed into the same training run. A modest fraction of real episodes anchors the policy to true contact behavior while simulation supplies volume and coverage. Tuning that ratio is the practical craft of sim-to-real, and it depends on how contact-heavy your task is.

Why the mix should live in one dataset

If simulated and real demonstrations live in separate silos, “what mix trained this policy?” becomes unanswerable, and that ratio is the single most important knob you have. On Neuracore, simulated and real episodes sit in the same versioned dataset, tagged by source. You set the blend for a training run, track which mix produced which success rate, and adjust with evidence instead of folklore. Simulation integrations feed rollouts in; real teleoperation data anchors them; both are just demonstrations to the trainer.