1
Lighting variance
Glare or low light reported on the event. Three or more can produce a starter lighting override.
Platform Capabilities
Sim2Real core features for deployment failure intelligence.
Sim2Real captures structured telemetry from real-world robot runs, groups failures by reported pattern tags, delivers actionable simulation parameter updates, and tracks validation across field deployments.
TL;DR: Sim2Real provides five core feature pillars: structured JSON telemetry intake, pattern-tagged failure clustering, reviewable simulator recommendations, a structured validation loop, and scoped robotics stack integrations. It works alongside your existing simulators and pipelines without replacing them.
Pillar 01
Telemetry Inputs
Sim2Real ingests structured JSON events from real robot runs. Media files such as camera video streams stay in your infrastructure and are referenced by URL or ID rather than uploaded.
Ingested Data Fields
- Task Outcomes: Success/failure status flags and event timestamps.
- Pattern Tags: Operator labels or automated failure-pattern flags.
- Confidence & Drift: Sim-vs-real confidence, success-rate, and Drift score metrics.
- Sensor Metadata: Compact sensor parameters, pose, and light reference metadata (JSON, not heavy media uploads).
- Identifiers: Robot ID, site name, and task name; other identifiers (software build, scene, deployment IDs) go inside
rawSensor.
Clean Telemetry Boundary
By keeping heavy binary media in your own storage system and relying on structured JSON payloads, your team maintains data control while giving engineering teams legible failure signals to analyze.
Ingestion format: Append-only JSON REST API, one event per request (send batches as sequential requests).
Pillar 02
Failure clustering
Sim2Real groups deployment failures by the pattern tags reported in telemetry so engineering teams can focus on systemic simulation gaps rather than ad hoc debugging.
2
Object tilt
Tray or object angle reported as affecting placement. Three or more can produce a starter contact and friction override.
3
Clutter occlusion
Partially hidden grasp targets. Three or more can produce a starter clutter-density override.
4
Physics mismatch
Friction or contact the event tags as diverging from simulation. The workspace lists the group. The starter file does not add a separate patch for this tag.
5
Sensor mismatch
Sensor readings the event tags as diverging from simulation. The starter export does not rewrite a sensor noise model.
6
Unknown
No recognized tag, or a value outside the list. Those events stay here for review. The product does not add a task-mismatch category.
Pillar 03
Simulation & Training Recommendations
When failure patterns are identified, Sim2Real produces reviewable parameter update recommendations and starter parameter override patches for your simulation environment.
Reviewable Parameter Overrides
For lighting variance, object tilt, and clutter occlusion, the export can include a starter lighting, contact, or clutter-density override once that tag appears at least three times. It does not rewrite pose ranges or a sensor noise model. Engineering teams review the file before applying it.
Engineering-Led Refinement
During a pilot, recommendations are refined with direct engineering support. Your team retains full control over which parameter overrides are merged into your simulator configuration or domain randomization pipeline.
Pillar 04
Field Validation Loop
Later events stay in the workspace so the team can compare the next run with the earlier one. The stored record does not by itself prove that a parameter change caused a difference.
1
Capture Failure Pattern
Identify a repeatable failure pattern from grouped deployment telemetry events.
2
Apply Simulator Patch
Review and apply recommended simulation parameter overrides or synthetic scene adjustments.
3
Replay & Retrain
Re-evaluate or retrain the policy against the newly calibrated synthetic test scenario.
4
Compare the next run
Post the next events and compare the reported outcome with the earlier run. A change in the reported success rate is not by itself proof of cause.
Pillar 05
Scoped Robotics Stack Integrations
Sim2Real is designed to complement existing robotics tooling, simulation engines, and software pipelines.
Pilot-supported
Isaac Sim
Isaac Sim
Pilot-supported
MuJoCo
MuJoCo
Scoped per engagement
ROS / ROS2
ROS / ROS2
Scoped per engagement
Omniverse
Omniverse
Scoped per engagement
Custom Telemetry APIs
Custom Telemetry APIs
Scoped per engagement
Proprietary Pipelines
Proprietary Pipelines
Pilot-supported: Works today with hands-on engineering support during a pilot engagement. Scoped per engagement: Feasibility is evaluated during commercial intake based on your team's event boundary and workflow; technical evaluation is not a guarantee of availability.
FAQ
Questions about Sim2Real features
What telemetry inputs does Sim2Real accept?
Sim2Real ingests structured JSON events containing task outcomes, failure flags and pattern tags, sim-vs-real confidence, success-rate, drift metrics, and compact sensor metadata references. Media such as camera streams is referenced by URL or ID, not uploaded.
How does failure clustering work?
Each failure is stored under the pattern tag on the event. Recognized tags are lighting variance, object tilt, clutter occlusion, physics mismatch, sensor mismatch, and unknown. Any other value is stored as unknown.
What simulator updates does Sim2Real produce?
For lighting variance, object tilt, and clutter occlusion, Sim2Real can export a starter override for MuJoCo or Isaac Sim once that tag appears at least three times. Engineers review the file and decide what to apply. A later change in the reported success rate is not by itself proof of cause.
Which robotics integrations are supported?
MuJoCo and Isaac Sim exports are pilot-supported with hands-on engineering support during a pilot. ROS/ROS2, Omniverse, custom APIs, and proprietary telemetry pipelines are scoped per engagement; feasibility is assessed during intake, and evaluation is not a commitment.
Does Sim2Real replace our existing simulation tools?
No. Sim2Real acts as a calibration and failure-intelligence layer for the simulation engines (such as Isaac Sim or MuJoCo) and robotics frameworks your team already runs.
Get Started
Evaluate Sim2Real features with your team.
Request a pilot assessment to explore telemetry ingestion, failure clustering, and simulation recommendations for your robotics deployment.