A
Perception Mismatch
Group events caused by lighting shifts, specular glare, unexpected object shadows, camera occlusion, clutter density, or surface texture differences.
Platform Capabilities
Sim2Real core features for deployment failure intelligence.
Sim2Real captures structured telemetry from real-world robot runs, groups failures by reported pattern tags, delivers actionable simulation parameter updates, and tracks validation across field deployments.
TL;DR: Sim2Real provides five core feature pillars: structured JSON telemetry intake, pattern-tagged failure clustering, reviewable simulator recommendations, a structured validation loop, and scoped robotics stack integrations. It works alongside your existing simulators and pipelines without replacing them.
Pillar 01
Telemetry Inputs
Sim2Real ingests structured JSON events from real robot runs. Media files such as camera video streams stay in your infrastructure and are referenced by URL or ID rather than uploaded.
Ingested Data Fields
- Task Outcomes: Success/failure status flags and event timestamps.
- Pattern Tags: Operator labels or automated failure-pattern flags.
- Confidence & Drift: Sim-vs-real confidence, success-rate, and Drift score metrics.
- Sensor Metadata: Compact sensor parameters, pose, and light reference metadata (JSON, not heavy media uploads).
- Identifiers: Robot, site, task, software build, and deployment IDs.
Clean Telemetry Boundary
By keeping heavy binary media in your own storage system and relying on structured JSON payloads, your team maintains data control while giving engineering teams legible failure signals to analyze.
Ingestion format: Append-only JSON REST API. Batch or live event stream scoped during pilot intake.
Pillar 02
Failure clustering
Sim2Real groups deployment failures by the pattern tags reported in telemetry so engineering teams can focus on systemic simulation gaps rather than ad hoc debugging.
B
Physics Mismatch
Group failures rooted in surface friction variance, contact compliance, inertial misalignments, actuator dynamics, or physical wear not represented in simulation.
C
Task & Sequence Mismatch
Isolate policy state errors, motion planning timeouts, trajectory hesitation, or edge-case execution sequences from raw sensor anomalies.
Untagged Events: Telemetry events without pre-assigned pattern tags land in an “unknown” cluster, allowing robotics engineers to review unclassified failures and determine if a new pattern cluster should be defined.
Pillar 03
Simulation & Training Recommendations
When failure patterns are identified, Sim2Real produces reviewable parameter update recommendations and starter parameter override patches for your simulation environment.
Reviewable Parameter Overrides
Sim2Real maps observed real-world discrepancies into starter parameter override patches (e.g. friction ranges, light angle variations, object pose distributions, clutter bounds). Engineering teams review these patches before applying them to training runs.
Engineering-Led Refinement
During a pilot, recommendations are refined with direct engineering support. Your team retains full control over which parameter overrides are merged into your simulator configuration or domain randomization pipeline.
Pillar 04
Field Validation Loop
Sim2Real closes the feedback loop by tracking whether updated simulation parameters improve robot performance in subsequent real-world deployments.
1
Capture Failure Pattern
Identify a repeatable failure pattern from grouped deployment telemetry events.
2
Apply Simulator Patch
Review and apply recommended simulation parameter overrides or synthetic scene adjustments.
3
Replay & Retrain
Re-evaluate or retrain the policy against the newly calibrated synthetic test scenario.
4
Compare Field Outcomes
Deploy the updated policy to the field and compare success rates against the original task boundary.
Pillar 05
Scoped Robotics Stack Integrations
Sim2Real is designed to complement existing robotics tooling, simulation engines, and software pipelines.
Pilot-supported
Isaac Sim
Isaac Sim
Pilot-supported
MuJoCo
MuJoCo
Scoped per engagement
ROS / ROS2
ROS / ROS2
Scoped per engagement
Omniverse
Omniverse
Scoped per engagement
Custom Telemetry APIs
Custom Telemetry APIs
Scoped per engagement
Proprietary Pipelines
Proprietary Pipelines
Pilot-supported: Works today with hands-on engineering support during a pilot engagement. Scoped per engagement: Feasibility is evaluated during commercial intake based on your team's event boundary and workflow; technical evaluation is not a guarantee of availability.
FAQ
Questions about Sim2Real features
What telemetry inputs does Sim2Real accept?
Sim2Real ingests structured JSON events containing task outcomes, failure flags and pattern tags, sim-vs-real confidence, success-rate, drift metrics, and compact sensor metadata references. Media such as camera streams is referenced by URL or ID, not uploaded.
How does failure clustering work?
Failures are grouped by reported pattern tags such as perception mismatch, physics mismatch, or task mismatch. Untagged events land in an “unknown” group, and engineers review whether a cluster represents a meaningful pattern.
What simulator updates does Sim2Real produce?
Sim2Real generates reviewable parameter override patches for MuJoCo and Isaac Sim during a pilot. Engineers review recommendations, apply the parameter overrides to simulation or training runs, and test whether real-world performance improves.
Which robotics integrations are supported?
MuJoCo and Isaac Sim exports are pilot-supported with hands-on engineering support during a pilot. ROS/ROS2, Omniverse, custom APIs, and proprietary telemetry pipelines are scoped per engagement; feasibility is assessed during intake, and evaluation is not a commitment.
Does Sim2Real replace our existing simulation tools?
No. Sim2Real acts as a calibration and failure-intelligence layer for the simulation engines (such as Isaac Sim or MuJoCo) and robotics frameworks your team already runs.
Get Started
Evaluate Sim2Real features with your team.
Request a pilot assessment to explore telemetry ingestion, failure clustering, and simulation recommendations for your robotics deployment.