A
Perception mismatch
Flag failures caused by lighting shifts, occlusion, glare, clutter, texture changes, or unexpected object appearance.
Product Architecture
See how Sim2Real turns deployment evidence into better training data.
Sim2Real captures what happened in the field, identifies where simulator assumptions broke, and creates updated synthetic conditions for the next training pass.
TL;DR: Sim2Real is a sim-to-real platform that ingests deployment telemetry from real robot runs, groups failures by the pattern tags your telemetry reports, and returns simulation parameter updates so your next training pass targets the right assumptions. It works alongside ROS/ROS2, Isaac Sim, Omniverse, and MuJoCo — it does not replace them.
Data Capture
How deployment telemetry is captured
Sim2Real ingests structured JSON events describing real robot execution: task outcomes, failure tags, sim-vs-real metrics, and compact sensor metadata. Media such as camera footage stays in your systems and is referenced, not uploaded.
Telemetry inputs
- Task outcomes, failure flags, and failure-pattern tags
- Sim-vs-real confidence, success-rate, and drift metrics
- Compact sensor metadata and references (JSON, not media uploads)
- Robot, site, task, and deployment identifiers
Why it matters
Robotics teams often know that a deployment failed, but not which part of the simulation model stopped matching reality. Sim2Real makes those mismatches legible enough to act on.
What you send
Enough context to explain one deployment task.
- Structured task outcomes and timestamps
- References or metadata for camera, force-torque, IMU, or controller context
- Robot, software, scene, and deployment identifiers
- Operator labels or incident notes when available
Setup: JSON API mapping scoped during pilot intake. Owner: a robotics or platform engineer who understands the event boundary.
What you get back
A prioritized path to the next simulation test.
- Grouped failure patterns with supporting evidence
- Affected deployments, robots, and task context
- Reviewable simulation or training recommendations
- A validation record for the next field run
Cadence: aligned to the team’s review loop. Owner: the engineer responsible for approving and testing the change.
Robot deploymentTelemetry APIFailure clustersSimulator correctionValidation run
Failure Analysis
How failures are grouped
Failures are grouped by the pattern tags your telemetry reports — untagged events land in an “unknown” group — so the training loop can update the right assumptions instead of treating every failure as generic noise.
B
Physics mismatch
Track failures rooted in friction, contact behavior, inertial differences, compliance, or surface wear that were not represented well in simulation.
C
Task mismatch
Separate policy, sequencing, or edge-case task errors from perception and environment issues so teams can target the right next action.
Simulator Updates
How simulation parameters are updated
Once discrepancies are identified, Sim2Real produces starter parameter override patches for MuJoCo and Isaac Sim, refined with engineering support during the pilot, for the next iteration.
1
Map observed discrepancies
Translate real scenes into structured changes such as friction, lighting, clutter density, object pose, and sensor noise.
2
Export starter override patches
Export starter parameter override patches for MuJoCo or Isaac Sim that target the failure patterns reported in deployment.
3
Design retraining scenarios with engineering support
Scenario combinations are designed with engineering support during the pilot and routed into your existing training workflow.
4
Measure improvement
Compare readiness, failure rate, and repeatability over time to see whether transfer quality is improving.
Integrations
Supported robotics stack integrations
Sim2Real is positioned to work alongside the tooling robotics teams already depend on for simulation, orchestration, and policy evaluation.
Scoped per engagement
ROS / ROS2
ROS / ROS2
Pilot-supported
Isaac Sim
Isaac Sim
Scoped per engagement
Omniverse
Omniverse
Pilot-supported
MuJoCo
MuJoCo
Scoped per engagement
Custom APIs
Custom APIs
Scoped per engagement
Telemetry pipelines
Telemetry pipelines
Pilot-supported: works today with hands-on engineering support during a pilot. Scoped per engagement: not a standard offering; feasibility is assessed during scoping, and evaluation is not a commitment.
Operational Outcome
Why the product reduces failure rates and pilot delays
Without a structured sim-to-real loop, teams collect more data than they need, retrain more broadly than necessary, and burn time onsite figuring out why a robot did not behave as expected. Sim2Real narrows that loop by identifying the most meaningful differences between synthetic assumptions and deployed reality.
Lower failure rates
Teams can focus on the scenarios most likely to break performance in production.
Fewer pilot delays
Deployment evidence turns into a structured backlog of calibration work instead of a vague debugging spiral.
Less real-world data overhead
Existing telemetry becomes more valuable because it feeds targeted simulator updates.
Cleaner executive reporting
Decision makers get repeatable signals around transfer quality and readiness.
Why not manual?
Keep engineering judgment. Remove the evidence scavenger hunt.
| Decision step | Manual workflow | Sim2Real workflow |
|---|---|---|
| Collect evidence | Search logs, videos, tickets, and notes separately | Use one scoped event boundary and linked deployment context |
| Find repeatable patterns | Rely on ad hoc queries and team memory | Review grouped failure clusters and supporting events |
| Choose a simulator change | Translate symptoms into scene or training changes by hand | Review a recommendation tied to the observed pattern |
| Validate the result | Compare runs with inconsistent documentation | Record the targeted failure, change, and next deployment outcome |
Use cases
Start with one expensive, repeatable failure pattern.
Failure patternMissed grasps under pose, clutter, or lighting variance
WorkflowCluster task failures, review evidence, vary the matching simulation conditions
KPI to monitorTask success within the same pick boundary
Failure patternFalse rejects or misses under surface and illumination changes
WorkflowGroup visual conditions, review examples, extend scene or data variation
KPI to monitorError mix on the same inspection class
Failure patternLocalization or planning degradation in changed environments
WorkflowCluster context shifts, identify common conditions, replay targeted scenarios
KPI to monitorMission completion within the same route class
Security and procurement
Scope the data boundary before telemetry moves.
Sim2Real uses authenticated per-user workspace access, API-key telemetry endpoints, server-side route guards, and browser security headers in the current application.
Review the privacy policyQualified during pilot intake
- Fields, files, and sensor references included in the telemetry schema
- Retention, deletion, and operational review requirements
- Deployment environment and integration responsibilities
- People authorized to review workspace outputs
Operator
Developer312, a subsidiary of NIGHT LITE USA LLC
Qualified pilots are handled as direct engineering engagements: define the event boundary, inspect the artifact, and agree on a validation plan before broader rollout claims are made.
Certifications, team-level isolation, and custom retention are not implied by this page; requirements are documented during commercial scoping. Sim2Real is currently cloud-hosted: VPC, on-premise, regional, and air-gapped deployments are not standard offerings and are evaluated individually for qualified enterprise engagements. Evaluation does not guarantee availability.
FAQ
Product questions engineering teams ask
Does Sim2Real replace our existing simulator?
No. Sim2Real is a calibration and retraining layer for whatever simulator your team already runs; it does not replace it. MuJoCo and Isaac Sim exports are pilot-supported with hands-on engineering support during a pilot; ROS/ROS 2, Omniverse, and custom stacks are scoped per engagement.
What kinds of telemetry does Sim2Real ingest?
Structured JSON events: task outcomes, failure flags and pattern tags, sim-vs-real confidence, success-rate, and drift metrics, plus a compact raw-sensor metadata field. Media such as camera streams is referenced, not uploaded. The schema is JSON-first and append-only.
How are failures grouped?
Sim2Real groups failures by the pattern tags your telemetry reports — for example perception mismatch (lighting, occlusion, glare, clutter), physics mismatch (friction, contact, compliance, surface wear), or task mismatch (policy, sequencing, edge cases). Untagged failures are grouped as “unknown”, and engineers review whether a cluster is meaningful.
What does the retraining output look like?
Starter parameter override patches for MuJoCo and Isaac Sim (for example lighting, contact friction, and clutter density overrides), refined with engineering support during the pilot. Your team decides which changes flow into the next training run.
How fast can we see results?
A pilot team receives an initial failure-cluster review once enough representative telemetry has been ingested. Timing depends on task volume, event quality, integration readiness, and failure frequency. During onboarding we define the minimum evidence needed for the first analysis.
Do we need new hardware?
No. Sim2Real runs in the cloud and ingests telemetry from the sensors and controllers you already operate. There is no on-premise requirement for the Pilot Optimizer plan.
Can telemetry be processed live or offline?
The current API can accept events as they are produced or from a controlled batch process. The pilot defines the cadence that matches the team’s validation workflow.
Do we need perfectly labeled failures?
No. Operator labels and incident notes can help, but a pilot can begin from task outcomes and available context. Engineers still review whether a cluster is meaningful.
What does a successful first 30 days look like?
A useful first month produces an agreed telemetry boundary, at least one reviewable failure cluster, and a documented validation loop for the next change.
Can we send images or video?
Media handling is scoped explicitly before transfer. Start with references or metadata unless the pilot agreement defines the media boundary, retention, and authorized reviewers.
How are security and retention requirements handled?
Authentication and route protections exist in the current application. Data fields, retention, deletion, hosting, and access requirements are documented during pilot qualification.
Next Step
See how it fits your deployment workflow.
Book a demo to walk through telemetry ingestion, failure classification, retraining workflow design, and deployment reporting.