1
Lighting variance
Glare or low light reported on the event. Three or more can produce a starter lighting override.
Product Architecture
See how Sim2Real turns deployment evidence into better training data.
Sim2Real captures what happened in the field, identifies where simulator assumptions broke, and creates updated synthetic conditions for the next training pass.
TL;DR: Sim2Real is a sim-to-real platform that ingests deployment telemetry from real robot runs, groups failures by the pattern tags your telemetry reports, and returns simulation parameter updates so your next training pass targets the right assumptions. It exports starter patches for Isaac Sim and MuJoCo (pilot-supported); ROS/ROS2 and Omniverse are scoped per engagement.
Data Capture
How deployment telemetry is captured
Sim2Real ingests structured JSON events describing real robot execution: task outcomes, failure tags, sim-vs-real metrics, and compact sensor metadata. Media such as camera footage stays in your systems and is referenced, not uploaded.
Telemetry inputs
- Task outcomes, failure flags, and failure-pattern tags
- Sim-vs-real confidence, success-rate, and drift metrics
- Compact sensor metadata and references (JSON, not media uploads)
- Robot, site, and task identifiers (deployment or build IDs go inside
rawSensor)
Why it matters
Robotics teams often know that a deployment failed, but not which part of the simulation model stopped matching reality. Sim2Real makes those mismatches legible enough to act on.
What you send
Enough context to explain one deployment task.
- Structured task outcomes and timestamps
- References or metadata for camera, force-torque, IMU, or controller context
- Robot, site, and task identifiers (build, scene, or deployment IDs go inside
rawSensor) - Operator labels or incident notes when available
Setup: JSON API mapping scoped during pilot intake. Owner: a robotics or platform engineer who understands the event boundary.
What you get back
A prioritized path to the next simulation test.
- Grouped failure patterns with supporting evidence
- Affected deployments, robots, and task context
- Reviewable simulation or training recommendations
- A validation record for the next field run
Cadence: aligned to the team’s review loop. Owner: the engineer responsible for approving and testing the change.
Robot deploymentTelemetry APIFailure clustersSimulator correctionValidation run
Failure Analysis
How failures are grouped
Each failure is stored under the pattern tag on the event. Recognized tags are lighting variance, object tilt, clutter occlusion, physics mismatch, sensor mismatch, and unknown. Any other value is stored as unknown. The product does not add a task-mismatch or deployment-noise category.
2
Object tilt
Tray or object angle reported as affecting placement. Three or more can produce a starter contact and friction override.
3
Clutter occlusion
Partially hidden grasp targets. Three or more can produce a starter clutter-density override.
4
Physics mismatch
Friction or contact the event tags as diverging from simulation. The workspace lists the group. The starter file does not add a separate patch for this tag.
5
Sensor mismatch
Sensor readings the event tags as diverging from simulation. The starter export does not rewrite a sensor noise model.
6
Unknown
No recognized tag, or a value outside the list. Those events stay here for review.
Simulator Updates
How simulation parameters are updated
Once discrepancies are identified, Sim2Real produces starter parameter override patches for MuJoCo and Isaac Sim, refined with engineering support during the pilot, for the next iteration.
1
Map observed discrepancies
For lighting variance, object tilt, and clutter occlusion, the export can include a starter lighting, contact, or clutter-density override. It does not rewrite pose ranges or a sensor noise model.
2
Export starter override patches
Export starter parameter override patches for MuJoCo or Isaac Sim that target the failure patterns reported in deployment.
3
Design retraining scenarios with engineering support
Scenario combinations are designed with engineering support during the pilot and routed into your existing training workflow.
4
Compare the next run
Post the next events and compare the reported outcome with the earlier run. The stored record does not by itself prove the override caused the difference.
Integrations
Supported robotics stack integrations
Sim2Real is positioned to work alongside the tooling robotics teams already depend on for simulation, orchestration, and policy evaluation.
Scoped per engagement
ROS / ROS2
ROS / ROS2
Pilot-supported
Isaac Sim
Isaac Sim
Scoped per engagement
Omniverse
Omniverse
Pilot-supported
MuJoCo
MuJoCo
Scoped per engagement
Custom APIs
Custom APIs
Scoped per engagement
Telemetry pipelines
Telemetry pipelines
Pilot-supported: works today with hands-on engineering support during a pilot. Scoped per engagement: not a standard offering; feasibility is assessed during scoping, and evaluation is not a commitment.
Operational Outcome
What the workspace organizes during a pilot
Teams often collect more data than they review and spend onsite time figuring out why a robot did not behave as expected. Sim2Real stores the events you post, groups them by the tag you sent, and can export a starter patch. It does not measure a reduction in failure rate, and the examples on this site are synthetic.
Grouped failure tags
The workspace shows one group per recognized tag, with the count of events you posted.
A calibration backlog
Reported patterns become a list of starter overrides to review, instead of an unstructured log search.
Use the events you already post
The same JSON events can feed a starter MuJoCo or Isaac Sim patch. Your team decides what to apply.
The same figures for the review
The dashboard shows the success rate and drift score calculated from the events stored for the workspace.
Why not manual?
Keep engineering judgment. Remove the evidence scavenger hunt.
| Decision step | Manual workflow | Sim2Real workflow |
|---|---|---|
| Collect evidence | Search logs, videos, tickets, and notes separately | Use one scoped event boundary and linked deployment context |
| Find repeatable patterns | Rely on ad hoc queries and team memory | Review grouped failure clusters and supporting events |
| Choose a simulator change | Translate symptoms into scene or training changes by hand | Review a recommendation tied to the observed pattern |
| Validate the result | Compare runs with inconsistent documentation | Record the targeted failure, change, and next deployment outcome |
Use cases
Start with one expensive, repeatable failure pattern.
Failure patternMissed grasps under pose, clutter, or lighting variance
WorkflowCluster task failures, review evidence, vary the matching simulation conditions
KPI to monitorTask success within the same pick boundary
Failure patternFalse rejects or misses under surface and illumination changes
WorkflowGroup visual conditions, review examples, extend scene or data variation
KPI to monitorError mix on the same inspection class
Failure patternLocalization or planning degradation in changed environments
WorkflowCluster context shifts, identify common conditions, replay targeted scenarios
KPI to monitorMission completion within the same route class
Security and procurement
Scope the data boundary before telemetry moves.
Sim2Real uses authenticated per-user workspace access, API-key telemetry endpoints, server-side route guards, and browser security headers in the current application.
Review the privacy policyQualified during pilot intake
- Fields, files, and sensor references included in the telemetry schema
- Retention, deletion, and operational review requirements
- Deployment environment and integration responsibilities
- People authorized to review workspace outputs
Operator
Developer312, a brand of Night Lite USA LLC
Qualified pilots are handled as direct engineering engagements: define the event boundary, inspect the artifact, and agree on a validation plan before broader rollout claims are made.
Certifications, team-level isolation, and custom retention are not implied by this page; requirements are documented during commercial scoping. Sim2Real is currently cloud-hosted: VPC, on-premise, regional, and air-gapped deployments are not standard offerings and are evaluated individually for qualified enterprise engagements. Evaluation does not guarantee availability.
FAQ
Product questions engineering teams ask
Does Sim2Real replace our existing simulator?
No. Sim2Real is a failure-intelligence layer beside the simulator your team already runs; it does not replace it or retrain the policy. MuJoCo and Isaac Sim exports are pilot-supported with hands-on engineering support during a pilot; ROS/ROS 2, Omniverse, and custom stacks are scoped per engagement.
What kinds of telemetry does Sim2Real ingest?
Structured JSON events: task outcomes, failure flags and pattern tags, sim-vs-real confidence, success-rate, and drift metrics, plus a compact raw-sensor metadata field. Media such as camera streams is referenced, not uploaded. The schema is JSON-first and append-only.
How are failures grouped?
Sim2Real groups each failure by the pattern tag on the event. Recognized tags are lighting variance, object tilt, clutter occlusion, physics mismatch, sensor mismatch, and unknown. Any other value is stored as unknown.
What does the retraining output look like?
Starter parameter override patches for MuJoCo and Isaac Sim (for example lighting, contact friction, and clutter density overrides), refined with engineering support during the pilot. Your team decides which changes flow into the next training run.
How fast can we see results?
A pilot team receives an initial failure-cluster review once enough representative telemetry has been ingested. Timing depends on task volume, event quality, integration readiness, and failure frequency. During onboarding we define the minimum evidence needed for the first analysis.
Do we need new hardware?
No. Sim2Real runs in the cloud and ingests telemetry from the sensors and controllers you already operate. There is no on-premise requirement for the Pilot Optimizer plan.
Can telemetry be processed live or offline?
The current API can accept events as they are produced or from a controlled batch process. The pilot defines the cadence that matches the team’s validation workflow.
Do we need perfectly labeled failures?
No. Operator labels and incident notes can help, but a pilot can begin from task outcomes and available context. Engineers still review whether a cluster is meaningful.
What does a successful first 30 days look like?
A useful first month produces an agreed telemetry boundary, at least one reviewable failure cluster, and a documented validation loop for the next change.
Can we send images or video?
Media handling is scoped explicitly before transfer. Start with references or metadata unless the pilot agreement defines the media boundary, retention, and authorized reviewers.
How are security and retention requirements handled?
Authentication and route protections exist in the current application. Data fields, retention, deletion, hosting, and access requirements are documented during pilot qualification.
Next Step
See how it fits your deployment workflow.
Book a demo to walk through telemetry ingestion, failure classification, retraining workflow design, and deployment reporting.