What Sim2Real actually returns after processing deployment telemetry.

1. Robot and task context

2. Observed symptoms

• Real-world grasp success: 64.3% (vs. 91.1% in the most recent simulation regression run)
• Failure mode: Gripper closes on specular vacuum-sealed packaging; item slips during transfer 81% of the time
• Time-of-day pattern: Failure rate climbs to 78% between 16:00 and 18:30 when the skylight raises scene luminance and the gripper shadow self-occludes the package face
• Operator overrides: Spiked 6.2× in the same window (mostly manual repositioning)
• Telemetry signal: Wrist-camera histograms show a bimodal luminance distribution with a clipped specular highlight that the simulation never produced

3. Clustered failure pattern

In this illustration the events are tagged lighting_variance. Sim2Real stores that tag as sent. The other recognized tags are object_tilt, clutter_occlusion, physics_mismatch, sensor_mismatch, and unknown. Any other value is stored as unknown. There is no separate task-mismatch tag.

The specular highlight and the low-angle skylight are observations in this story, not stored tags. The live product does not create extra tag names from those details.

4. Simulator assumptions that turned out to be wrong

• Lighting randomization range was set to a uniform distribution matching the simulation regression fixture, not the production scene. The skylight angle at end-of-aisle never appeared in any training episode.
• Material randomization included diffuse plastic and matte cardboard but excluded the vacuum-sealed reflective SKU that the pilot now ships 3× per day.
• Pose randomization kept the package face within ±15° of the gripper camera. Production logging showed the actual distribution spans ±42° because totes are tipped at varied fill levels.
• Sensor noise model assumed Gaussian noise on a calibrated sensor. Production cameras in this aisle show a clipped-highlight region that the sensor model does not produce. The starter illustration does not change that sensor noise model, and neither does the live export.

5. Starter patch illustration Synthetic demo

The live product exposes .json patch exports via the /api/export/isaacsim and /api/export/mujoco endpoints. For a lighting_variance group of three or more events, that file can include a starter lighting override. It does not rewrite pose ranges or a sensor noise model. The JSON below is a fictional illustration, and its schema is not the live export. Your engineering contact tunes any values against your setup during the pilot.

{
  "cluster_id": "C-0042",
  "site": "Pilot Site B / Aisle 7B",
  "robot": "arm-6dof-rgbd-gripper-v3",
  "tags": ["lighting_variance"],
  "patch": {
    "domain_randomization": {
      "lighting": {
        "skylight_intensity_range": [0.6, 1.4],
        "key_light_angle_range_deg": [10, 75],
        "add_secondary_skylight": true
      },
      "materials": {
        "add_specular_variants": ["vacuum_seal_clear", "vacuum_seal_metalized"],
        "specular_roughness_range": [0.02, 0.35]
      }
    },
    "note": "Illustration only. The live export does not include pose ranges, a sensor noise model, a training-mix weight, or a held-out validation set."
  },
  "evidence_refs": ["telem:7b-1742", "telem:7b-1881", "telem:7b-2104"]
}

6. Illustrative field metrics from a later window Synthetic demo

Synthetic demo. These numbers are fabricated. They are not a customer result, and they are not a held-out validation. The live product shows the success rate and drift score calculated from the events stored for the workspace. It does not show a before-and-after table.

7. Next retraining action

1. Review the starter lighting override with your engineering contact and decide which ranges, if any, belong in your Isaac Sim domain-randomization config.
2. Your team authors any new training scenario. Sim2Real does not generate that scenario.
3. Your team decides how to check the next run before promoting a policy. The product does not run a held-out validation set.
4. Later events can be posted for the next review. A change in the reported success rate is not by itself proof of cause.