What Sim2Real actually returns after processing deployment telemetry.

1. Robot and task context

2. Observed symptoms

• Real-world grasp success: 64.3% (vs. 91.1% in the most recent simulation regression run)
• Failure mode: Gripper closes on specular vacuum-sealed packaging; item slips during transfer 81% of the time
• Time-of-day pattern: Failure rate climbs to 78% between 16:00 and 18:30 when the skylight raises scene luminance and the gripper shadow self-occludes the package face
• Operator overrides: Spiked 6.2× in the same window (mostly manual repositioning)
• Telemetry signal: Wrist-camera histograms show a bimodal luminance distribution with a clipped specular highlight that the simulation never produced

3. Clustered physical cause

Sim2Real groups this set of failures under perception mismatch — lighting variance and reflective material handling. The cluster is distinct from the physics-mismatch and task-mismatch clusters the same robot is producing, which is why the recommendation can target the right layer of the stack.

Within the perception-mismatch family, the cluster is further tagged specular_highlight_self_occlusion and low_angle_skylight. Both tags co-occur in 71% of the cluster’s failure episodes, which is why we treat them as a single recommended action rather than two independent ones.

4. Simulator assumptions that turned out to be wrong

• Lighting randomization range was set to a uniform distribution matching the simulation regression fixture, not the production scene. The skylight angle at end-of-aisle never appeared in any training episode.
• Material randomization included diffuse plastic and matte cardboard but excluded the vacuum-sealed reflective SKU that the pilot now ships 3× per day.
• Pose randomization kept the package face within ±15° of the gripper camera. Production logging showed the actual distribution spans ±42° because totes are tipped at varied fill levels.
• Sensor noise model assumed Gaussian noise on a calibrated sensor. Production cameras in this aisle show a clipped-highlight region that the sensor model does not produce.

5. Parameter patch (what Sim2Real ships back)

The cluster returns a JSON-mergeable update your training pipeline can apply directly. Below is the rendered shape — the same content is available as a .json download via the /api/export/isaacsim and /api/export/mujoco endpoints in the live product.

{
  "cluster_id": "C-0042",
  "site": "Pilot Site B / Aisle 7B",
  "robot": "arm-6dof-rgbd-gripper-v3",
  "tags": ["perception_mismatch", "specular_highlight", "low_angle_skylight"],
  "patch": {
    "domain_randomization": {
      "lighting": {
        "skylight_intensity_range": [0.6, 1.4],
        "key_light_angle_range_deg": [10, 75],
        "add_secondary_skylight": true
      },
      "materials": {
        "add_specular_variants": ["vacuum_seal_clear", "vacuum_seal_metalized"],
        "specular_roughness_range": [0.02, 0.35]
      },
      "pose": {
        "package_face_yaw_range_deg": [-42, 42],
        "package_face_pitch_range_deg": [-25, 25]
      }
    },
    "training": {
      "increase_specular_weight_x": 2.5,
      "new_failure_mix": "specular_seal:0.35,matte:0.45,transparent:0.20",
      "recommended_min_episodes": 1200
    },
    "validation": {
      "holdout_set": "aisle_7b_last_48h",
      "regression_set": "isaac_sim_v2024.1_pilot_b"
    }
  },
  "evidence_refs": ["telem:7b-1742", "telem:7b-1881", "telem:7b-2104"]
}

6. Before / after on the held-out validation set

Numbers are illustrative. The live product reports the actual measured deltas against your own held-out set; the metrics above are illustrative of the report shape your team will see.

7. Next retraining action

1. Apply the patch above to your Isaac Sim domain-randomization config.
2. Route the new specular_seal scenario batch into your next training run alongside the existing scenario set.
3. Validate against the held-out aisle_7b_last_48h set before promoting the policy to production.
4. Sim2Real automatically opens a new cluster review once production telemetry from the updated policy lands, so the loop stays closed.