Everything so far needed one thing: a simulator that produces point clouds whose statistics match reality — to generate the rare, broken, perfectly-labeled data robustness depends on. Here's how you actually build it, and how you prove it works. （如何构建并验证激光雷达仿真器）
A camera simulator renders pixels. A LiDAR simulator must reproduce three physical channels per point — and getting all three right is the whole game.
Where each return lands in 3D. The easy part — pure ray–geometry intersection. If this were all that mattered, a game engine would suffice.
How strong each return is — depends on material, incidence angle, range. Lane paint and signs blaze; dark asphalt and wet road barely return. Models that skip this leave a signal detectors actually use on the table.
Which rays come back as nothing. When a return is too weak — grazing angle, black/wet/specular surface, far range — the ray simply drops. The single hardest channel to fake, and the one that most betrays a synthetic cloud as fake.
Left = a real reference scan. Right = your simulated version. Switch on each modeling stage and watch the synthetic cloud — and the domain-gap meter — converge toward reality. （逐层逼近真实）
Drag either panel to rotate both. Notice the order of impact: raw geometry looks too clean and too perfect; raydrop is what makes it look real. In LiDARsim, adding a learned raydrop network lifted downstream detection AP ~2% over no raydrop — and landed within ~0.7% of using oracle ground-truth raydrop.
Pure physics says "ray hits surface → return." Reality says "...maybe." That gap between deterministic geometry and probabilistic return is where sim-to-real lives or dies.
A raycaster gives a clean, complete point on every surface. But real LiDAR drops rays for reasons too complex to hand-model: a grazing incidence angle, a black car, a wet road, a retroreflector blooming, the beam clipping an edge. The pattern of missing points is a fingerprint of the real sensor.
The trick that defined modern LiDAR sim: treat raydrop as a binary classification per ray, and train a small neural network on real scans to predict drop probability from geometry, range, incidence angle and intensity. The physics proposes; the network disposes. This single move is what separates "looks like a game" from "fools a detector."
You can't pure-physics your way to realism, and you can't pure-learn your way to control. The winning recipe is physics for structure, learning for the residual — raycast the geometry you can compute, then let a data-driven model add the sensor quirks you can't. That hybrid is the throughline of every method below.
From hand-built worlds to neural reconstruction. Each generation traded a weakness of the last. Tap to expand.
A sim cloud can look great and still teach the wrong thing. Realism is judged by downstream agreement, not by eye. （用下游任务验证，而非肉眼）
Take a detector trained on real data. Run it on your simulated clouds. If it produces the same detections — same boxes, same confidences, same false positives — your sim is real enough. The metric isn't "does it look like LiDAR," it's "does a model behave the same on it."
| Test | Question it answers | Good result |
|---|---|---|
| Train-sim → test-real | Does sim data transfer to reality? | small accuracy drop |
| Detection agreement | Same boxes on sim vs real? | high overlap |
| Statistical match | Same point density / intensity / drop dist? | distributions align |
| Augment real + sim | Does adding sim help real performance? | accuracy goes up |
← swipe · these are the real acceptance tests →
A subtle finding: modeling even rare phenomena matters. Adding LiDAR secondary echoes — only ~5% of points — measurably shrinks the domain gap, because it changes false positives in a way that ripples all the way to planning.
Pulling all four parts together — this is the engine the whole series was describing.
Drive, record real LiDAR. The source of truth and the calibration target.
Reconstruct scenes (Gaussian splats), learn the sensor's raydrop & intensity from real scans. Physics for structure, networks for the residual.
Edit scenes, drop sensors, add fog, place the once-a-year edge case — unlimited, perfectly-labeled, sensor-accurate clouds.
Mix real + synthetic; apply sensor dropout & corruption augmentation (Part 3.5). Clean labels, corrupted inputs.
Measure downstream agreement & sim-to-real transfer. Where it still fails tells you what to capture or model next.
Part 1 was the sensor; Part 2 the stack that consumes its points; Part 3 the data quality that governs every model; Part 3.5 robustness to failure. Part 4 is the engine underneath all of it: whoever can simulate point clouds whose statistics match reality controls the quality ceiling of every model trained on them.
That's not adjacent to world models — it is the world-model thesis, applied to a sensor. A simulator that predicts realistic future point clouds is a learned model of physical reality. This is the lever, and it's exactly where the simulation / world-models work you're aiming at sits.