Point Clouds as Data — Quality & Training

00 / WHY THIS MATTERS

Garbage geometry in, confident errors out

With images, bad data usually looks bad. With point clouds, a corrupted cloud can look perfectly plausible to a human yet teach a model exactly the wrong lesson — because the defects are geometric and statistical, not visual.

Training data defines the model's world. If distant cars always arrive as 10 points in your data, the model learns "10 points = car is far" — and inherits that blind spot forever.

Evaluation data defines what "good" means. If your test set under-represents rain or night, a model can score 95% and still be unsafe where it matters. The metric lies by omission.

Labels are data too. A 3D box drawn 20 cm off, or a missed pedestrian in an occluded frame, is a wrong answer the model is rewarded for matching.

01 / THE CORRUPTION LAB

Break a clean cloud, watch the lesson rot

Left = an ideal scan of a street (two cars, a pedestrian, a wall). Right = the same scene after a defect is applied. Pick a defect, push the slider, and read what it teaches a model.

CLEAN · 真值

CORRUPTED

Severity

Drag either view to rotate both together. The clean side never changes — it's the ground truth you wish you had. The gap between the two panels is the data-quality problem.

02 / THE FULL CATALOG

Eight ways a cloud betrays a model

Each defect, its root cause, what it does to training vs. evaluation, and the common mitigation. Tap to expand.

▚

Sparsity & non-uniform density

稀疏 / 密度不均

▾

Root cause

Beams spread with distance; a 32-line LiDAR may put thousands of points on a near car and a dozen on a far one.

Effect on training

Model overfits to dense near-field objects; learns features that don't exist for far ones. Far-object recall collapses.

Effect on eval

A test set heavy on near objects inflates your score — the metric hides the long-range failure that actually causes crashes.

Mitigation

Multi-frame aggregation, density-aware heads (DARP-style), range-stratified metrics so you score near/mid/far separately.

◑

Occlusion & incomplete objects

遮挡 / 不完整观测

▾

Root cause

A truck blocks the car behind it; LiDAR only ever sees surfaces facing it. Every object is a partial shell, and some are mostly hidden.

Effect on training

Labelers must guess hidden extents — so the model learns from inconsistent "imagined" box sizes. Partial objects may go unlabeled entirely.

Effect on eval

Two valid annotators disagree on an occluded box, so ground truth itself is noisy — your mAP penalizes correct predictions.

Mitigation

Track objects across frames to fill occluded moments; flag visibility level per box and report metrics by visibility bucket.

∴

Sensor noise & outliers

传感器噪声 / 离群点

▾

Root cause

Range jitter, multi-path reflections, dust, exhaust, retro-reflective signs that "bloom." Some points are simply wrong.

Effect on training

Phantom points teach phantom objects; the model learns to fire on noise, raising false positives — dangerous when it phantom-brakes.

Effect on eval

If noise is only in real data and your synthetic eval is clean, you never measure this failure until it's on the road.

Mitigation

Statistical outlier removal, intensity filtering, and deliberately injecting realistic noise so the model learns to ignore it.

⤳

Motion distortion / skew

运动畸变

▾

Root cause

A spinning LiDAR takes ~100ms per sweep. The car (and others) move during the sweep, so a single "frame" is smeared in time.

Effect on training

Object shapes warp at speed; without correction the model learns distorted geometry and mis-sizes fast objects.

Effect on eval

Box centers drift, so even a perfect detector misses the warped ground-truth box — error you can't fix by training harder.

Mitigation

Motion compensation / de-skew using ego-velocity and per-point timestamps before anything else touches the cloud.

⇄

Domain gap (sim & sensor)

域差异 / 仿真-真实差距

▾

Root cause

Train on a 64-line sensor, deploy on 128-line. Or train on synthetic clouds that lack real dropout noise. The point statistics differ.

Effect on training

Model latches onto sensor-specific patterns (beam spacing, density signature) instead of true shape — and breaks on the new sensor.

Effect on eval

In-domain test scores look great; cross-domain performance silently craters. Your eval validated the wrong thing.

Mitigation

Domain adaptation (ePointDA-style dropout rendering), sensor-agnostic representations, and always testing cross-sensor / sim-to-real.

▤

Class & scenario imbalance

类别 / 场景不均衡

▾

Root cause

Roads are mostly empty asphalt and cars. Cyclists, debris, strollers, overturned trucks — the things that kill — are rare in raw logs.

Effect on training

Model optimizes for the common case and barely learns rare classes — exactly the long-tail safety cases that matter most.

Effect on eval

Overall accuracy is dominated by easy majority cases; a model can ace the benchmark and still miss every cyclist.

Mitigation

Object-paste augmentation, targeted mining of rare events, synthetic generation of the long tail, and per-class (not just overall) metrics.

✗

Annotation error

标注错误

▾

Root cause

Humans draw 3D boxes in a sparse, occluded cloud — loose fit, wrong heading, missed object, inconsistent class definitions between labelers.

Effect on training

The model faithfully learns the labelers' mistakes. Garbage labels cap the achievable accuracy no matter how good the architecture.

Effect on eval

Noisy ground truth means your metric has a built-in error floor — and can rank a worse model above a better one.

Mitigation

Consensus labeling, auto-label + human review (VESPA-style), label-quality audits, and inter-annotator agreement tracking.

⊕

Calibration & sync error

标定 / 时间同步误差

▾

Root cause

Fusion needs LiDAR, cameras and radar aligned in space (extrinsics) and time. A few cm or a few ms of misalignment and the modalities disagree.

Effect on training

Camera color gets painted onto the wrong points; the fused model learns from contradictory cross-sensor signals.

Effect on eval

Ground-truth boxes from one sensor's frame don't line up with another's — phantom errors that aren't the model's fault.

Mitigation

Rigorous extrinsic/intrinsic calibration, hardware timestamp sync, and ego-motion compensation to a unified coordinate frame.

03 / THE EVALUATION LENS

When the metric itself is the problem

Training defects make a weaker model. Eval defects are scarier: they make a weak model look strong, so you ship it. Same clouds, different stakes.

Data property	If wrong in TRAINING	If wrong in EVALUATION
Sparsity / range 稀疏	Blind to far objects	Hides the blind spot behind a high score
Scenario mix 场景分布	Underfits rare cases	Benchmark ≠ real road; false confidence
Label quality 标注质量	Learns the errors	Noisy ground truth, unstable ranking
Domain match 域匹配	Sensor-specific shortcuts	In-domain only; cross-sensor unmeasured
Weather / night 天气	Fragile in adversity	Sunny-day metric, rainy-day reality

← swipe · training pain vs eval danger →

The distribution-match principle

One rule underneath all of this: your training and eval distributions must match the deployment distribution （训练、评估、部署三者分布必须一致） — in scenes, sensors, weather, geography, and object mix. Most real-world model failures are not architecture failures; they're distribution-mismatch failures hiding in the data.

This is also why a held-out test set isn't enough. You need stratified, scenario-aware evaluation: score by range bucket, by class, by weather, by sensor — so a single happy number can't hide a fatal gap.

04 / THE WAY OUT — AND YOUR ANGLE

Synthetic data & world models

If real clouds are sparse, imbalanced, and brutally expensive to label, the obvious move is to generate them. This is where data quality meets your simulation north star.

Why synthetic 为什么用合成数据

Simulated LiDAR comes with perfect, free labels — the simulator knows every object's exact box because it placed it. You can generate unlimited rare events: the overturned truck, the child chasing a ball, the sensor failing in fog. It directly attacks sparsity, imbalance, and label cost in one shot.

The catch: the domain gap 难点：域差异

A synthetic cloud that's too clean teaches a model that real noise is anomalous. Closing the sim-to-real gap — matching real LiDAR's dropout, intensity, and density signatures — is the central research problem, and it's exactly the value a strong simulation team provides.

World models as data engines 🌐

This is the throughline from Part 2. A LiDAR world model （点云世界模型） that can forecast realistic future point clouds is, in effect, a learned data generator — it can synthesize sensor-accurate scenes and rare scenarios on demand, with labels attached. The frontier isn't just perceiving point clouds; it's generating high-fidelity ones to train and stress-test the next model.

So the data-quality problem and the world-models opportunity are the same coin: whoever can produce point clouds whose statistics match reality controls the quality ceiling of every model trained on them. That's the leverage point.

05 / PRACTITIONER'S CHECKLIST

Auditing a point-cloud dataset

Questions worth asking of any 3D dataset before you trust a number it produces.

▸ What's the point density vs. range curve, and is the test set stratified by it?
▸ What sensor(s) generated this, and does it match deployment?
▸ How is occlusion / visibility recorded per object?
▸ Is the cloud motion-compensated? With what ego-source?
▸ What's the class distribution, and are rare classes scored separately?
▸ What's the inter-annotator agreement, and how were labels audited?
▸ Are weather / night / geography represented in proportion to deployment?
▸ Is any of it synthetic, and was the domain gap measured, not assumed?

If a benchmark can't answer these, its headline accuracy is a number without a denominator. For your simulation/world-models direction, owning the answers to these is the moat.