Projects
Eighteen hands-on mini-projects, organized into eight phases that mirror the four-loop data flywheel: COLLECT → CURATE → LABEL & TRAIN → EVAL. Filter by phase, loop, or hardware tier.
20 of 20 projects
Phase AData fluency
Phase BLabeling fundamentals
- 03SAM 2 + Grounding DINO auto-labelingLABEL
The "traditional" auto-labeling pipeline; understand cost-per-label economics
Hardware: Laptop GPU
- 04BEVFormer 3D detectionLABEL
Real AV is 3D multi-sensor; introduces sensor calibration (intrinsics, extrinsics, frames)
Hardware: Workstation 24GB+
- 05VLM zero-shot labelingLABELCURATE
The frontier — Gemini/GPT/Claude as labelers; produces a 2x2 decision matrix for traditional vs VLM
Hardware: Laptop
Phase CProduction hygiene
Phase DSimulation and world models
- 07CARLA OpenSCENARIO scenariosCOLLECTEVAL
Scenario authoring as a discipline — the Simian core
Hardware: Workstation Linux
- 083D Gaussian Splatting real-to-simCOLLECTEVAL
The reconstruct half of sim — Wayve PRISM-1 / Waymo EmerNeRF / AI Neural Sim
Hardware: Workstation 24GB+
- 09CARLA + Cosmos Transfer hybrid simCOLLECTLABEL
Classical sim ground truth + WM photoreal — the pattern that's actually winning
Hardware: H100 / A100 80GB
- 10Cosmos Predict driving rolloutCOLLECTEVAL
Measure WM action-conditioning drift with numbers, not marketing
Hardware: H100 / A100 80GB
Phase ERobotics adjacency
Phase FBehavior, sim agents, closed-loop
- 13Argoverse 2 motion forecastingLABELEVAL
Motion prediction is half of AV planning; behavior labels come "from the future"
Hardware: Workstation GPU
- 14Waymax sim-agentsEVAL
Without realistic NPCs, closed-loop sim is unfalsifiable — the gating problem
Hardware: Workstation GPU
- 15Bench2Drive closed-loop evalEVAL
A learned planner under closed-loop scoring; reuses CARLA from project 07
Hardware: Workstation 16GB+
Phase GActive learning + capstone
Phase HStrategy
Phase IFrontier extensions (Round 3)
- 19World-model-as-evaluatorEVALCURATE
WMs-as-evaluators went from thesis to subfield in 2026; measure judge-vs-sim agreement with numbers
Hardware: Laptop
- 20Driving preference + reward model (RLHF for E2E)LABELEVAL
Preference data is the fastest-growing label category; converts LLM post-training skills into physical-AI credibility
Hardware: Laptop