Machine Learning Engineer Intern- Scenario Simulation
Santa ClaraOn-siteInternship
AI Summary
Internship focusing on bridging the sim-to-real gap for BEV embeddings and enabling closed-loop training for planning models in autonomous driving.
About this role
PlusAI is a Physical AI company pioneering AI-based virtual driver software for factory-built autonomous trucks. Headquartered in Silicon Valley with operations in the United States and Europe, Plus was named by Fast Company as one of the World’s Most Innovative Companies. Partners including TRATON GROUP’s Scania, MAN, and International brands, Hyundai Motor Company, Iveco Group, Bosch, and DSV are working with Plus to accelerate the deployment of next-generation autonomous trucks. If you’re ready to make a huge impact and drive the future of autonomy, Plus is looking for talented individuals to join its fast-growing teams.
This project addresses the sim-to-real gap as in our current simulator pipeline. Closed-loop simulation systems are highly effective at generating massive amounts of diverse, high-fidelity ground truth data. However, they assume perfect perception. However, this ideal GT data does not perfectly align with the BEV feature representations extracted from real-world, noisy sensor data.
The project provides two outcomes:
Bridge Sim-to-Real Gap in Data Augmentation with Closed-Loop Simulation: By translating simulation GTs into a realistic BEV embedding space, the synthetic data becomes directly usable for training and fine-tuning BEV-based planning models.
Allow BEV Space Planning Model to be directly fine-tunable via Self-play RL: Once the sim-to-real gap is bridged in the BEV feature space, the planning models can thus be directly finetuned / post-trained in a self-play RL setting.
The project provides two outcomes:
Bridge Sim-to-Real Gap in Data Augmentation with Closed-Loop Simulation: By translating simulation GTs into a realistic BEV embedding space, the synthetic data becomes directly usable for training and fine-tuning BEV-based planning models.
Allow BEV Space Planning Model to be directly fine-tunable via Self-play RL: Once the sim-to-real gap is bridged in the BEV feature space, the planning models can thus be directly finetuned / post-trained in a self-play RL setting.
Responsibilities:
Bridge the Sim-to-Real Gap: Develop and implement a translation layer that converts idealized simulator Ground Truth (GT) into realistic, "noisy" Bird’s-Eye View (BEV) embeddings.
Optimize Simulation Throughput: Research and implement a "shortcut" pipeline that bypasses slow image/LiDAR rendering to generate BEV features directly from state data.
Enable Closed-Loop Training: Integrate the translated BEV embeddings into a training pipeline to make synthetic data directly usable for planning models.
Support Reinforcement Learning: Create the infrastructure necessary for planning models to undergo self-play RL fine-tuning within the bridged BEV feature space.
Optimize Simulation Throughput: Research and implement a "shortcut" pipeline that bypasses slow image/LiDAR rendering to generate BEV features directly from state data.
Enable Closed-Loop Training: Integrate the translated BEV embeddings into a training pipeline to make synthetic data directly usable for planning models.
Support Reinforcement Learning: Create the infrastructure necessary for planning models to undergo self-play RL fine-tuning within the bridged BEV feature space.
Required Skills:
Preferred Skills:
Nice-to-have:
Skills
BEV GenerationCARLAComputer VisionData GenerationDeep LearningGazeboIsaacSimKubeflowMLflowML InfrastructurePythonPyTorchSensor FusionSimulation PipelinesSpatial TransformationWeights & Biases
Explore related jobs
More jobs at PlusAI
Similar BEV Generation jobs
Jobs in Santa Clara
Leasing Consultant (Hearth)Prometheus Real Estate Group · Santa Clara, California- Technician, ATE MaintenanceEurofins · Santa Clara, CA
General Manager - Santa ClaraJETSET Pilates · Santa Clara, Canada- Senior Director of Integrated Marketing Program - ITServicenow · Santa Clara, CALIFORNIA
- Senior Staff Technical Program ManagerServicenow · Santa Clara, CALIFORNIA
Mobile Dog BatherBarkbus · Santa Clara, Canada