Senior Perception Engineer, Obstacle Foundation Models - Autonomous Vehicles
$184k - $357k • Remote • US, CA, Santa Clara
Posted 17d ago
Job Location
US, CA, Santa Clara
Tech Stack
Remote Work Policy
Fully remote
Categories
Machine Learning Engineer
About the job
NVIDIA is seeking a Senior Perception Engineer to design and productize the next generation of autonomous driving perception systems. This role focuses on the core 3D obstacle perception pipeline, involving architecture and algorithm design, with a hands-on approach to implementation using modern transformer-based, multi-modal, and vision-language techniques. You will contribute to the development of state-of-the-art deep learning models for obstacle detection and tracking, leveraging multi-camera and multi-sensor fusion. The position requires defining objectives, prototyping architectures, running experiments, and adhering to best practices for training and evaluation, including techniques like large-scale pretraining and parameter-efficient fine-tuning. You will also help define and maintain performance metrics, analyze large datasets to identify failure modes, and contribute to the data strategy for perception, including model-assisted workflows and model-in-the-loop tooling. Collaboration with safety, systems, and software teams is essential to ensure solutions meet product requirements for safety, latency, resource usage, and robustness for large-scale deployment.
Responsibilities
- Develop and improve the technical design, architecture, and roadmap for 3D obstacle perception.
- Design and implement advanced 3D perception models using multi-camera inputs and/or multi-sensor fusion for obstacle detection and tracking.
- Build efficient, production-grade deep learning models, including prototyping architectures and running experiments.
- Define and maintain KPI frameworks to quantify perception performance and analyze datasets to improve accuracy and robustness.
- Contribute to the data strategy for perception, specifying data requirements and collaborating with data teams.
- Collaborate with safety, systems, and software teams to ensure perception solutions meet product requirements.
Requirements
- PhD with 4+ years, MS with 6+ years, or BS with 8+ years of relevant experience in Computer Science, Computer Engineering, or a related technical field.
- Hands-on experience developing deep learning-based perception systems for complex real-world problems.
- Strong proficiency in frameworks such as PyTorch.
- Proven experience in data-driven development and collaboration with data/labeling teams.
- Strong programming skills in Python and/or C++.
- Experience building reliable, high-performance, production-quality software.
Benefits
- Equity
- Benefits