Senior Research Scientist, Multimodal Foundation Models and Robotics
$192k - $357k • Remote • US, CA, Santa Clara
Posted 1d ago
Job Location
US, CA, Santa Clara
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Research Engineer
About the job
NVIDIA is seeking an outstanding research scientist to build humanoid robot foundation models and systems within the Generalist Embodied Agent Research (GEAR) group. The mission is to create general-purpose embodied agents capable of exploring and mastering complex skills in both virtual and physical environments. You will collaborate with a research team known for influential work in multimodal foundation models, large-scale robot learning, game AI, and physical simulation, contributing to significant research projects and product roadmaps, including milestones like Project GR00T, a foundation model for humanoid robots.
Responsibilities
- Design and implement novel AI algorithms and models for general-purpose humanoid robots and embodied agents.
- Develop large-scale AI training and inference methods for foundation models.
- Optimize and deploy AI models in physical simulation and on robot hardware.
- Collaborate with research and engineering teams to transfer research to products and services.
Requirements
- Ph.D. in Computer Science/Engineering, Electrical Engineering, or equivalent research experience.
- 5 years of relevant work/research experience in Multimodal Foundation Models and/or Robotics.
- Hands-on training experience and publications in LLMs, large vision-language models, video generative models/diffusion algorithms, or action-based transformers.
- Outstanding engineering skills in rapid prototyping and model training frameworks (PyTorch, Jax, Tensorflow).
- Python proficiency required; C++ and CUDA proficiencies are a plus.
- Excellent skills in working with large-scale machine learning/AI systems and compute infrastructure.
- Hands-on training experience and publications in robot learning (e.g., reinforcement learning, imitation learning, classical control methods).
- Strong programming skills in Python, C++, ROS, and machine learning frameworks like PyTorch.
- Deep understanding of robot kinematics, dynamics, and sensors.
- Ability to safely operate robot hardware, lab equipment, and tools.
- Knowledge of control methods (PID, model predictive control, whole-body control).
- Familiarity with physics simulation frameworks (MuJoCo, Isaac Sim).
- Robot hardware design and hands-on building experience.
Benefits
- Equity
- Benefits