Principal Research Scientist – Scaling
$280k - $350k • Remote • San Francisco, California
Posted 26d ago
Job Location
San Francisco, California
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Research Engineer
About the job
Databricks AI Research is seeking a Principal Research Scientist – Scaling to lead a team focused on advancing the state of the art in large-scale machine learning, particularly in LLM training and inference efficiency. This role involves defining and executing a research roadmap to improve how customers train, serve, and adapt LLMs at scale, working closely with product, data, and engineering leaders. The goal is to push the boundaries of LLM efficiency beyond existing models, exploring novel algorithms, systems, and infrastructure improvements. You will drive independent research agendas and collaborate on low-level implementation details to bring cutting-edge methods into production and enhance the Databricks AI platform.
Responsibilities
- Lead and grow a multidisciplinary research team focused on LLM scaling, efficiency, and systems performance.
- Define the scaling research roadmap aligned with strategic objectives, prioritizing foundation model efficiency and large-scale training/inference.
- Drive algorithmic innovations for large-scale neural network training and inference, including optimizers, low-precision techniques, and model adaptation.
- Optimize end-to-end ML systems for distributed training, RL, memory efficiency, and compute efficiency.
- Translate research breakthroughs into customer-impacting capabilities within the Databricks AI platform.
- Foster a culture of scientific excellence, reproducibility, and knowledge sharing.
- Represent Databricks AI research externally through publications, talks, and collaborations.
- Mentor and develop research scientists and engineers, providing technical and career guidance.
- Define and lead independent research programs on foundation model efficiency.
- Oversee the design and execution of large-scale experiments, evaluating trade-offs in quality, latency, throughput, and cost.
- Work hands-on with a team on high-quality, efficient code in Python and PyTorch for research implementation and prototyping.
- Collaborate with distributed systems and infra teams to optimize distributed training, parallelism, memory management, and hardware utilization for LLMs.
- Establish and drive adoption of metrics and best practices for scaling-focused research.
- Champion responsible and robust deployment of scaling innovations, ensuring model behavior, reliability, and safety.
Requirements
- Proven ability to lead a research team in developing novel techniques for foundation model efficiency with strong industry impact.
- Deep expertise in at least one of: generative AI, LLMs, distributed ML systems, model optimization, or responsible AI, with a focus on scaling and efficiency for large-scale neural networks.
- Strong programming skills and demonstrated ability to write high-quality, efficient code in Python and PyTorch for research implementation and experimentation.
- Demonstrated ability to translate research innovation into scalable product capabilities in partnership with product and engineering teams.
- Excellent communication, leadership, and stakeholder management skills.
- Experience influencing cross-functional roadmaps and aligning research with business impact.