Principal AI Research Scientist, Research Director - AI Scaling

About the job

Responsibilities

• Lead and grow a multidisciplinary research team focused on foundational and applied AI problems, with an emphasis on LLM scaling, efficiency, and systems performance.
• Define the scaling research roadmap, prioritizing advances in foundation model efficiency and large-scale training and inference.
• Drive algorithmic innovations for large-scale neural network training and inference, including novel optimizers, low-precision techniques, and model adaptation methods.
• Optimize end-to-end ML systems for distributed training and RL, memory efficiency, and compute efficiency in collaboration with systems and platform teams.
• Partner with product and engineering to translate research breakthroughs into customer-impacting capabilities.
• Foster a culture of scientific excellence, reproducible experimentation, and knowledge sharing.
• Represent Databricks AI research externally through publications, conference talks, and collaborations.
• Mentor and develop talent, providing technical guidance and career development support.
• Define and lead independent research programs on foundation model efficiency.
• Oversee the design and execution of large-scale experiments, including benchmarking and evaluating trade-offs.
• Work hands-on with a team on high-quality, efficient code in Python and PyTorch for research implementation and prototyping.
• Collaborate with distributed systems and infra teams to push the limits of distributed training, parallelism strategies, memory management, and hardware utilization.
• Establish metrics, evaluation protocols, and best practices for scaling-focused research.
• Champion responsible and robust deployment of scaling innovations, ensuring model behavior, reliability, and safety.

Requirements

• Proven ability to lead a research team to develop novel techniques for foundation model efficiency and related topics, with a strong track record of industry impact.
• Deep expertise in at least one of: generative AI, LLMs, distributed ML systems, model optimization, or responsible AI, with a strong emphasis on scaling and efficiency for large-scale neural networks.
• Strong programming skills and demonstrated ability to write high-quality, efficient code in Python and PyTorch for research implementation and experimentation.
• Demonstrated ability to translate research innovation into scalable product capabilities in partnership with product and engineering teams.
• Excellent communication, leadership, and stakeholder management skills, with experience influencing cross-functional roadmaps and aligning research with business impact.
• Prior work at the intersection of systems and ML, such as distributed training frameworks, is a plus.