Senior AI Researcher- Pre-training (f/m/d)
Heidelberg • FullTime
Posted 8d ago
Job Location
Heidelberg
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
AI Research Engineer
About the job
Aleph Alpha is seeking a Senior AI Researcher to join their Pre-training team, focusing on advancing the architecture and training of next-generation foundation models. This role involves designing inference-efficient architectures, optimizing large-scale training recipes, and working with thousands of NVIDIA GPUs. The position is ideal for someone excited about the intersection of research and engineering, translating mathematical reasoning into principled training decisions from small-scale experiments to large-scale runs. The researcher will influence model quality, run reliability, and efficiency by combining rigorous experimental design with high-quality production code.
Responsibilities
- Optimize core training recipe elements (optimizers, schedules, initialization) and design PyTorch-based architectural improvements for convergence, stability, and efficiency.
- Develop hyperparameter scaling laws and scale-up methodologies to predict multi-thousand-GPU behavior from small-scale proxy experiments.
- Investigate and resolve complex convergence issues and distributed system failures, including loss spikes, divergence, communication bottlenecks, race conditions, and synchronization errors.
- Collaborate with Compute Performance, Data, Evaluation, and Post-Training teams to align the model lifecycle with hardware constraints, memory bandwidth, and communication topologies.
Requirements
- Proficiency in Python and deep familiarity with PyTorch-based training workflows.
- Strong track record in machine learning research and software engineering, demonstrated through shipped models, open-source contributions, or published research.
- Strong mathematical foundation for reasoning about optimization, scaling behavior, and training dynamics.
- Deep understanding of transformer training dynamics, optimization, and large distributed training jobs.
- Ability to design rigorous experiments, interpret noisy results, and translate empirical observations into robust training decisions.
- Hands-on experience pre-training large models (e.g., 7B+ parameters) on substantial infrastructure (e.g., 100+ GPU clusters).
- Application of strong software engineering practices, including writing maintainable, well-tested code and supporting reproducible experimentation.
- Ability to implement complex model architectures efficiently and debug across model code, training dynamics, and distributed systems.
- Effective collaboration within a research and engineering team and clear communication across departments.
- Ability to work in Germany and collaborate regularly on site in Heidelberg.
Benefits
- 30 days of paid vacation
- Access to a variety of fitness & wellness offerings via Wellhub
- Mental health support through nilo.health
- Substantially subsidized company pension plan
- Subsidized Germany-wide transportation ticket
- Budget for additional technical equipment
- Flexible working hours
- Hybrid working model
- JobRad® Bike Lease