Staff Machine Learning Engineer, ML Platform
$184k - $314k • San Francisco • Full-time
Posted 1mo ago
Job Location
San Francisco
Also open in Chicago, Austin, New York City
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
Machine Learning Engineer
About the job
Braze is seeking a Staff Machine Learning Engineer to join our Predictive and Generative AI (PGAI) team, which focuses on creating ML and AI-enhanced marketing solutions for personalized customer experiences. These solutions are deployed as production systems at a global scale, involving distributed pipelines for model training and high-throughput APIs for predictions. The Staff Engineer will be responsible for the underlying platform, ensuring that deploying, operating, and scaling ML at Braze is fast, safe, and efficient. This is a hands-on role where you will own complex infrastructure initiatives from design to production, including multi-region model serving, customer-specific model pipelines, and CI/deployment tooling. You will also define the platform's technical vision, set standards for model training, deployment, serving, and observation, lead incident response, and drive reliability and cost efficiency. Collaboration with partner teams on shared infrastructure and data systems is also a key aspect of this role, as is raising the team's engineering quality through reviews and mentoring.
Responsibilities
- Identify and drive transformative initiatives to improve ML production operations, such as replatforming queueing/orchestration or overhauling deployment.
- Build and ship complex infrastructure initiatives, including multi-region model serving fleets and customer-specific model pipelines.
- Own the platform's technical vision and production quality, setting direction for model lifecycle management and leading incident response.
- Drive initiatives that span multiple teams, managing technical relationships with partner teams for shared infrastructure.
- Raise engineering quality through design and code reviews, and mentor other senior engineers and data scientists.
- Connect technical decisions to business outcomes and represent the team's technical perspective to leadership.
Requirements
- 8+ years of experience building and operating distributed systems in production, with expertise in deployment and operations.
- Experience designing services for scale and reliability, owning CI/CD and infrastructure as code.
- Hands-on experience with ML workloads in production, including training pipelines, model serving, feature systems, or ML platform tooling.
- Proven technical leadership, including owning team direction, leading cross-team initiatives, and growing senior engineers.
- Deep working knowledge of Kubernetes and cloud infrastructure, including IAM, networking, and cost management.
- Effective verbal and written communication skills to build consensus and drive decision-making.
- Experience with queueing and orchestration systems (e.g., Celery, RabbitMQ, Kafka, Ray) is a plus.
- Experience with ML platform tooling (e.g., MLflow, model registries, feature stores, ML observability) is a bonus.
- Familiarity with Python, Ruby on Rails, MongoDB, Redis, and Kubernetes is a plus.
- Experience operating under compliance regimes like SOX or HIPAA is a bonus.
- Customer engagement, personalization, or marketing technology domain experience is a bonus.
Benefits
- Competitive compensation
- Equity grants (RSUs)
- Retirement and Employee Stock Purchase Plans
- Flexible paid time off
- Comprehensive medical, dental, vision, life, and disability insurance
- Fertility benefits
- Equal paid parental leave
- Professional development support (career pathing, learning platforms, yearly stipend)
- Curated in-office employee experience