Sr Director of Software Engineering- AI Infrastructure Platforms
Salary not listed • Seattle, WA, United States • Full-time
Posted 9d ago
No salary is listed. Many US states require employers to post a pay range, so ask about pay early.
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
As a Senior Director of Software Engineering at JPMorgan Chase, you will lead multiple technical areas and manage departments responsible for delivering a unified AI infrastructure layer across on-premises environments, public cloud, and emerging accelerated-compute vendors. You will collaborate across AI/ML engineering, infrastructure, security, and vendor teams to ensure the firm remains at the forefront of AI platform capabilities and operational excellence. In this role, you will own training and experimentation on a Kubernetes-standardized platform, acting as an active design partner to guide architectural trade-offs and ensure designs translate into reliable, secure, and operable systems at enterprise scale.
Responsibilities
- Lead multiple technology and platform implementations across departments to deliver firmwide AI infrastructure objectives, focusing on training and experimentation platforms at enterprise scale.
- Own the design, delivery, and evolution of a Kubernetes-first AI Infrastructure platform, including native support for batch and distributed training & inference jobs, lifecycle management, and failure recovery.
- Standardize AI developer workflows for experimentation, enabling self-service job submission, reusable templates, reproducibility, and consistent runtime behavior across hybrid environments.
- Build and evolve platform APIs and automation, including Kubernetes controllers and operators, to ensure the platform is safe, scalable, and easy to adopt.
- Drive measurable improvements in GPU availability and utilization through reliability engineering, fleet readiness, and accelerated capacity onboarding.
- Define and implement governance-based scheduling and placement strategies, including multitenant GPU quotas, priority, admission control, reservation patterns, preemption policies, and fragmentation reduction.
- Embed enterprise-grade security, risk, and control requirements into platform defaults, including IAM, RBAC, secrets management, audit logging, policy enforcement, and network segmentation.
- Drive operational excellence by establishing SLIs/SLOs, managing error budgets, leading incident management, forecasting capacity, and delivering end-to-end platform observability.
- Act as the primary interface with senior leaders and stakeholders, driving alignment and consensus across competing priorities.
- Lead multiple engineering teams and managers, building a high-performing organization with strong engineering standards and a culture of accountability.
Requirements
- 15+ years of engineering experience, including 8+ years of senior engineering leadership managing managers.
- Demonstrated experience delivering platform products with strong adoption, reliability, and operational maturity.
- Experience developing and leading large, cross-functional engineering teams in complex enterprise environments.
- Proven track record of leading complex initiatives supporting distributed system design, testing, and operational stability at scale.
- Deep hands-on expertise with Kubernetes-based platforms, including multi-tenancy, RBAC, admission control, network policy, multi-cluster operations, and controller/operator design patterns.
- Experience supporting AI training and experimentation platforms, including PyTorch, distributed training concepts, and frameworks like Ray.
- Familiarity with Slurm or equivalent HPC/batch schedulers and core concepts.
- Understanding of modern AI inference stacks and how serving constraints influence platform design.
- Strong understanding of GPU infrastructure fundamentals, including NVIDIA ecosystem capabilities and scheduling constraints.
- Extensive practical experience with cloud-native technologies and hybrid infrastructure environments.
- Experience hiring, developing, coaching, and retaining high-performing engineering talent.