Staff AI Platform Engineer
$205k - $250k • Remote • Boston Hub • FullTime
Posted 12d ago
Job Location
Boston Hub
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
Code Metal is seeking a Staff AI Platform Engineer to architect and build the foundational AI enablement stack that internal engineering teams rely on. This role involves designing and operating production systems for GPU inference serving, model gateways, agent harnesses, context management, and observability. While primarily an engineering role focused on building and operating production systems, it also requires solid data science and AI research fundamentals to work closely with the AI Research team and conduct experiments when necessary. The platform is built to product standards, ensuring reliability and scalability for AI-driven code transpilation, modernization, optimization, modeling, and simulation solutions.
Responsibilities
- Set the technical direction and architecture for the AI platform and its team.
- Deploy, benchmark, and tune production inference for open-weight models on runtimes like vLLM, SGLang, and TensorRT-LLM.
- Own the model gateway for accessing self-hosted and commercial models, managing auth, routing, failover, quotas, and cost attribution.
- Design reusable agent harnesses and orchestration primitives for reliable, verifiable workflows.
- Build context-engineering services for memory, retrieval, and data discovery for agents.
- Instrument the stack end-to-end with traces and metrics, and build experiment-tracking and artifact layers.
- Design for productization, including versioned APIs and security, and communicate the roadmap to business and product teams.
Requirements
- Production-grade Python and strong platform engineering fundamentals (API/service design, distributed systems, containers, Kubernetes, CI/CD, testing).
- Shipped production agentic systems and understand their reliability challenges.
- Experience with context engineering (RAG, embeddings, vector/hybrid search, agent memory), ideally over code or large technical corpora.
- Experience instrumenting services (e.g., OpenTelemetry) and operating AI services against SLOs.
- Solid data science and AI research fundamentals (transformers, LLM inference, experiment design, benchmarking, model evaluation).
- Working familiarity with PyTorch and Hugging Face, and experience with language model fine-tuning, evaluation, or serving.
- Staff-level technical leadership experience (owning architecture, writing design docs/RFCs, turning ambiguous needs into roadmaps, mentoring engineers).
Benefits
- Health care plan with 100% premium coverage (medical, dental, vision).
- 401k with 5% matching.
- Uncapped vacation, plus sick days and public holidays.
- Flexible hybrid or remote work arrangement.
- Relocation assistance for qualifying employees.