Senior AI Platform Engineer
$170k - $210k • Remote • Boston Hub • FullTime
Posted 9d ago
Job Location
Boston Hub
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
Code Metal is seeking a Senior Software Engineer for its AI Platform team. This role involves designing, building, and operating the foundational AI enablement stack that powers the company's AI-driven code solutions. You will contribute to critical components such as GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. While this is an internal platform, it is built to product standards. The position offers the opportunity to eventually own a specific area, like model serving or agentic infrastructure, while also contributing broadly across the stack. The role requires a blend of strong software engineering skills with solid data science and AI research fundamentals, including occasional experimental work to inform platform decisions.
Responsibilities
- Own one or more platform components end-to-end, from design to production operation.
- Build and operate services within your focus area and contribute to other parts of the stack.
- Write clean, well-tested code, ensuring AI-generated code meets high standards for correctness, modularity, and testing.
- Debug production issues in owned components, including latency, GPU memory, or agent failures.
- Scope and estimate efforts with internal customers and proactively identify risks.
- Review teammates' code and designs, and assist in onboarding new engineers.
- Conduct focused benchmarks and experiments to support platform decisions.
Requirements
- Production-grade Python and strong platform engineering fundamentals (API/service design, distributed systems, containers, Kubernetes, CI/CD, testing).
- Production experience in at least one focus area: model serving (LLM inference, gateways), agentic infrastructure (agent systems, context engineering, RAG, embeddings, search), or observability/experimentation (instrumentation, SLOs, evaluation, experiment tracking).
- Solid data science and AI research fundamentals, including understanding of transformers and LLM inference, experiment design, benchmarking, and model evaluation.
- Working familiarity with PyTorch and Hugging Face.
- Experience owning a service or component in production, including debugging and reliability improvement.
- Experience writing design docs, reviewing code, and mentoring or onboarding teammates.
Benefits
- Pay at the upper end of the salary range, depending on experience.
- Health care plan with 100% premium coverage (medical, dental, vision).
- 401k with 5% matching.
- Uncapped paid vacation, plus sick days and public holidays.