Software Engineer, ML Developer Experience
$226k - $283k • Remote • San Francisco • FullTime
Posted 1mo ago
About the job
Anyscale is seeking a Software Engineer for its ML Developer Experience (MLDevX) team. This role focuses on the user-facing interfaces of the Anyscale platform, enabling users and coding agents to discover, configure, run, observe, debug, and productionize AI workloads. You will work across the full stack, from developer tooling to cloud infrastructure and the Ray runtime, making complex distributed systems more intuitive and manageable for developers. This is a high-impact individual contributor role with end-to-end ownership, involving direct collaboration with users and field teams to identify problems, shape product and technical design, and build and operate solutions. The scope covers the entire AI workload and developer loop, from data preparation and fine-tuning to code submission, monitoring, and optimization.
Responsibilities
- Build next-generation developer tooling and MLOps capabilities for Ray, catering to both developers and coding agents.
- Develop an agent-first CLI and cohesive SDK, API, and MCP surfaces with features like self-discovery and structured errors.
- Improve the Anyscale Workspaces stack to streamline the path from local code to distributed execution.
- Build tools and frameworks for the AI development lifecycle, including data preparation, fine-tuning, evaluation, and serving.
- Develop the process for moving trained models to reliable production endpoints, including registration, deployment, and performance benchmarking.
- Surface observability across interfaces to help users diagnose failures in jobs, tasks, actors, nodes, and GPUs.
- Design open integrations using standards like OpenAI-compatible APIs and OpenTelemetry.
- Design and operate highly available backend services and platform architecture for serverless and bring-your-own-cloud environments.
- Collaborate with users and field teams to scope, ship, and iterate on the product.
- Work with distributed systems and machine learning experts to advance AI infrastructure.
Requirements
- Bachelor's degree in Computer Science, Engineering, or equivalent practical experience.
- 5+ years of experience writing high-quality production code.
- Solid background in algorithms, data structures, and system design.
- Experience with modern machine learning tooling such as PyTorch and MLflow.
- Hands-on experience building and operating highly available services in production.
- Strong product instincts and a track record of shipping developer-facing tools.
- Experience building and maintaining open-source projects (bonus).
- Experience building and operating machine learning infrastructure in production (bonus).
- Experience building highly available serving systems (bonus).
- Experience using Ray (bonus).