Member of Technical Staff - Inference Runtime
$220k - $300k • Remote • San Francisco • FullTime
Posted 9d ago
About the job
Modal's Inference Runtime team is responsible for the container runtime stack that powers inference and training workloads across our infrastructure. We operate at the intersection of Linux, containers, filesystems, storage, GPU drivers, and distributed systems. Our primary objective is to ensure that demanding ML workloads start quickly, run efficiently, and remain securely isolated, regardless of their resource requirements. We are seeking a systems engineer with a passion for deep-stack work to build production runtime infrastructure in Rust and Go, tackle complex Linux and performance issues, and contribute to the architecture of our container platform. This role involves close collaboration with gVisor maintainers and potential contributions to the upstream runtime.
Responsibilities
- Accelerate container startup, checkpoint, and restore for large inference and training workloads.
- Develop multi-GPU and accelerator-aware snapshotting, including efficient GPU memory and RDMA handling.
- Design zero-copy and low-copy data paths between container memory, filesystems, storage, and the runtime.
- Optimize large snapshot pipelines using parallel uploads, direct I/O, incremental snapshots, and advanced memory handling.
- Enhance container image and filesystem performance across EROFS, FUSE, page caches, overlay filesystems, and remote storage.
- Extend the sandboxed runtime to support new GPUs, drivers, profiling tools, and device capabilities on NVIDIA and AMD hardware.
- Debug complex failures involving system calls, virtual memory, process lifecycle, kernel behavior, GPU drivers, and container isolation.
- Safely deploy runtime and kernel changes across a heterogeneous fleet using compatibility controls, scheduling constraints, feature flags, and observability.
- Address ambiguous production problems across the runtime, scheduler, storage, and GPU infrastructure, from investigation to deployment.
Requirements
- Strong Linux systems knowledge, including processes, virtual memory, filesystems, system calls, scheduling, namespaces, cgroups, and signals.
- Experience building or debugging container runtimes, sandboxes, Linux kernel, or similar low-level infrastructure.
- Proficiency in Rust, Go, or another systems language, with a willingness to become productive in both.
- Experience profiling and optimizing systems where memory movement, I/O, synchronization, or kernel interactions are performance bottlenecks.
- Comfort debugging across abstraction layers, from application behavior to runtimes, drivers, and the kernel.
- Ability to translate loosely defined production problems into well-designed, reliable systems.
- A desire to own critical infrastructure and collaborate closely with internal teams and customers.