Software Engineer, Storage Systems (Agent Infra)
$160k - $330k • Remote • San Francisco • FullTime
Posted 4h ago
About the job
Baseten is seeking a Software Engineer to join the Agent Infra team, focusing on building and enhancing Agent Drive, a distributed filesystem critical for agent sandboxes. This role involves developing a storage solution that supports thousands of concurrent sandbox mounts with strict data isolation and consistency. The work leverages advanced storage techniques like tiered caching, replication, and content-addressed data, with a focus on Linux filesystem internals. The team operates with a high degree of ownership, shipping features rapidly and contributing to the core infrastructure that powers leading AI companies.
Responsibilities
- Design and implement core Agent Drive features including metadata replication, volume placement, garbage collection, and FUSE client behavior.
- Address multi-attach correctness issues such as cache coherency across FUSE mounts, dentry/inode invalidation, write ordering, and crash consistency.
- Optimize read and write path tail latency through strategies like page cache management, writeback, prefetching, and metadata RPC batching.
- Integrate tenant isolation into the storage layer, enforcing quotas and mitigating noisy-neighbor effects.
- Take ownership of the storage aspects of sandbox startup and resume performance.
- Develop durable block storage for sandboxes in collaboration with other Baseten storage teams.
- Build observability into all storage layers, manage storage incidents, and implement permanent fixes.
- Contribute to the upstream open-source storage projects utilized by the team.
Requirements
- Strong systems programming background in Go, Rust, and/or C/C++, with the ability to read kernel code.
- Deep understanding of Linux filesystem and storage internals (VFS, page cache, dentry/inode caches, FUSE, block I/O stack, fsync semantics, crash consistency).
- Hands-on experience building distributed storage systems, content-addressable storage, or large-scale caching infrastructure.
- Experience profiling I/O, memory, and RPC bottlenecks, and optimizing tail latency in production environments.
- Ability to debug concurrency and partial failure correctness bugs.
- Demonstrated ability to move fast, take ownership, and solve problems end-to-end within a small team.
Benefits
- Competitive compensation and meaningful equity.
- 100% coverage of medical, dental, and vision insurance for employee and dependents (U.S. only).
- Flexible PTO policy with company-wide Winter Break.
- Paid parental leave.
- Fertility and family-building stipend.
- Company-facilitated 401(k) (U.S. only).
- Exposure to a variety of ML startups for learning and networking.