Inference Software Engineer
$175k - $400k • San Jose • FullTime
Posted 1y ago
About the job
Etched is building hardware for frontier intelligence, focusing on inference. We co-design chips, racks, software, and manufacturing to deliver best-in-class throughput and latency. Backed by significant investment and staffed by leading engineers, Etched is redefining the infrastructure layer for the fastest-growing industry. We are a fully in-person team in San Jose, valuing engineering skills and expecting technical staff to contribute across disciplines, blurring the lines between engineering and research.
Responsibilities
- Support porting state-of-the-art models to our architecture, building programming abstractions and testing capabilities for rapid iteration.
- Build, enhance, and scale our runtime, including multi-node inference, intra-node execution, state management, and error handling.
- Optimize routing and communication layers using our collectives.
- Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.
Requirements
- Proficiency in C++ or Rust.
- Understanding of performance-sensitive or complex distributed software systems (e.g., Linux internals, accelerator architectures, compilers, high-speed interconnects).
- Familiarity with PyTorch or JAX.
- Experience porting applications to non-standard accelerator hardware or platforms.
- Experience developing low-latency, high-performance applications using kernel-level and user-space networking stacks.
- Deep understanding of distributed systems concepts, algorithms, and challenges.
- Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).
- Experience building applications with extensive SIMD optimizations.
Benefits
- Medical, dental, and vision packages with generous premium coverage
- $500 per month credit for waiving medical benefits
- $2.5k per month housing subsidy for those living within walking distance of the office
- Relocation support for those moving to San Jose (Santana Row)
- Various wellness benefits covering fitness, mental health, and more
- Daily lunch and dinner in our office
- Unlimited compute budget subject to ROI justification