Principal System Software Engineer, AI Inference Execution
$195k - $285k • Santa Clara • FullTime
Posted 2mo ago
Job Location
Santa Clara
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
d-Matrix is at the forefront of software and hardware innovation, aiming to unleash the potential of generative AI. We are seeking passionate individuals driven by execution to join our inclusive and collaborative team. This role is for a Principal System Software Engineer focused on AI Inference Execution, contributing to the productization of our AI compute engine's software stack. You will be instrumental in developing, enhancing, and maintaining next-generation AI deployment software, leveraging your expertise in full-stack toolchains and hardware-software co-design to optimize performance and scale deliverables within tight development windows.
Responsibilities
- Develop, enhance, and maintain the next-generation AI deployment software stack for the AI compute engine.
- Optimize and trade-off various aspects of hardware-software co-design.
- Build and scale software deliverables within tight development windows.
- Collaborate with system software experts to build out deployment infrastructure.
- Work closely with ML, compiler, and hardware experts within the company.
Requirements
- BS in Computer Science, Engineering, Math, Physics, or related degree with 12+ years of industry software development experience (MS preferred with 6+ years).
- Strong grasp of system software, data structures, computer architecture, and machine learning fundamentals.
- Proficient in C/C++/Python development in a Linux environment using standard development tools.
- Experience with distributed, high-performance software design and implementation.
- Self-motivated team player with a strong sense of ownership and leadership.
- Experience with inference servers/model serving frameworks (e.g., TensorRT-LLM, vLLM, SGLang).
- Experience with deep learning frameworks (e.g., PyTorch, TensorFlow).
- Experience with deep learning runtimes (e.g., ONNX Runtime, TensorRT).
- Experience with distributed systems collectives (e.g., NCCL, OpenMPI).
- Experience with software testing fundamentals.
- Experience deploying ML workloads (LLMs, VLMs, NLP, etc.) on distributed systems.
- Experience with Kubernetes, Ray, or other MLOps tools.
- Prior startup, small team, or incubation experience.
- Work experience at a cloud provider or AI compute/subsystem company.