Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)
$220k - $270k • US • Full-time
Posted 4d ago
About the job
Accountabilities:
- Partner with customers and product teams to translate deployment requirements into structured performance studies, defining assumptions, scenarios, metrics, and actionable recommendations.
- Conduct objective, apples-to-apples performance comparisons between competing hardware platforms, including GPUs and other AI accelerators, identifying the factors driving differences in performance.
- Design, develop, and deliver a standalone performance-estimation tool that customers can use to evaluate their own AI inference workloads and deployment configurations.
- Build and maintain deployment-level performance models for large-scale inference systems, accounting for tensor, pipeline, expert, and data parallelism strategies.
- Evaluate the impact of batch sizes, context lengths, workload characteristics, and mixed hardware configurations involving different accelerator platforms.
- Model network and system-level performance, including interconnect latency, bandwidth, topology, and redundancy, to understand how these factors influence throughput and scalability as the number of chips increases.
- Develop and extend analytical and simulation-based deployment models that leverage existing chip performance simulators, ensuring that all deployment estimates remain consistent with a shared hardware performance model.
- Analyze system configurations and identify performance bottlenecks, scalability limitations, and opportunities to optimize deployment efficiency.
- Validate performance projections against actual hardware measurements and benchmark results, continuously calibrating models as hardware architectures, software environments, and workloads evolve.
- Collaborate closely with chip modeling and engineering teams to incorporate new hardware capabilities and improve the accuracy of deployment predictions.
- Communicate technical findings clearly to customers and internal stakeholders, explaining performance estimates, key assumptions, influencing factors, and confidence levels.
- Depending on experience and seniority, take ownership of either the overall modeling architecture or individual model deployment studies and the development of customer-facing performance tools.
Requirements:
- At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, high-performance computing (HPC), or data-center infrastructure.
- Demonstrated experience developing analytical or queueing-based performance models and calibrating predictions against real-world system measurements.
- Deep understanding of large language model (LLM) inference at the systems level, including prefill and decode phases, KV cache management, parallelism strategies, mixture-of-experts (MoE) routing, batching, time-to-first-token (TTFT), and tokens-per-second performance metrics.
- Strong knowledge of large-scale deployment architectures and the factors affecting inference throughput, latency, hardware utilization, and scalability.
- Solid understanding of data-center networking concepts, including interconnect latency, bandwidth, network topology, and their impact on distributed system performance.
- Good knowledge of GPU architecture and performance characteristics, with the ability to evaluate and compare different accelerator platforms.
- Experience building analytical models, conducting performance experiments, interpreting benchmark results, and translating technical findings into practical deployment recommendations.
- Ability to work with incomplete technical information, establish and document assumptions, and deliver useful performance estimates with clearly communicated uncertainty and error margins.
- Strong programming skills in Rust, or expert-level C/C++ proficiency combined with a willingness to work extensively in Rust.
- Proficiency in Python for performance analysis, experimentation, data processing, and visualization.
- Excellent analytical and problem-solving abilities, with a strong understanding of the relationship between hardware architecture, workload characteristics, and system-level performance.
- Excellent written and verbal communication skills, including the ability to explain deployment sizing recommendations, performance drivers, and confidence levels to both technical teams and customers.
- Hands-on experience sizing, benchmarking, or optimizing production inference deployments across multiple accelerators is highly desirable.
- Previous experience developing customer-facing performance estimation, infrastructure sizing, or total cost of ownership (TCO) tools is a strong advantage.
- Candidates with more extensive experience may be considered for Principal-level opportunities, with responsibilities and technical ownership adapted to their expertise and seniority.
Benefits:
- Competitive annual base salary based on experience and seniority:
- Senior: $220,000–$270,000.
- Principal: $270,000–$350,000.
- Competitive equity opportunities as part of the overall compensation package.
- Flexibility to offer compensation above the stated salary ranges for candidates with exceptional expertise, specialized skills, or significant experience.
- Fully company-paid medical, dental, and vision insurance for employees and their dependents.
- Company-paid life and disability insurance, with additional voluntary coverage options.
- Access to supplemental hospital, critical illness, and accident insurance.
- Unlimited paid time off, encouraging employees to rest, recharge, and maintain a healthy work-life balance.
- 13 paid company holidays annually.
- Remote-first working environment, offering flexibility to work from anywhere in the United States.
- Company-provided computer and home office setup.
- 401(k) retirement plan with company matching, available from the first day of employment.
- Opportunity to work at the intersection of advanced AI silicon, large-scale systems architecture, and customer-facing engineering.
- Direct access to chip performance simulators, real hardware measurements, and cutting-edge inference technologies.
- Close collaboration with experienced engineers and architects developing next-generation AI accelerator platforms.
- Opportunity to build impactful performance models and tools that directly influence customer infrastructure planning, hardware selection, and large-scale AI deployment decisions.
