New: upload your resume and get matched to AI jobs. Free to start →

Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)

$220k - $270k • US • Full-time

Posted 4d ago

Job Location

US

Tech Stack

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Engineer

About the job

Accountabilities:

  • Partner with customers and product teams to translate deployment requirements into structured performance studies, defining assumptions, scenarios, metrics, and actionable recommendations.
  • Conduct objective, apples-to-apples performance comparisons between competing hardware platforms, including GPUs and other AI accelerators, identifying the factors driving differences in performance.
  • Design, develop, and deliver a standalone performance-estimation tool that customers can use to evaluate their own AI inference workloads and deployment configurations.
  • Build and maintain deployment-level performance models for large-scale inference systems, accounting for tensor, pipeline, expert, and data parallelism strategies.
  • Evaluate the impact of batch sizes, context lengths, workload characteristics, and mixed hardware configurations involving different accelerator platforms.
  • Model network and system-level performance, including interconnect latency, bandwidth, topology, and redundancy, to understand how these factors influence throughput and scalability as the number of chips increases.
  • Develop and extend analytical and simulation-based deployment models that leverage existing chip performance simulators, ensuring that all deployment estimates remain consistent with a shared hardware performance model.
  • Analyze system configurations and identify performance bottlenecks, scalability limitations, and opportunities to optimize deployment efficiency.
  • Validate performance projections against actual hardware measurements and benchmark results, continuously calibrating models as hardware architectures, software environments, and workloads evolve.
  • Collaborate closely with chip modeling and engineering teams to incorporate new hardware capabilities and improve the accuracy of deployment predictions.
  • Communicate technical findings clearly to customers and internal stakeholders, explaining performance estimates, key assumptions, influencing factors, and confidence levels.
  • Depending on experience and seniority, take ownership of either the overall modeling architecture or individual model deployment studies and the development of customer-facing performance tools.

Requirements:

  • At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, high-performance computing (HPC), or data-center infrastructure.
  • Demonstrated experience developing analytical or queueing-based performance models and calibrating predictions against real-world system measurements.
  • Deep understanding of large language model (LLM) inference at the systems level, including prefill and decode phases, KV cache management, parallelism strategies, mixture-of-experts (MoE) routing, batching, time-to-first-token (TTFT), and tokens-per-second performance metrics.
  • Strong knowledge of large-scale deployment architectures and the factors affecting inference throughput, latency, hardware utilization, and scalability.
  • Solid understanding of data-center networking concepts, including interconnect latency, bandwidth, network topology, and their impact on distributed system performance.
  • Good knowledge of GPU architecture and performance characteristics, with the ability to evaluate and compare different accelerator platforms.
  • Experience building analytical models, conducting performance experiments, interpreting benchmark results, and translating technical findings into practical deployment recommendations.
  • Ability to work with incomplete technical information, establish and document assumptions, and deliver useful performance estimates with clearly communicated uncertainty and error margins.
  • Strong programming skills in Rust, or expert-level C/C++ proficiency combined with a willingness to work extensively in Rust.
  • Proficiency in Python for performance analysis, experimentation, data processing, and visualization.
  • Excellent analytical and problem-solving abilities, with a strong understanding of the relationship between hardware architecture, workload characteristics, and system-level performance.
  • Excellent written and verbal communication skills, including the ability to explain deployment sizing recommendations, performance drivers, and confidence levels to both technical teams and customers.
  • Hands-on experience sizing, benchmarking, or optimizing production inference deployments across multiple accelerators is highly desirable.
  • Previous experience developing customer-facing performance estimation, infrastructure sizing, or total cost of ownership (TCO) tools is a strong advantage.
  • Candidates with more extensive experience may be considered for Principal-level opportunities, with responsibilities and technical ownership adapted to their expertise and seniority.

Benefits:

  • Competitive annual base salary based on experience and seniority:
  • Senior: $220,000–$270,000.
  • Principal: $270,000–$350,000.
  • Competitive equity opportunities as part of the overall compensation package.
  • Flexibility to offer compensation above the stated salary ranges for candidates with exceptional expertise, specialized skills, or significant experience.
  • Fully company-paid medical, dental, and vision insurance for employees and their dependents.
  • Company-paid life and disability insurance, with additional voluntary coverage options.
  • Access to supplemental hospital, critical illness, and accident insurance.
  • Unlimited paid time off, encouraging employees to rest, recharge, and maintain a healthy work-life balance.
  • 13 paid company holidays annually.
  • Remote-first working environment, offering flexibility to work from anywhere in the United States.
  • Company-provided computer and home office setup.
  • 401(k) retirement plan with company matching, available from the first day of employment.
  • Opportunity to work at the intersection of advanced AI silicon, large-scale systems architecture, and customer-facing engineering.
  • Direct access to chip performance simulators, real hardware measurements, and cutting-edge inference technologies.
  • Close collaboration with experienced engineers and architects developing next-generation AI accelerator platforms.
  • Opportunity to build impactful performance models and tools that directly influence customer infrastructure planning, hardware selection, and large-scale AI deployment decisions.