Senior System Software Engineer - Local AI
Salary not listed • India, Pune • Full-time
Posted 12d ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
The Local AI Inference team is seeking a Senior System Software Engineer to lead the development of robust system-level software and infrastructure for deploying, benchmarking, and qualifying high-quality AI applications and models across Linux and Windows systems powered by RTX and DGX-class GPUs. You will partner with inference engineers, model developers, and AI application teams to build reliable, scalable infrastructure that enables reproducible evaluation and improves local AI performance. Using AI tools and developing autonomous agents, you will accelerate infrastructure development, automate operations, and help engineers diagnose complex system issues.
Responsibilities
- Define requirements, design, and maintain shared infrastructure for deploying, benchmarking, and qualifying AI workloads across inference backends (e.g., llama.cpp, Ollama, PyTorch, vLLM, WinML).
- Build Git-based CI infrastructure for automated builds, integration testing, and regression detection across hardware, operating systems, and runtime configurations.
- Develop model management services for downloading, versioning, caching, and synchronizing checkpoints and dependencies.
- Design databases, backend services, and data pipelines to store and query model metadata, benchmark results, system telemetry, and execution artifacts.
- Create dashboards and reports to help engineers compare inference configurations, identify regressions, and investigate performance bottlenecks.
- Develop and operate server infrastructure, including environment provisioning, workload scheduling, resource management, and failure recovery.
- Develop autonomous AI agents for infrastructure management, workload orchestration, monitoring, and diagnostics.
- Maintain Git repositories and companion websites with current code, documentation, model recommendations, and evaluation results.
- Collaborate with inference developers to implement instrumentation and debugging capabilities, resolve system issues, and improve infrastructure efficiency and scalability.
Requirements
- B.Tech or higher degree in Computer Science, Computer Engineering, Information Technology, Software Engineering, or a related field, or equivalent experience.
- 5+ years of relevant software development experience.
- Strong programming skills in Python, C/C++, C#, or another systems programming language.
- Experience developing system software, backend services, or infrastructure supporting engineering development.
- Understanding of system APIs, multithreading, process and resource management, and systems debugging.
- Hands-on experience with Git-based CI, databases, SQL, and data modeling.
- Strong analytical and problem-solving skills, with the ability to independently own projects and manage competing priorities.
- Excellent written and verbal communication skills.
Benefits
- Competitive salaries
- Generous benefits package