Software Engineer, AI Infrastructure - LVM Inference & Evaluation

About the job

Responsibilities

• Design, build, and maintain AI infrastructure for real-time computer vision, LLM, LVM, and multimodal inference workloads.
• Build scalable systems for running state-of-the-art models across large volumes of video and sensor data.
• Optimize inference performance for latency, throughput, GPU utilization, reliability, and cost.
• Develop evaluation harnesses and benchmarking systems to measure model quality and system performance.
• Build infrastructure for continuous model evaluation, experimentation, and deployment.
• Partner with research scientists to productionize advances in computer vision, LLMs, LVMs, RAG, and multimodal AI.
• Improve model-serving architecture, including batching, caching, routing, quantization, and hardware utilization.
• Develop data engines and feedback loops for collecting training data and evaluating model behavior.
• Create observability, monitoring, and debugging tools for production AI systems.
• Define best practices for deploying, evaluating, and operating AI systems in enterprise environments.

Requirements

• 2+ years of industry experience building infrastructure, distributed systems, machine learning platforms, or production AI systems.
• BS/MS in Computer Science or equivalent practical experience.
• Strong programming background, especially in Python, with solid software engineering fundamentals.
• Experience designing and building scalable machine learning infrastructure for training, inference, evaluation, and deployment.
• Hands-on experience running deep learning models in production, ideally including LLMs, LVMs, vision-language models, or multimodal models.
• Strong understanding of inference optimization techniques (batching, caching, quantization, parallelism, GPU utilization, latency reduction).
• Experience with model-serving frameworks (e.g., vLLM, Triton Inference Server).
• Experience building evaluation frameworks, test harnesses, or benchmarks.
• Strong background in machine learning and deep learning; computer vision experience is a plus.
• Experience designing data engines or pipelines for collecting and managing training/evaluation data.
• Familiarity with integrating advanced AI systems (LLMs, LVMs, RAG, embedding models) into production applications.
• Experience with cloud infrastructure, containers, orchestration, distributed systems, and GPU workloads.
• Strong collaboration and communication skills.
• Proactive problem-solving ability and ownership mindset.