Senior Applied Research Scientist - Foundation Models

About the job

Responsibilities

• Design and optimize transformer-based vision-language models for understanding images, videos, and text, optimizing for real-time inference.
• Manage the full training pipeline, including pre-training on image-text data and fine-tuning for physical security applications.
• Apply techniques like distillation, quantization, and pruning to reduce model size and latency for efficient edge deployment.
• Utilize and extend state-of-the-art open-source models, prototyping new architectures and training methods.
• Collaborate with engineering and product teams to integrate models into the platform and iterate based on feedback.
• Conduct research and experimentation in vision, NLP, and multimodal AI, designing experiments to test new algorithms.

Requirements

• Ph.D. or Master’s in CS, EE, or related field with a strong AI/ML foundation.
• Proficiency in Python/C++ and deep learning frameworks like PyTorch or TensorFlow.
• Experience with large-scale training pipelines.
• Hands-on experience with CNNs, Transformers, and Vision Transformers (ViT).
• Strong understanding of vision-language models and their fine-tuning/adaptation.
• Proven skills in model training and optimization, including fine-tuning on large datasets and applying distillation/quantization.
• Experience with foundation or multimodal models is a plus.
• Strong problem-solving ability, including quick prototyping and diagnosing failure cases.
• Startup experience is preferred, with comfort in ambiguity and end-to-end project ownership.

Benefits

• Stock options
• Comprehensive health + welfare package (Medical, Dental, Vision, Life, EAP, Legal Services, 401k plan)
• Flexible time off, including Winter Break
• Cutting-edge equipment and branded gear
• Opportunities to connect with co-workers