Research Scientist - Vision Language Model
$150k - $450k • Sunnyvale, CA • Full-time
Posted 4mo ago
Job Location
Sunnyvale, CA
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Research Engineer
About the job
As a Research Scientist on the Vision Language Model (VLM) team, you will be instrumental in advancing state-of-the-art multimodal foundation models. This role focuses on integrating visual understanding, reasoning, and agentic capabilities into large-scale VLM systems. You will engage in research and development covering model architectures, data strategies for pre-training and post-training, and evaluation benchmarks. The position blends advanced research with practical engineering, involving large-scale data processing, distributed training systems, and reinforcement learning for multimodal reasoning and agent development.
Responsibilities
- Research and develop next-generation Vision Language Models, covering pre-training, instruction tuning, reasoning, and agents.
- Develop novel architectures and training methodologies for integrating visual understanding, language reasoning, and tool-use capabilities.
- Research efficient multimodal learning techniques, including data-efficient training, long-context modeling, model modularity, and inference optimization.
- Build and improve large-scale multimodal datasets, synthetic data generation pipelines, and evaluation benchmarks for VLM capabilities.
- Investigate multimodal reasoning, agentic behavior, OCR, grounding, document understanding, chart understanding, and visual question answering capabilities.
- Contribute to technical reports, research publications, and open-source software.
- Represent MBZUAI at research conferences and industry events, showcasing advancements in multimodal foundation models and large-scale AI systems.
- Mentor junior researchers and collaborate across teams to drive impactful research initiatives.
Requirements
- PhD or equivalent research experience in Machine Learning, Computer Vision, Natural Language Processing, or Multimodal AI.