Member of Technical Staff - VLM
€130k - €340k • Freiburg (Germany) • FullTime
Posted 4mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Black Forest Labs is seeking a Staff / Senior Individual Contributor to pioneer the integration of vision-language models (VLMs) directly into their advanced generative AI systems, like FLUX. This role focuses on developing novel VLM approaches and architectures, rather than just applying existing ones. You will explore how vision and language representations inform each other, how multimodal understanding enhances generation quality, and how to deploy these capabilities at scale. The ideal candidate will have a proven track record of pretraining or significantly advancing a VLM that has been deployed or publicly released.
Responsibilities
- Lead development and training of state-of-the-art multimodal vision-language models within the FLUX stack, innovating on architectures.
- Design fine-tuning strategies to adapt VLMs for specialized creative use cases like captioning, editing instructions, and prompt enhancement.
- Research integrations between VLM/LLM capabilities and diffusion/flow pipelines to improve generation quality and controllability.
- Evaluate emerging multimodal architectures and translate research into practical improvements.
Requirements
- Pretrained or significantly advanced a VLM (not just SFT'd or LoRA'd) that was deployed in a production system or released publicly.
- Strong publication record or production track record demonstrating advancements in multimodal architectures.
- Deep understanding of vision and language representation interaction, including tokenization, alignment, grounding, cross-modal attention, and their failure modes.
- Experience with distributed training at multi-node scale.
- Comfortable working at the research/production boundary, ensuring work ships and generalizes.
- Experience with diffusion or flow-based generative models is a strong plus, especially regarding paradigm composition.
Benefits
- Equity
- EU €130,000-€340,000 base annual salary
- Travel costs covered for in-person collaboration