Applied Researcher 4 (AI Foundations: Multimodal Guardrails and VLM's)
$219k - $250k • Cambridge, MA • Full-time
Posted 23d ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Research Engineer
About the job
At Capital One, we are building trustworthy and reliable AI systems to transform banking. We are seeking an Applied Researcher to join our AI Foundations team, which is central to bringing our AI vision to life. This role involves partnering with a cross-functional team to deliver AI-powered products, leveraging a broad stack of technologies to analyze large datasets and build AI foundation models through all phases of development. You will engage in high-impact applied research, translating complex work into tangible business goals and collaborating with senior researchers on novel methods and findings.
Responsibilities
- Partner with data scientists, software engineers, ML engineers, and product managers to deliver AI-powered products.
- Leverage technologies like PyTorch, AWS Ultraclusters, Huggingface, and Lightning to analyze large volumes of numeric and textual data.
- Build AI foundation models from design through training, evaluation, validation, and implementation.
- Conduct high-impact applied research to advance customer experiences with AI.
- Translate complex technical work into tangible business goals.
- Collaborate with senior researchers to prepare internal tech reports or conference submissions.
Requirements
- PhD in Electrical Engineering, Computer Engineering, Computer Science, AI, Mathematics, or related fields, or an M.S. with 2 years of applied research experience.
- Experience building large deep learning models (language, images, events, or graphs).
- Expertise in training optimization, self-supervised learning, robustness, explainability, or RLHF.
- Engineering mindset with a track record of delivering models at scale for training data and inference volumes.
- Experience delivering libraries, platform-level code, or solution-level code to existing products.
- Track record of generating high-quality ideas or improvements in machine learning, demonstrated by publications or projects.
- Ability to own and pursue a research agenda, including identifying impactful problems and autonomously carrying out long-running projects.
- Deep understanding of AI methodologies.
- Comfort with open-source languages and developing AI foundation models and solutions using open-source tools and cloud platforms.
- PhD focus on NLP or Masters with 5 years of industrial NLP research experience (preferred).
- Multiple publications on pre-training large language models (preferred).
- Experience training a large language model from scratch (10B+ parameters, 500B+ tokens) (preferred).
- Publications in deep learning theory or at top AI conferences (ACL, NAECM, EMNLP, Neurips, ICML, ICLR) (preferred).
- Experience with optimization techniques for training and inference, model sparsification, quantization, training parallelism, gradient checkpointing, or model compression (preferred).
- Experience optimizing training for a 10B+ model (preferred).
- Deep knowledge of deep learning algorithmic and/or optimizer design (preferred).
- Experience with compiler design (preferred).
- Experience with fine-tuning LLMs (Supervised Finetuning, Instruction-Tuning, Dialogue-Finetuning, Parameter Tuning) (preferred).
- Demonstrated knowledge of transfer learning, model adaptation, and model guidance principles (preferred).
- Experience deploying a fine-tuned large language model (preferred).