Research Engineer, Content Understanding
$180k - $350k • Remote • San Francisco, California • FullTime
Posted 16d ago
Job Location
San Francisco, California
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Research Engineer
About the job
Exa is an applied AI lab building a novel search engine. We focus on massive-scale infrastructure for web crawling, training state-of-the-art embedding models, and designing high-performance vector databases. Our goal is to create perfect search over all the world's information, surpassing current capabilities. This role is critical to our search architecture, with flexibility in project assignments based on individual skills and interests. We are seeking a research engineer to tackle the complex challenges of understanding web page content, including parsing, classification, credibility assessment, and misinformation detection, all at a web scale.
Responsibilities
- Parse web pages to distinguish content from furniture.
- Classify page types and topics.
- Determine page usability and publication dates.
- Assess page quality and trustworthiness.
- Identify and address credibility and misinformation issues.
- Handle AI-generated and machine-spun content.
- Differentiate between pages written for readers versus crawlers.
- Develop models to judge page quality and establish consensus on its definition.
- Model credibility and misinformation by assessing page claims and source reliability.
- Determine semantic similarity between documents for deduplication.
- Build accurate and cost-effective classification and extraction systems for web scale.
- Design supervision strategies for unlabeled data and assess learnability.
- Trace and fix bad search results at the source by analyzing page-level predictions.
Requirements
- Graduate-level ML experience (Master’s or PhD with at least 2 years of relevant experience), or an exceptionally strong undergrad.
- Ability to build a transformer from scratch in PyTorch.
- Experience training models that are optimized for efficient deployment.
- Proficiency in building large-scale datasets and working extensively with data.
- Comfort with problems where ground truth is undefined and defining it is part of the task.
- Understanding of the importance of finding high-quality knowledge.