Member of Technical Staff (Search Quality Analyst)
Remote • Belgrade • FullTime
Posted 12d ago
About the job
Perplexity is seeking an experienced analyst to contribute to the development and enhancement of its core search technologies. This position operates at the nexus of data analysis and engineering, involving the creation of metrics, the construction of data pipelines, and the refinement of search and answer systems.
Responsibilities
- Identify and troubleshoot quality issues within the search pipeline.
- Develop novel metrics for tracking and measuring search quality.
- Construct datasets for model training, including LLM-as-a-judge labeling pipelines.
- Enhance search snippet quality and page selection algorithms for indexing.
- Design and analyze A/B experiments to validate proposed improvements.
Requirements
- 4+ years of experience as a data analyst or in a related field.
- Proficiency in SQL and Python.
- Demonstrated experience in designing metrics from scratch.
- Experience building labeling pipelines using LLM-as-a-judge.
- Experience training ML models that have been deployed to production with measurable metric improvements.
- Experience designing evaluations with known ground truth or driving improvements on such evaluations.