Subject Matter Expert
Remote • Canada; United States
Posted 1mo ago
Job Location
Canada; United States
Remote Work Policy
Fully remote
Categories
Applied AI Engineer
About the job
The Subject Matter Expert is a specialized, part-time role focused exclusively on quality evaluation and standards definition within assigned projects. This role is designed for subject-matter experts who provide high-value insights through audits, quality frameworks, and performance analysis. The Subject Matter Expert does not engage in contributor management, real-time support, or operational task coordination. Instead, the role centers on assessing output quality, identifying systemic issues, and recommending improvements to ensure high standards are consistently met.
Responsibilities
- Conduct structured audits of task outputs to evaluate accuracy, consistency, and adherence to project guidelines.
- Apply domain expertise to assess nuanced or complex work requiring specialized knowledge.
- Identify patterns of quality issues and flag systemic gaps impacting output standards.
- Define and refine quality benchmarks, scoring rubrics, and evaluation criteria based on project needs.
- Contribute to the development of gold standards and reference materials for high-quality outputs.
- Ensure quality frameworks reflect domain-specific rigor where applicable.
- Analyze audit results to generate actionable insights on quality trends and performance gaps.
- Provide structured reports with clear recommendations to improve overall output quality.
- Partner with internal stakeholders to calibrate on quality expectations and scoring consistency.
- Recommend improvements to workflows, guidelines, and evaluation processes based on audit findings.
- Support calibration efforts to ensure alignment across reviewers and quality stakeholders.
- Contribute to continuous improvement initiatives strictly related to quality outcomes.
Requirements
- Deep domain expertise in a specialized field (e.g., medicine, law, engineering, etc.).
- Strong analytical skills with the ability to evaluate complex outputs against defined standards.
- Experience in quality assurance, auditing, or evaluation frameworks is preferred.
- Ability to produce clear, structured, and actionable feedback.
- High attention to detail and commitment to accuracy and consistency.