Software quality engineer (US)
Remote • San Francisco, CA • FullTime
Posted 2mo ago
Job Location
San Francisco, CA
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
WRITER is seeking a Software Quality Engineer to ensure the reliability, performance, and trustworthiness of their AI-powered work orchestration platform. This role involves defining and implementing quality strategies for enterprise-grade LLMs and AI agents, directly impacting how global companies leverage AI. It's an opportunity to address the unique challenges of AI quality assurance in a fast-paced, evolving field.
Responsibilities
- Define and implement comprehensive quality assurance strategies and test plans for AI agents and LLM-powered applications.
- Design and develop automation frameworks using languages like Typescript or Python.
- Collaborate with product managers, ML engineers, and data scientists to translate complex AI features into test cases.
- Drive continuous improvement of testing processes and infrastructure, integrating automated checks into CI/CD pipelines.
- Identify, document, and track software defects, performing root cause analysis.
- Monitor production systems and AI model performance to proactively identify issues.
- Champion quality best practices across engineering teams.
- Design, manage, and maintain test data strategies and mock services.
- Analyze customer feedback and debugging logs to manage the full defect lifecycle.
- Experience designing, developing, or integrating agentic AI systems, AI skills, and the Model Context Protocol (MCP).
Requirements
- 5+ years of experience in software quality assurance or engineering, with a focus on testing complex distributed systems or AI/ML applications.
- 5+ years of experience working with GitHub Actions.
- Proficiency in programming languages like Python or Typescript for test automation.
- Experience with modern testing frameworks such as Playwright.
- Solid understanding of AI/ML concepts, including model evaluation metrics, data pipelines, and generative AI testing challenges.
- Experience with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes) in a CI/CD environment is a plus.
- Exceptional analytical skills and attention to detail.
- Ability to connect cross-functional teams to common quality goals.
- Demonstrated ability to drive initiatives independently in a fast-paced startup environment.
Benefits
- Generous PTO, plus company holidays
- Medical, dental, and vision coverage
- 16 weeks paid parental leave
- Fertility and family planning support
- Early-detection cancer testing
- Flexible spending account and dependent FSA options
- Health savings account with company contribution
- Annual work-life stipends (Wellness, Learning and development)
- Company-wide and team off-sites
- Competitive compensation
- Company stock options
- 401k