Member of Engineering (Evaluations / Engineering)
Remote • Remote (EMEA/East Coast) • FullTime
Posted 5mo ago
About the job
Poolside is building a company to create Artificial General Intelligence, aiming to be a leader in applied research, engineering, infrastructure, and large-scale deployment. We are a remote-first team across Europe and North America, coming together monthly for in-person collaboration. Our R&D and production teams value strong software development knowledge and believe good engineering drives faster iteration. This role focuses on evaluation, a critical pillar for building frontier models and products. You will design and implement a powerful, self-serve evaluation framework to enable easy building, running, and insight extraction from evaluations at scale, supporting our research and development efforts.
Responsibilities
- Design a Python framework for centralized implementation of internal and public benchmarks.
- Build and maintain a pipeline for running distributed evaluations at scale.
- Collaborate with modeling and product teams to enhance experimentation and evaluation tooling.
Requirements
- Strong engineering background.
- Experience leading cross-functional software projects.
- Experience building highly reliable and well-tested services.
- Experience with distributed systems.
- Experience with data pipelines and distributed processing.
- Experience with message queues and event-driven architectures (e.g., Kafka, Google Pub/Sub).
- Experience with cloud managed services and storage (GCP, AWS, Azure).
- Experience with monitoring and alerting tools (e.g., Grafana, Prometheus, Datadog).
- A product mindset for building developer-facing software.
- Ability to collaborate with researchers and ML engineers to identify opportunities.
- Experience designing frameworks or tooling for developers is a plus.
- Experience with MLOps and data visualization platforms is a plus.
Benefits
- Fully remote work & flexible hours
- 37 days/year of vacation & holidays
- Health insurance allowance for you and dependents
- Company-provided equipment
- Wellbeing, always-be-learning and home office allowances
- Frequent team get togethers
- Great diverse & inclusive people-first culture