Staff Software Engineer, Data
$176k - $230k • Remote • Remote
Posted 23h ago
About the job
The Data team builds and runs the systems that make Stability AI's data usable for business operations and model training. This is a collaborative, outcome-focused role where you'll have significant technical access and autonomy. You will own the company's data platform end-to-end, including the AWS data lake, transformation layer, integrations between business systems, and the Kubernetes infrastructure. You will connect and maintain data from various systems like billing, usage telemetry, identity, CRM, and ERP. Additionally, you will serve as the data engineer for research teams, building pipelines to transform raw media into training-ready datasets for audio, video, image, and 3D modalities.
Responsibilities
- Own the company data platform end-to-end, including the S3/Athena/Glue lake, dbt project, ingestion from various systems, and the Postgres serving layer.
- Build and operate integrations between billing, CRM, ERP, identity, and telemetry systems for automatic reconciliation.
- Design and maintain the warehouse layer with dimensional models, data contracts, quality tests, lineage, and a catalog.
- Enforce PII tiering through access grants in collaboration with Security and Legal.
- Process data at scale, including billions of telemetry rows, terabyte log streams, and Iceberg tables, while managing costs.
- Own orchestration and infrastructure on EKS, Helm, Terraform, CI/CD, and monitoring, including on-call rotation.
- Deliver operational reporting for Finance and leadership on revenue, credits, and cloud costs.
- Support research teams by ingesting, versioning, and preparing large training datasets for various media modalities.
- Build and scale pipelines for generation, filtering, deduplication, and captioning of media for training datasets.
- Maintain lineage, quality metrics, and dataset documentation for reproducibility.
- Lead design reviews, mentor teammates, and write runbooks to eliminate single-person dependencies.
- Partner with Finance, Marketing, SRE, Security, Product, and Research on company-wide data initiatives.
Requirements
- 6+ years building and running production data platforms, including architecture ownership.
- Experience integrating SaaS business systems (e.g., Stripe, HubSpot, ERP, identity provider) into a warehouse.
- Production experience preparing large multimodal training datasets and running batch workloads on HPC or cloud schedulers (SLURM, Kubernetes, or similar).
- Comfort working independently in a small team with significant ownership.
- Expert SQL and Python, with strong software engineering habits (tests, code review, CI).
- Warehouse and lake experience with real modelling (Iceberg or Delta on S3, Athena/Trino, Snowflake or BigQuery, and dbt).
- Experience with an orchestrator (dbt, Airflow, or Argo) and distributed processing (Spark or equivalent) in production.
- Proficiency in AWS, Kubernetes (EKS), Helm, and Terraform, with the ability to debug related issues.
Benefits
- Health coverage for you and your family (PPO and HMO plans, with some plans covering 100% of premiums for employee and dependents).
- HSA and FSA accounts for pre-tax healthcare and eligible expenses.
- Company-paid life insurance, with optional voluntary coverage.
- Short- and long-term disability coverage.
- 401(k) with a company match of up to 4%.
- Paid parental leave.