Staff Data Infrastructure Engineer
$247k - $339k • Remote • New York City, NY; San Francisco, CA
Posted 16d ago
About the job
Faire is a technology wholesale platform connecting independent retailers globally. We are seeking a Staff Engineer to own and evolve our data plumbing, ensuring data is accessible, accurate, and queryable for various teams including Product Engineering, Data Science, and Machine Learning. This hands-on role involves designing and building the next generation of our data ingestion and processing systems, and serving as the go-to expert for data movement at Faire.
Responsibilities
- Set the technical direction and roadmap for data movement from production systems to analytical stores.
- Build and maintain the CDC and streaming ingestion layer, handling complexities like ordering, deduplication, and schema changes.
- Implement data contracts and quality checks across the data platform.
- Establish and enforce SLAs for critical datasets, integrating quality monitoring tools.
- Manage and optimize Airflow and Fivetran, making build vs. buy decisions.
- Ensure platform reliability through SLOs, on-call rotations, and incident management.
- Collaborate with stakeholders to migrate existing pipelines to the new platform.
- Mentor other engineers to improve the team's data engineering practices.
Requirements
- Proven experience building and operating large-scale data infrastructure with a focus on setting technical direction.
- Deep expertise in change data capture (CDC) and streaming ingestion from operational databases via Kafka.
- Hands-on experience with lakehouse architectures, particularly Iceberg on S3, including partitioning, compaction, and catalog management.
- Strong Spark skills and experience with Databricks and Snowflake.
- Practical experience with data quality and observability tools (e.g., Anomalo, Monte Carlo), data contracts, and SLAs.
- Experience operating Airflow at scale and working with managed ingestion tools like Fivetran.
- Proficiency in SQL and data modeling for analytical use cases.
- Solid Python skills, plus experience with Kotlin, Java, Scala, or Go.
- Experience shipping infrastructure on AWS using Terraform.
- Understanding of data governance principles (access control, PII, lineage, audit).
- Track record of leading cross-team data initiatives and mentoring senior engineers.
- Ability to communicate technical trade-offs effectively to diverse audiences.
- Willingness to take ownership of unaddressed issues and participate in on-call rotations.
- Experience in marketplace, e-commerce, or transaction-heavy businesses is a plus.
Benefits
- Equity
- Health insurance
- Paid time off