Data Engineer
$180k - $220k • San Francisco (Hybrid)
Posted 29d ago
About the job
We are seeking a hands-on Data Engineer to contribute to the development and scaling of our modern data platform. This role involves close collaboration with Finance, Engineering, Product, and Analytics teams to build robust, high-performance data pipelines and systems. You will work with both batch and real-time data processing technologies, ensuring data quality, accessibility, and usability across the organization. A key aspect of this role will be enabling data activation, ensuring high-quality data flows into business tools and powering next-generation AI-driven applications, including agent-based systems, by establishing strong data foundations and pipelines. This is an excellent opportunity for individuals who thrive on solving end-to-end data challenges, from ingestion to insights.
Responsibilities
- Build and maintain scalable data pipelines (batch and streaming) using Databricks, Spark, Kafka, and AWS services.
- Construct and manage pipelines from source systems (Salesforce, billing, product events, API logs) into clean analytics layers.
- Design, develop, and optimize ETL/ELT workflows using DBT, PySpark, SQL, and tools like Fivetran.
- Collaborate with finance teams to develop financial data solutions, metrics, and forecasting models.
- Partner with finance on revenue accounting, COGS, and margin reporting.
- Work closely with marketing and growth teams to support data use cases like segmentation, campaign targeting, and lifecycle analytics.
- Develop and maintain reverse ETL pipelines to synchronize data from the warehouse to systems such as Salesforce, HubSpot, and Braze.
- Create and manage curated datasets for analytics, reporting, and go-to-market initiatives.
- Build and maintain dashboards and reporting layers for marketing and business performance tracking.
- Support AI/ML and agent-based applications by preparing and serving high-quality datasets for MCP integrations and AI-driven applications.
- Monitor pipeline performance, troubleshoot issues, and ensure high data reliability and quality.
- Implement data quality checks, validations, and alerting mechanisms.
- Collaborate with cross-functional teams to define data contracts and ensure system consistency.
Requirements
- 6+ years of experience in data engineering or a related field.
- Strong hands-on experience with Databricks, AWS (S3, Glue, Athena, EMR, etc.), and Kafka.
- Proficiency in Python (PySpark) and SQL for large-scale data processing.
- Experience building and maintaining ETL/ELT pipelines (DBT/Airflow or similar).
- Experience with data ingestion tools such as Fivetran (or similar).
- Familiarity with reverse ETL/data activation workflows and syncing data to tools like Salesforce, HubSpot, Braze.
- Exposure to or experience with AI/ML data pipelines, including RAG architectures, vector databases, or embeddings workflows.
- Familiarity with agent-based systems, MCP integrations, or LLM-powered applications is a strong plus.
- Experience working with finance and building finance-specific metrics and pipelines is a strong plus.
- Understanding of data modeling and working with large-scale datasets (batch and streaming).
- Experience creating dashboards and supporting reporting workflows (BI tools).
- Strong problem-solving skills and ability to debug production data issues.
- Strong communication skills and ability to work collaboratively across teams.