Data Engineer
$150k - $180k • Remote • United States • Full Time
Posted 1mo ago
About the job
The Data Engineer, Data Platform will be responsible for building and operating data capabilities that enable AHEAD teams to access trusted, usable, and well-managed information. This role involves developing ingestion pipelines, transformations, data models, and curated data products within a modern cloud data platform, with a focus on Snowflake and dbt. The position will support data from enterprise applications and services like Salesforce, Hatch, NetSuite, Signal, and approved APIs, making data available for analytics, applications, automation, and AI-enabled workflows through consistent engineering patterns, documented definitions, appropriate access controls, and dependable operational practices. A core expectation is the active use of AI throughout the software development lifecycle, including AI-assisted code generation, automated testing, documentation, troubleshooting, and review with human validation.
Responsibilities
- Build, maintain, and improve batch and low-latency data ingestion pipelines from enterprise systems, APIs, and other approved sources.
- Follow the AI SDLC by actively using approved AI coding tools and agents to generate, refactor, explain, and review code; validate generated output through engineering judgment, testing, and peer review.
- Use AI to generate and improve unit, integration, data-quality, and regression tests, then verify that automated tests accurately validate the intended behavior.
- Use AI-assisted workflows to create and maintain technical documentation, data-product documentation, runbooks, lineage notes, and change summaries as part of delivery.
- Build toward coordinated multi-agent delivery patterns that can divide and accelerate discovery, implementation, testing, documentation, and operational support while preserving human accountability.
- Develop SQL and Python solutions that collect, validate, transform, and publish data for downstream consumption.
- Use Snowflake and dbt to implement reliable transformations, reusable models, curated datasets, and data products across raw, common, and curated layers.
- Translate business and technical requirements into source mappings, data models, acceptance criteria, and maintainable engineering solutions.
- Partner with analytics, application, AI, Integration Platform, and business teams to make data available through governed and documented access patterns.
- Apply data quality checks for completeness, freshness, uniqueness, consistency, referential integrity, and other relevant quality dimensions.
- Add metadata, documentation, lineage, ownership, and usage guidance to data products so consumers can find and understand the data they use.
- Implement secure access patterns in partnership with Data Governance and Security teams, including role-based access, classification tags, masking, and row- or column-level controls when appropriate.
- Build automated tests and deployment processes that support consistent delivery through development, quality assurance, and production environments.
- Monitor pipeline health, data freshness, processing performance, and failures; troubleshoot issues and participate in incident resolution.
- Optimize Snowflake workloads, queries, transformations, and storage patterns for performance, reliability, and cost discipline.
- Support the curation and publication of cross-system data needed for shared business context, entity-aware access, reporting, automation, and AI use cases.
- Work with the Integration Platform and semantic-layer capabilities, including Horizon, to support consistent business meaning and reusable data access.
- Participate in backlog refinement, estimation, code review, technical documentation, and iterative delivery within an Agile engineering team.
- Identify opportunities to simplify delivery, reduce duplicate work, improve platform standards, and strengthen the reliability of data engineering practices.
Requirements
- Bachelor’s degree in computer science, information systems, engineering, mathematics, or a related field, or equivalent experience.
- 3 or more years of experience in data engineering, software engineering, analytics engineering, or a related technical role.
- Professional experience writing production-quality SQL and Python.
- Experience building or supporting data pipelines, transformations, and data models in a cloud data environment.
- Experience with Snowflake, dbt, or comparable cloud data warehouse and transformation technologies.
- Understanding of data modeling, ELT/ETL patterns, pipeline orchestration, APIs, and source-system integration.
- Experience with software engineering practices including source control, code review, automated testing, and CI/CD.
- Demonstrated active use of AI-assisted software development tools for code generation, test creation, documentation, debugging, or review.
- Ability to follow an AI SDLC and identify practical opportunities for multiple cooperating agents to improve delivery speed, consistency, and coverage.
- Understanding of data quality, metadata, lineage, access control, privacy, and secure handling of enterprise data.
- Ability to investigate data issues, communicate findings clearly, and work through ambiguity with teammates and stakeholders.
- Ability to collaborate effectively with engineers, analysts, product owners, governance partners, security teams, and business stakeholders.
- Experience with Azure services, serverless functions, cloud storage, or other cloud-native data engineering.