Data Engineer
Salary not listed • Bengaluru • Full-time
Posted 1mo ago
About the job
Prodigal is seeking a passionate and driven Data Engineer to join their team in Bengaluru. This role is instrumental in building scalable data pipelines, generating powerful insights, and supporting AI/ML initiatives. The ideal candidate will enjoy working across data engineering and analytics to help shape the future of Agentic AI. Prodigal is a connected AI platform used by financial institutions to run their operations, analyzing billions of interactions to power AI agents that guide human agents, conduct customer conversations, and orchestrate outreach. They are backed by prominent investors and work with over 100 financial institutions.
Responsibilities
- Design, build, and manage robust data pipelines for collecting, transforming, and modeling data within a Databricks data lake, primarily using dbt.
- Own the dbt layer end-to-end, including models, sources, tests, macros, incremental strategies, and documentation.
- Transform raw data into reliable, well-modeled assets for downstream teams, ensuring cleanliness, testability, and re-runnability.
- Collaborate with cross-functional teams to deliver actionable insights for product, AI, and business decisions.
- Support AI and ML initiatives by ensuring clean, validated, and well-structured data availability.
- Identify and resolve performance bottlenecks in dbt models, Databricks queries, and reporting pipelines.
Requirements
- 2-3 years of hands-on experience in production data engineering.
- Strong SQL skills, including complex joins, aggregations, window functions, and query tuning.
- Python for data engineering with production-grade PySpark or Spark SQL experience (Databricks preferred).
- Experience with dimensional modeling (star schemas, fact vs. dimension grain, SCD Type 2, incremental loads).
- Proficiency with dbt as a transformation framework.
- Hands-on experience with AWS services (Lambda, S3, CloudFront, SQS).
- Familiarity with AI-native workflows and tools (e.g., Claude Code, Cursor).
- Sharp problem-solving skills with the ability to balance quality and delivery.
- Self-driven, curious, and able to thrive in fast-paced environments.
- Foundational knowledge in ML/AI fundamentals (bonus).
Benefits
- Health insurance for you and your family
- Meals at office
- Travel reimbursement
- Unlimited leaves
- Subsidized gym membership
- Unlimited learning & development
- Flexible work schedule