AI Native Data Engineer
Salary not listed • PALO IT Mexico • Full-time
Posted 2d ago
About the job
PALO IT is seeking a Data Engineer in CDMX to design and implement scalable data solutions for a leading insurance company. This role focuses on building resilient data pipelines and creating strategic insights for underwriting, risk management, fraud prevention, and customer experience. As a Data Engineer, you will lead and accelerate software innovation by designing, building, and maintaining scalable and secure data pipelines in cloud environments, integrating large volumes of data, and automating data processes. You will also build and maintain modern data lakes and warehouses, implement data quality and governance, and design optimized datasets to enable machine learning models and analytics.
Responsibilities
- Design, build, and maintain scalable and secure data pipelines in cloud environments (AWS, GCP, or Azure).
- Integrate large volumes of structured and unstructured data from core insurance systems.
- Automate data ingestion, transformation, and quality processes using tools like Apache Spark, dbt, Kafka, and Airflow.
- Build and maintain modern data lakes and warehouses (Snowflake, BigQuery, Redshift).
- Implement data quality, lineage, and governance validations.
- Ensure compliance with data regulations and internal security policies.
- Design optimized datasets to enable machine learning models, predictive analytics, and dashboards.
- Collaborate with data scientists, architects, and business stakeholders to democratize access to trusted data.
- Document data architectures, pipelines, transformation standards, and lineage.
Requirements
- Experience with modern data processing frameworks: Spark, Databricks, dbt, Kafka, Apache Beam.
- Strong command of Python and advanced SQL.
- Hands-on experience with Azure cloud platforms (Data Factory, Synapse).
- Knowledge of modern Data Lake / Data Warehouse design (Snowflake, Redshift, BigQuery).
- Familiarity with DataOps, testing, and CI/CD practices in data pipelines.
- Experience with workflow orchestration systems such as Airflow or Dagster.
- Understanding of data quality and governance frameworks: Great Expectations, Deequ, DataHub, OpenLineage.
- Knowledge of streaming technologies: Kafka, Pub/Sub, Kinesis.
- Upper-intermediate English level (B2 or higher).
Benefits
- Competitive compensation and benefits reflecting skills, impact, and leadership.
- Continuous personal and professional growth through PALO IT Academy, specialized training, R&D projects, and knowledge-sharing culture.
- Global opportunities for international mobility and collaboration.
- Empowerment to innovate, shape, and implement next-level AI methodologies.