Lead Software Engineer- Financial Services Data Engineering: Pyspark / Java / BigData / Datalake / AI
Salary not listed • Jersey City, NJ, United States • Full-time
Posted 1mo ago
No salary is listed. Many US states require employers to post a pay range, so ask about pay early.
About the job
JPMorgan Chase is seeking a Lead Software Engineer for their Asset and Wealth Management- Analytics & Applied AI Team. This role is integral to an agile team focused on enhancing, building, and delivering secure, stable, and scalable technology products. You will be a core technical contributor, responsible for developing critical technology solutions across various business functions to meet firm objectives. The ideal candidate is passionate about solving business problems through innovation, analytics, and an AI-first engineering mindset, focusing on building reusable, governed analytical data products and accelerating regulatory and CEO-priority analytics. You will lead an AI-first transformation of data engineering and analytics by productizing the data product lifecycle and building human-in-the-loop autonomous agents to reduce manual effort, improve auditability, and enable self-service consumption on the strategic data mesh, while also owning the modernization of this mesh.
Responsibilities
- Collaborate with business and technology teams to develop AI-first analytics and data product solutions.
- Define and enforce architecture for an AI-driven data product lifecycle, including semantic extraction/alignment, automated lineage and DQ, pipeline code generation, mesh registration, and self-service consumption.
- Build and operate autonomous agents for data engineering tasks such as detecting schema drift, proposing transformations, reconciling semantics, triaging data incidents, and maintaining governance evidence under human-in-the-loop controls.
- Design analytics platforms for reporting and other analytics, exploring innovative real-time and batch analytics solutions.
- Establish monitoring and alerting for solution events related to performance, scalability, availability, and reliability.
- Provide technical leadership, guidance, and direction to team members, building prototypes for demonstrations.
- Drive team adoption of enterprise-authorized AI-assisted engineering practices to improve code quality, delivery speed, and operational outcomes.
- Apply knowledge of the Software Development Life Cycle toolchain, including AI-assisted development and automation capabilities, to enhance automation value.
- Lead migration and modernization from legacy analytics/reporting stacks to the strategic mesh ecosystem.
- Industrialize entity resolution and parent identification with ML/LLM solutions and standardize analytical product packaging for reuse and monetization.
- Embed governance, lineage, and DQ by design across critical domains and regulatory reporting.
Requirements
- Formal training or certification on software engineering concepts and 5+ years of applied experience.
- Proven leadership in delivering AI-first analytics and data engineering at scale, including productized data mesh patterns, semantic layers, and analytical data product lifecycle ownership.
- Experience developing data ingestion and integration processes, sourcing data from multiple platforms, and applying data cleansing/transformation rules.
- Deep hands-on experience with big data and modern data platforms (e.g., Spark, Databricks, Snowflake, Iceberg) and building robust pipelines and data lake/lakehouse frameworks.
- Strong programming capability in Python and PySpark, or Java, with strong CI/CD and containerization practices.
- Applied AI expertise in ML pipelines, NLP/LLMs, and agentic frameworks to build autonomous agents for engineering tasks under human-in-the-loop controls.
- Governance proficiency across lineage, data quality, and access control with evidence generation aligned to regulatory expectations.
- Comfortable working in an agile and collaborative environment with strong communication skills.
- Demonstrated experience leading effective use of approved AI-assisted software development tools and setting team expectations for validating AI outputs.
- Strong understanding of responsible AI use in engineering workflows, including data sensitivity, secure handling, and adherence to resiliency and security expectations.