Databricks Genie AI Engineer
India • Full-time
Posted 5d ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Engineer
About the job
Accountabilities:
- Design, develop, and maintain the semantic layer supporting Databricks Genie, including table and column descriptions, synonyms, certified data assets, sample questions, and business metric definitions.
- Build, configure, and continuously optimize Databricks Genie Spaces to improve the accuracy, relevance, and reliability of natural language query responses.
- Define and maintain metric views, business logic, and semantic models to ensure that natural language questions are accurately translated into governed business KPIs.
- Develop evaluation frameworks, automated validation processes, and regression tests to measure query accuracy, SQL quality, and the overall performance of AI-generated insights.
- Analyze and troubleshoot AI response failures, including ambiguous questions, incorrect joins, inaccurate metric interpretations, and SQL generation issues.
- Improve AI-generated results by refining data schemas, metadata, semantic definitions, instructions, and configurations.
- Collaborate with data engineering teams to enhance upstream data quality, data structures, dimensional models, and overall analytical reliability.
- Support the deployment, versioning, and monitoring of semantic assets in collaboration with DevOps and platform engineering teams.
- Translate business user feedback into actionable improvements to semantic models, data structures, prompts, and Genie configurations.
- Work closely with business stakeholders to understand analytical requirements and ensure that AI-powered data experiences deliver meaningful and trustworthy insights.
- Establish best practices for semantic modelling, data governance, testing, and continuous improvement across AI-enabled analytics solutions.
- Monitor the performance and reliability of natural language analytics capabilities and proactively identify opportunities for optimization.
Requirements:
- Hands-on experience with Databricks, particularly Databricks SQL, Unity Catalog, and Databricks Genie.
- Strong SQL proficiency, with the ability to develop, optimize, and troubleshoot complex queries.
- Practical programming experience in Python and/or Scala for data processing, automation, testing, and analytics-related development.
- Proven experience with semantic modelling, metric layers, or business-friendly data abstraction frameworks.
- Strong understanding of data warehousing concepts, dimensional modelling, data structures, and analytical data architectures.
- Experience designing and implementing testing, evaluation, and validation frameworks for analytics platforms or AI-driven systems.
- Familiarity with Large Language Models (LLMs), prompt engineering, and Retrieval-Augmented Generation (RAG) concepts.
- Experience working with at least one major cloud platform, such as AWS, Microsoft Azure, or Google Cloud Platform (GCP).
- Understanding of data governance, metadata management, and techniques for ensuring data accuracy, consistency, and reliability.
- Ability to investigate complex analytical issues, identify root causes, and implement effective technical solutions.
- Strong analytical and problem-solving skills, with a keen attention to detail and a quality-focused mindset.
- Excellent communication and stakeholder management abilities, including the capacity to translate business requirements into technical specifications.
- Ability to collaborate effectively with data engineers, platform teams, DevOps professionals, and business stakeholders.
- Comfortable working in a fast-paced, cross-functional environment with evolving technical and business requirements.
Preferred qualifications:
- Experience with semantic modelling and analytics tools such as dbt, LookML, or Databricks Metric Views.
- Familiarity with business intelligence and visualization platforms such as Power BI, Tableau, or Looker.
- Experience integrating BI tools with Databricks SQL to support enterprise reporting and analytics.
- Previous experience implementing AI-powered analytics, natural language querying, or conversational business intelligence solutions.
- Exposure to cloud-based data engineering architectures and modern data platform development practices.
Benefits:
- Competitive compensation package aligned with experience and technical expertise.
- Government-mandated employee benefits.
- Supplemental health insurance coverage.
- Fully remote working arrangement.
- Collaborative and professional working environment.
- Opportunities to work with modern cloud technologies, AI-powered analytics, and advanced data engineering platforms.
- Career development and advancement opportunities within a growing organization.
- Opportunities to collaborate with international teams and contribute to innovative data and AI initiatives.
- Continuous learning and professional growth through exposure to emerging technologies and industry practices.
