Middle AI Analytics Engineer
Germany • Full-time
Posted 5d ago
About the job
Accountabilities:
- Self-Service Analytics Enhancement: Improve the quality and accessibility of self-service analytics through Metabase by developing and integrating AI skills, contextual information, and intelligent workflows that help users retrieve accurate and relevant insights.
- Analytics Workflow Automation: Identify and automate repetitive tasks within the Data & Analytics team, developing AI-powered tools such as automated code reviewers, reusable AI skills, and other productivity-enhancing solutions.
- AI Quality Evaluation: Design and implement a comprehensive framework for evaluating the quality, accuracy, and reliability of AI-driven solutions used by the analytics team.
- Claude and Context Engineering: Design and optimize Claude skills, including their purpose, instructions, examples, and contextual information, to improve response quality and minimize hallucinations.
- RAG Development and Optimization: Design, implement, and improve Retrieval-Augmented Generation (RAG) systems, including source indexing, document chunking, information retrieval, and reranking strategies.
- AI Performance Monitoring: Develop evaluation methodologies using golden answers, automated evaluations (evals), LLM-as-a-judge techniques, regression testing, and performance monitoring to identify quality issues and detect model drift.
- Root Cause Analysis: Investigate AI-related performance issues, identify underlying causes, and implement improvements to ensure reliable and consistent analytical outputs.
- Cross-Functional Collaboration: Work closely with analytics and technical teams to identify opportunities for AI adoption, improve existing processes, and deliver scalable solutions that support business objectives.
- Continuous Innovation: Explore new AI capabilities, tools, and development approaches to improve analytical efficiency and encourage the adoption of intelligent automation across the organization.
Requirements:
- Practical experience in AI engineering, analytics engineering, data engineering, or a related technical role.
- Strong SQL skills, with the ability to query, manipulate, and analyze structured data effectively.
- Proficiency in Python for developing automation solutions, analytical tools, and AI-powered applications.
- Hands-on experience with Claude, Claude Code, and Model Context Protocol (MCP).
- Strong understanding of Claude skills and context engineering, including designing reusable skills, defining clear instructions, providing relevant examples, and optimizing prompt context to reduce hallucinations.
- Experience designing and improving Retrieval-Augmented Generation (RAG) systems, including document indexing, chunking strategies, retrieval optimization, and reranking.
- Knowledge of AI evaluation methodologies, including golden datasets or answers, evaluation frameworks, LLM-as-a-judge, regression testing, model drift monitoring, and root cause analysis.
- Ability to assess the quality, reliability, and effectiveness of AI-generated outputs and continuously improve AI-powered solutions.
- Strong analytical and problem-solving abilities, with a structured approach to investigating technical challenges.
- Good communication and collaboration skills, with the ability to exchange ideas, share feedback, and work effectively within a multidisciplinary team.
- A proactive and adaptable mindset, with a willingness to experiment, take ownership, and continuously develop technical expertise.
- Nice to have: Experience developing AI agents through LLM APIs or SDKs, including tool integration, context management, and cost optimization.
- Familiarity with Business Intelligence platforms and their role as data access layers for AI systems, including data models, metrics, and query structures. Experience with Metabase or Tableau is an advantage.
- Previous experience working with dbt for data transformation and analytics engineering.
Benefits:
- Fully Remote Work: Work remotely from a location where you feel comfortable and safe, with additional access to company hubs in Kyiv and Warsaw.
- Paid Time Off: 20 working days of paid annual leave, in addition to public holidays and paid sick leave.
- Health Insurance: Access to medical insurance covering services at leading healthcare facilities.
- Mental Health Support: Coverage for psychological support through the Pleso platform.
- Flexible Benefits Budget: Monthly Benefit Café allowance that can be used for personal interests, hobbies, sports, and other lifestyle activities.
- Professional Development: Individual learning budgets for courses, workshops, and other educational opportunities.
- Language Learning: Access to corporate English lessons to support international collaboration and career growth.
- Training and Mentoring: Performance reviews, structured career development plans, mentoring programmes, and professional training opportunities.
- Team Events: Opportunities to participate in workshops, team-building activities, corporate events, and social initiatives.
- Innovative Environment: Access to in-house technological capabilities and the opportunity to develop new solutions from the ground up.
- Modern Working Culture: A collaborative and transparent environment that encourages initiative, open communication, autonomy, and minimal bureaucracy.
- Automation-Focused Culture: Opportunities to reduce repetitive work through technology and dedicate more time to innovation and professional growth.
- Ethical Business Practices: Collaboration with international clients, with no clients using the company's products or services in the Ukrainian market or clients operating in Russia or other Russian-speaking markets.
