Data Annotator - Applied AI Engineer
Remote • India • Part-time
Posted 8d ago
Job Location
India
Tech Stack
Remote Work Policy
Fully remote
Employment Type
Part-time
Categories
AI Engineer
About the job
We are seeking a Data Annotator with a focus on Applied AI Engineering to analyze and evaluate existing AI-powered applications. This role involves assessing system architectures, implementation logic, and technical correctness, with a particular emphasis on large language models, transformer-based architectures, and LLM integration workflows. You will also evaluate Retrieval-Augmented Generation (RAG) pipelines, AI agent architectures, and perform specialized technical data annotation for prompts, AI-generated responses, code, and documentation. The goal is to ensure factual accuracy, logical reasoning, and technical quality of AI outputs, providing clear explanations and actionable feedback for model improvement.
Responsibilities
- Analyze AI application architectures, implementation logic, and technical correctness.
- Evaluate large language models, transformer architectures, and LLM integration workflows.
- Assess Retrieval-Augmented Generation (RAG) pipelines, including embedding models and vector databases.
- Review AI agent architectures, tool-calling mechanisms, and multi-agent workflows.
- Perform technical data annotation for prompts, AI responses, code, and documentation.
- Evaluate AI-generated outputs for accuracy, reasoning, relevance, and quality.
- Review and assess prompt engineering techniques and AI-generated responses.
- Examine Python-based AI implementations and recommend improvements.
- Evaluate LLM APIs, model integrations, fine-tuning strategies, and inference pipelines.
- Assess AI applications developed using frameworks like LangChain and LlamaIndex.
- Produce technical explanations, annotations, justifications, and feedback.
- Maintain high standards of annotation accuracy, consistency, and quality.
Requirements
- 2-6 years of professional experience, with at least 2 years in Applied AI Engineering, Generative AI, LLM Engineering, or a related technical field.
- Strong hands-on programming experience with Python.
- Practical experience building, integrating, or maintaining applications powered by large language models.
- Strong understanding of LLMs, transformer-based architectures, and generative AI concepts.
- Experience working with LLM APIs and models (e.g., OpenAI, Anthropic, Google Gemini, open-source alternatives).
- Practical knowledge of Retrieval-Augmented Generation (RAG), embeddings, and vector databases.
- Familiarity with AI agent frameworks, tool calling, and multi-agent systems.
- Experience with AI application development frameworks like LangChain or LlamaIndex.
- Understanding of prompt engineering, context management, and LLM evaluation methodologies.
- Familiarity with model fine-tuning, inference pipelines, and AI application deployment.
- Ability to independently read, understand, analyze, and evaluate AI application codebases.
- Experience using Git/GitHub and collaborative software development workflows.
- Strong analytical thinking, technical reasoning, and problem-solving abilities.
- Excellent written communication skills for explaining technical findings.
- Ability to work independently, maintain availability, and deliver high-quality results on time.
Benefits
- Competitive hourly compensation: USD 30-80
- Fully remote working arrangement within India
- Flexible part-time engagement (minimum 2 hours/day)
- Flexible working hours
- Opportunity to work on advanced AI projects (LLMs, generative AI, RAG, agents)
- Hands-on exposure to modern AI technologies and frameworks
- Opportunity to apply and expand technical expertise
- Project-based work offering flexibility
