Frontier Agents Engineer

About the job

Responsibilities

• Partner with enterprise customers to understand their technical infrastructure, data pipelines, and business requirements.
• Design and implement custom integrations between Scale AI's platform and customer data environments.
• Build data connectors and ETL pipelines to ingest, process, and prepare customer data for AI workflows.
• Deploy and configure AI models and agents within customer security and compliance boundaries.
• Develop production-grade AI agents tailored to customer use cases.
• Architect multi-agent systems that orchestrate between different models, tools, and data sources.
• Implement evaluation frameworks to measure agent performance and iterate toward business objectives.
• Design human-in-the-loop workflows and feedback mechanisms for continuous agent improvement.
• Create sophisticated prompt engineering strategies optimized for customer-specific domains and data.
• Build and maintain prompt libraries, templates, and best practices.
• Conduct systematic prompt experimentation and A/B testing to improve model outputs.
• Implement RAG systems and fine-tuning pipelines.
• Serve as the primary technical point of contact for strategic enterprise accounts.
• Collaborate with customer data scientists, ML engineers, and software developers.
• Provide technical training and knowledge transfer to customer teams.
• Work closely with Scale's product and engineering teams to translate customer needs into product improvements.
• Document technical architectures, integration patterns, and best practices.
• Debug complex technical issues across the entire stack.
• Rapidly prototype solutions to unblock customers and prove out new use cases.
• Stay current on the latest AI/ML research and tools.
• Identify opportunities for productization based on common customer patterns.

Requirements

• 4+ years of software engineering experience with strong fundamentals in data structures, algorithms, and system design.
• Production Python expertise with experience in modern ML/AI frameworks (e.g., LangChain, LlamaIndex, HuggingFace, OpenAI API).
• Experience with cloud platforms (AWS, GCP, or Azure) and modern data infrastructure.
• Strong problem-solving skills with the ability to navigate ambiguous requirements and rapidly iterate toward solutions.
• Excellent communication skills with the ability to explain complex technical concepts to both technical and non-technical audiences.
• Deep understanding of LLMs including prompting techniques, embeddings, and RAG architectures.
• Experience building and deploying AI agents or autonomous systems in production.
• Knowledge of vector databases and semantic search systems.
• Experience with containerization (Docker, Kubernetes) and CI/CD pipelines.
• Experience using Terraform, Bicep, or other Infrastructure as Code (IaC) tools.
• Familiarity with enterprise security, compliance, and governance requirements (SOC 2, GDPR, HIPAA).
• Proven ability to work with customers in a technical consulting, solutions engineering, or product engineering role.
• Domain expertise in verticals like finance, healthcare, government, or manufacturing.
• Experience with technical enablement or teaching programs.