Staff Frontier Agents Engineer
Remote • San Francisco, CA; New York, NY
Posted 1mo ago
Job Location
San Francisco, CA; New York, NY
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Agent Engineer
About the job
Scale AI is seeking a Staff Frontier Agents Engineer to act as a technical bridge between Scale AI's AI capabilities and strategic customers. This role involves understanding customer challenges, architecting custom AI solutions, and ensuring successful deployment in production environments. It's a hands-on technical position combining deep engineering expertise with customer-facing problem-solving, requiring direct collaboration with customer engineering teams to integrate AI into critical workflows.
Responsibilities
- Partner with enterprise customers to understand their technical infrastructure, data pipelines, and business requirements.
- Design and implement custom integrations between Scale AI's platform and customer data environments.
- Build data connectors and ETL pipelines to ingest, process, and prepare customer data for AI workflows.
- Deploy and configure AI models and agents within customer security and compliance boundaries.
- Develop production-grade AI agents tailored to customer use cases.
- Architect multi-agent systems that orchestrate between different models, tools, and data sources.
- Implement evaluation frameworks to measure agent performance and iterate toward business objectives.
- Design human-in-the-loop workflows and feedback mechanisms for continuous agent improvement.
- Create sophisticated prompt engineering strategies optimized for customer-specific domains and data.
- Build and maintain prompt libraries, templates, and best practices.
- Conduct systematic prompt experimentation and A/B testing to improve model outputs.
- Implement RAG systems and fine-tuning pipelines.
- Serve as the primary technical point of contact for strategic enterprise accounts.
- Collaborate with customer data scientists, ML engineers, and software developers.
- Provide technical training and knowledge transfer to customer teams.
- Work closely with Scale's product and engineering teams to translate customer needs into product improvements.
- Document technical architectures, integration patterns, and best practices.
- Debug complex technical issues across the entire stack.
- Rapidly prototype solutions to unblock customers and prove out new use cases.
- Stay current on the latest AI/ML research and tools.
- Identify opportunities for productization based on common customer patterns.
Requirements
- 8+ years of software engineering experience with strong fundamentals in data structures, algorithms, and system design.
- Production Python expertise with experience in modern ML/AI frameworks (e.g., LangChain, LlamaIndex, HuggingFace, OpenAI API).
- Experience with cloud platforms (AWS, GCP, or Azure) and modern data infrastructure.
- Strong problem-solving skills with the ability to navigate ambiguous requirements and rapidly iterate toward solutions.
- Excellent communication skills with the ability to explain complex technical concepts to both technical and non-technical audiences.
- Deep understanding of LLMs including prompting techniques, embeddings, and RAG architectures.
- Experience building and deploying AI agents or autonomous systems in production.
- Knowledge of vector databases and semantic search systems.
- Experience with containerization (Docker, Kubernetes) and CI/CD pipelines.
- Experience using Terraform, Bicep, or other Infrastructure as Code (IaC) tools.
- Proven ability to work with customers in a technical consulting, solutions engineering, or product engineering role.