Staff AI & Agentic Platform Engineer
$253k - $336k • Costa Mesa, California, United States; Washington, District of Columbia, United States
Posted 1d ago
Job Location
Costa Mesa, California, United States; Washington, District of Columbia, United States
Tech Stack
Remote Work Policy
On-site
Categories
AI Agent Engineer
About the job
Discovery is seeking an AI & Agentic Platform Engineer to join their expeditionary force. This role involves taking unproven concepts and building the analysis or prototype to test them, working on challenges across autonomy, sensor analysis, and battle space management. The team focuses on moving AI models from research into production, specifically onto classified platforms and edge hardware where reliability is paramount. Rigorous and repeatable evaluation of model performance is critical as the model portfolio and classified work expand.
Responsibilities
- Develop comprehensive test scenarios and simulation environments to assess agentic AI performance in classified simulations.
- Lead the integration and validation of agentic AI systems onto classified platforms and edge hardware.
- Build reusable evaluation pipelines, automated test harnesses, and monitoring dashboards for continuous validation.
- Define and establish systematic, historic capture of performance metrics for AI and traditional models.
- Partner with cross-functional teams to define requirements, document test results, and contribute to AI model cards and deployment readiness reviews.
- Architect, build, and maintain the end-to-end evaluation and validation infrastructure for agentic AI and ML models.
- Analyze and resolve issues uncovered during evaluation and in deployment to ensure reliability and operational success.
Requirements
- Bachelor's or Master's degree in Computer Science, Machine Learning, Software Engineering, Computer Engineering, Electrical Engineering, or a related technical field.
- 8-10 years of hands-on experience writing production-grade code, with strong Python skills.
- Demonstrated experience owning evaluation or automated testing for production ML or software systems.
- Hands-on experience designing evaluation methodologies for AI or machine learning models.
- Experience designing or working extensively with simulation environments.
- Working knowledge of how to define, capture, and reason about performance metrics for AI and traditional models.
- Ability to navigate and contribute to complex systems and established codebases.
- Comfort operating between technical program management and software engineering.
- Must have a TS/SCI clearance.