Staff/Senior Staff Software Engineer, Agent Infrastructure
$240k - $360k • Remote • Toronto • Full-time
Posted 9d ago
Remote Work Policy
Fully remote
Employment Type
Full-time
Categories
AI Agent Engineer
About the job
Cohere is seeking a Staff/Senior Staff Software Engineer to join their Agent Infrastructure team. This role is at the forefront of building the next generation of agentic AI infrastructure, blending ML systems, distributed infrastructure, and developer experience to create a platform for autonomous AI agents at scale. You will tackle complex, forward-looking challenges such as secure code execution, agent state management, model routing, and resource management for agent workflows. This position is ideal for individuals with a strong systems background and a deep understanding of ML, comfortable with building reliable distributed systems and designing platforms for emerging agentic capabilities.
Responsibilities
- Build core platform capabilities for execution, storage, and state management of AI agents.
- Develop secure execution environments for agent-generated code.
- Implement identity, authentication, and trust boundaries for agents.
- Design and implement model routing and orchestration across different model types and environments.
- Manage rate limiting, quotas, and resource allocation for agent workflows.
- Create state management, memory, and filesystem abstractions for agents.
- Translate emerging ML research into production-ready infrastructure.
- Prototype and evaluate new technologies for agent infrastructure.
- Collaborate with research teams to define future infrastructure needs for agent systems.
Requirements
- Experience building production ML infrastructure with strong systems fundamentals.
- Hands-on experience with agentic systems, multi-agent workflows, or agent development frameworks.
- Familiarity with model routing and LLM provider frameworks.
- Experience with scalable, fault-tolerant distributed systems and Kubernetes.
- Proven ability to move quickly on prototypes and make sound productionization decisions.
- Experience across on-prem, private cloud, and public cloud environments.
- Familiarity with storage systems, embedded databases, or filesystem abstractions.
- Experience with code execution sandboxes (e.g., gVisor, Firecracker, Kata, WASM).
- Interest in emerging ML infrastructure, edge inference, or browser-native models.
- Open-source contributions to LLM or agent infrastructure projects.
- Experience with identity, workload auth, or capability-based security systems.
Benefits
- Weekly lunch stipend of $75/£75 or equivalent.
- Full health and dental benefits, including a mental health budget.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months.
- Annual enrichment benefits (arts & culture, fitness/wellness, quality time, workspace improvement).
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days).
- Budget for traveling to other offices if remote.
- Annual company offsite.
- Co-working benefit for those not near an office.
- $500 home office stipend.