Senior Software Engineer, Agent Infrastructure
Remote • Toronto • FullTime
Posted 4mo ago
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Agent Engineer
About the job
Cohere is seeking a Senior Software Engineer to join their Agent Infrastructure team. This role is at the forefront of building the next generation of agentic AI infrastructure, blending ML systems, distributed infrastructure, and developer experience. You will tackle complex, forward-looking challenges in areas such as secure code execution, agent state management, model routing, and resource management for autonomous AI agents at scale. The ideal candidate possesses deep systems knowledge combined with ML intuition, enabling them to build reliable infrastructure and navigate distributed systems tradeoffs while understanding how emerging agentic capabilities influence platform design.
Responsibilities
- Build core platform capabilities for execution, storage, and state management for agentic AI.
- Turn emerging ML research ideas into production-ready infrastructure.
- Prototype and evaluate new technologies, influencing production adoption.
- Partner with research teams to define infrastructure needs for future agent systems.
- Develop secure execution environments for agent-generated code.
- Implement identity, authentication, and trust boundaries for agents.
- Design model routing and orchestration across different model types and environments.
- Manage rate limiting, quotas, and resource allocation for agent workflows.
- Create state management, memory, and filesystem abstractions for agents.
Requirements
- Experience building production ML infrastructure with strong systems fundamentals.
- Hands-on experience with agentic systems, multi-agent workflows, or agent development frameworks.
- Familiarity with model routing and LLM provider frameworks.
- Experience with scalable, fault-tolerant distributed systems and Kubernetes.
- Proven ability to move quickly on prototypes and make sound productionization decisions.
- Experience across on-prem, private cloud, and public cloud environments.
- Familiarity with storage systems, embedded databases, or filesystem abstractions.
- Experience with code execution sandboxes (e.g., gVisor, Firecracker, Kata, WASM runtimes).
- Interest in emerging ML infrastructure, edge inference, or browser-native models.
- Open-source contributions to LLM or agent infrastructure projects.
- Experience with identity, workload auth, or capability-based security systems.
Benefits
- Weekly lunch stipend ($75/£75 or local equivalent).
- Full health and dental benefits, including a mental health budget.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months.
- Annual enrichment benefits (Arts & culture, fitness/wellness, quality time, workspace improvement).
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days).
- Budget for traveling to other offices (for remote employees).
- Annual company offsite.
- $500 home office stipend.