Senior/Staff Software Engineer (Back-End), Inference
$205k - $315k • Remote • Canada • Full-time
Posted 15d ago
About the job
Cohere is seeking a highly experienced Back-End Software Engineer to join their core systems team, focusing on the Model Vault platform. This platform is Cohere's inference solution for deploying and operating AI models securely at enterprise scale. The role involves defining and evolving the architecture of systems that handle identity, deployments, metering, billing, security, privacy, and confidential computing. You will work on scaling these systems to support millions of users and demanding enterprise workloads, ensuring reliability, security, performance, and simplicity in a domain with unique challenges.
Responsibilities
- Evolve and scale backend systems for authentication, authorization, deployment management, billing, metering, and customer lifecycle.
- Lead major architectural initiatives for identity, payments, usage systems, and platform scalability.
- Build reliable control-plane services for provisioning, configuring, scaling, and operating model deployments across various clouds and hardware.
- Design APIs and distributed systems with strong guarantees for correctness, consistency, isolation, failure recovery, idempotency, and observability.
- Integrate security and privacy into the platform architecture, including tenant isolation, least-privilege access, auditability, secrets management, encryption, and confidential computing.
- Solve complex problems spanning backend systems, cloud infrastructure, security, and AI inference.
- Drive projects end-to-end, from architecture and threat modeling to implementation, rollout, operations, and iteration.
Requirements
- Strong backend engineering fundamentals in distributed systems, databases, networking, operating systems, and security.
- Experience building production systems in languages like Go, Python, Java, C++, or Rust.
- Ability to write clean, maintainable, well-tested code and make thoughtful tradeoffs.
- Experience designing and operating production systems at significant scale where correctness, availability, security, and performance are critical.
- Ability to take ambiguous, high-impact technical problems, define architecture, and drive them to production.
- Strong systems-design instincts, understanding concepts like concurrency, consistency, caching, queues, rate limiting, backpressure, failure domains, idempotency, and graceful degradation.
- Deep understanding of security boundaries, authentication, authorization, identity, cryptography, secrets, and tenant isolation.
- Comfort reasoning across backend services, APIs, databases, distributed systems, and cloud infrastructure.
- Ability to debug complex problems across service or system boundaries.
- Experience with production engineering details like instrumentation, capacity planning, migrations, backwards compatibility, incident response, and safe rollouts.
- Willingness to challenge existing architecture and evolve foundational components.
- Track record of delivering technically difficult projects from unclear starting points.
Benefits
- Weekly lunch stipend of $75/£75 or equivalent.
- Full health and dental benefits, including a mental health budget.
- RRSP matching, 401K, or Pension Scheme.
- 100% Parental Leave top-up for up to 6 months.
- Annual enrichment benefits (arts & culture, fitness/wellness, quality time, workspace improvement).
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks (30 working days) of paid vacation.
- Budget for traveling to other offices (if remote).
- Annual company offsite.
- Daily lunch program, snacks, and social events (for in-office).
- Co-working benefit (for those not near an office).
- $500 home office stipend.