Staff Software Engineer (Agentic Search) - Index
Salary not listed • Zurich, Switzerland • Full-time
Posted 1mo ago
About the job
Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. We are seeking a Staff Software Engineer to focus on the indexing and data processing layer of a novel search engine designed for AI agents. This role involves building systems to ingest, process, and organize massive datasets into efficient, queryable structures, operating on offline and nearline pipelines to ensure data freshness and accessibility for retrieval systems. The position requires designing and implementing large-scale indexing systems capable of handling tens of gigabytes per second across continuously evolving datasets, with a strong emphasis on throughput, scalability, and correctness.
Responsibilities
- Design, implement, and operate large-scale indexing systems and data pipelines for search infrastructure.
- Develop and optimize indexing strategies balancing performance, freshness, and resource efficiency.
- Work on storage formats, compaction strategies, and update mechanisms for data accessibility and currency.
- Ensure reliability and predictability of high-throughput pipelines.
- Build well-tested components with clear responsibilities and interaction contracts.
- Define and implement observability primitives, including structured logs, metrics, and data quality signals.
- Monitor throughput, resource usage, and cost, driving optimizations as needed.
- Collaborate with runtime and ML teams to align indexing outputs with retrieval and ranking requirements.
- Enable safe experimentation on indexing strategies and data processing logic through controlled rollouts and quality signals.
Requirements
- 8+ years of experience building production backend or data infrastructure systems.
- Strong Go experience (C++/Rust is a plus).
- Experience with large-scale data processing systems (10+ GiB/sec throughput, petabyte-scale datasets).
- Experience building or operating databases, storage systems, data planes, or indexing pipelines.
- Strong understanding of distributed systems, fault tolerance, consistency, and scalability.
- Experience running production systems and handling operational incidents.
- Systems-thinking mindset and ability to reason about end-to-end data flows.
Benefits
- Competitive compensation
- Career growth and learning opportunities
- Flexibility and ownership
- Collaborative and innovative culture
- Opportunity to work on impactful AI projects
- International environment and talented teams