Senior Software Engineer (Agentic Search) - Web Access Engineer
Salary not listed • Amsterdam, Netherlands; Israel; London, United Kingdom • Full-time
Posted 15d ago
Job Location
Amsterdam, Netherlands; Israel; London, United Kingdom
Also open in Zurich, Switzerland
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Agent Engineer
About the job
Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data training to production deployment. We are seeking a Senior Software Engineer to focus on the web access infrastructure for a novel search engine designed for agentic AI consumption. This role involves ensuring reliable and efficient access to public web content by investigating website and browser communication, diagnosing and fixing request failures, and building scalable systems. Your work will be crucial for providing high-quality content to downstream crawling, indexing, and retrieval systems, enabling AI agents to retrieve, filter, and reason over real-world information at scale.
Responsibilities
- Diagnose failed or degraded requests across network, TLS, HTTP/2, headers, session state, client-side scripts, and browser characteristics.
- Analyze how websites verify and classify clients based on client-side scripts and protocol-level behavior.
- Build clients and tooling that are compatible with standard browsers across all layers, using a real browser when necessary.
- Monitor access quality and detect regressions as websites change, identifying causes and measuring success.
- Develop tooling for large-scale traffic capture, comparison against real browsers, and tracking success rates and fetch costs.
- Select the appropriate fetch method (lightweight HTTP client or full browser) for each host, balancing quality and cost.
Requirements
- 5+ years of experience building production systems.
- Expertise in the request lifecycle, including HTTP/1.1, HTTP/2, HTTP/3, TLS handshakes, TLS fingerprints (JA3/JA4), and header behavior.
- Understanding of what a server observes and what a client controls during a request.
- Proficiency in debugging methods such as replaying captured browser traffic and comparing against a real browser.
- Experience analyzing complex or obfuscated JavaScript and observing script behavior by hooking browser APIs.
- Knowledge of how browser, device, and network characteristics are measured and their consistency across layers.
- Familiarity with browser instrumentation and the Chrome DevTools Protocol.
- Experience running systems in production with measurable metrics for requests per day, sites/hosts, and success rates.
- Strong programming skills in at least one of Go, Python, Rust, or C++.
- Experience working at a web data provider, browser/web-security company, or data collection company (preferred).
- Contributions to browser networking or fingerprinting open-source tooling, or public write-ups on client verification (preferred).
- Experience building or maintaining a patched browser (preferred).
- Knowledge of networks, proxies, and IP reputation (preferred).
- Experience with internet-scale distributed systems, Kubernetes, and cloud infrastructure (preferred).
Benefits
- Competitive compensation
- Career growth and learning opportunities
- Flexibility and ownership
- Collaborative and innovative culture
- Opportunity to work on impactful AI projects
- International environment and talented teams