Staff Engineer, Customer Insights & Alerts
$210k - $255k • Remote • San Francisco, CA - US • FullTime
Posted 3mo ago
Job Location
San Francisco, CA - US
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Crusoe is seeking a Staff Engineer to be a key contributor to platform detection, working closely with Fleet, CMK, and Data teams. This role involves owning the logic for correlating, deduplicating, and managing the lifecycle of detector signals to generate customer-facing insights, while also enhancing the platform's core detection capabilities. It's a comprehensive software engineering position where you will deliver customer-facing features, develop new systems, and scale existing services. The ideal candidate possesses deep expertise in anomaly detection, heuristics, machine learning (including RL), and agentic applications within global-scale infrastructure. Beyond technical skills, you will be instrumental in elevating the team's overall performance and standards.
Responsibilities
- Own correlation and deduplication logic, building heuristics for thresholds, grouping rules, and severity scoring to create precise, customer-facing insights.
- Partner with the Data team on fleet-level anomaly detection and map outputs to actionable insights for tenants.
- Drive detection fidelity by reducing false positives, expanding signal coverage, and building feedback loops for accurate threshold maintenance.
- Integrate ML/RL outputs into customer alerting, translating model signals into actionable judgment calls.
- Ship customer-facing features end-to-end across the CIA stack, including alert rule engines, control plane APIs, action workflows, and insight delivery surfaces.
- Build new systems from scratch and scale existing ones to support a rapidly growing global fleet.
- Drive cross-team design discussions, evaluate technical trade-offs, and collaborate with product teams on early requirements.
- Mentor engineers through code reviews, design feedback, and coaching, and help define hiring standards.
Requirements
- Deep experience building anomaly detection systems, heuristics-based rule engines, or ML/RL systems for infrastructure or data-intensive domains.
- Proven ability to manage precision/recall trade-offs and build continuous feedback loops for detection system accuracy.
- Strong foundation in building reliable, scalable backend systems and leading complex architectural discussions.
- 5+ years of experience shipping production software using modern compiled or systems languages (Go, Rust, C++, Java, or similar).
- Expertise with time-series data, telemetry pipelines, and observability primitives.
- Ability to clearly explain complex logic and trade-offs to both technical and non-technical partners.
- Experience with GPU profiling tools (Nsight, NCCL Inspector) or hardware-level infrastructure diagnostics.
- Background in observability platforms or products.
- Experience shipping production agentic applications, such as LLM-based agents for triage, root-cause analysis, or automation.
- Experience with reinforcement learning applied to operational or infrastructure problems.
- Familiarity with large-scale fleet management or cloud infrastructure.
- Passion for building team culture and engineering quality of life.
Benefits
- Competitive compensation and equity packages
- Restricted Stock Units
- Paid time off, paid holidays & leave of absence programs
- Comprehensive health, dental & vision insurance
- Employer contributions to HSA account
- Paid parental leave
- Paid life insurance, short-term and long-term disability
- Professional development & tuition reimbursement
- Mental health & wellness support
- Commuter benefits (parking & transit)
- Cell phone stipend
- 401(k) Retirement plan with company match up to 4% of salary
- Volunteer time off
- Global travel insurance & emergency assistance
- Daily meals allowance
- Additional perks & programs specific to location