Staff Software Engineer, Anti-Abuse & Security
Remote • Foster City, CA • FullTime
Posted 3mo ago
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
The Anti-Abuse team at Replit is responsible for defending the platform from exploitation by detecting and preventing various forms of abuse, including phishing, cryptomining, LLM token farming, and malicious use of the platform. This role involves adversarial work, requiring the development of detection systems, heuristics, and automated responses to stay ahead of constantly adapting attackers. A unique aspect of this role is the AI-native nature of Replit's platform, presenting opportunities to build guardrails for AI-generated code, detect prompt injection attacks at scale, and leverage LLMs for defense. The position offers hands-on experience applying AI to security problems in a production environment with real attackers, with end-to-end ownership from identifying abuse patterns to shipping scalable solutions.
Responsibilities
- Design and implement LLM guardrails for AI-generated code and agent interactions.
- Build AI-powered detection systems using LLMs for threat identification and automated response.
- Develop and operate abuse detection systems for phishing, cryptomining, account takeover, and financial fraud.
- Design automated response mechanisms to enforce platform policies.
- Manage the full abuse response lifecycle: detection, investigation, enforcement, and appeals.
- Analyze attack patterns using BigQuery and Hex to create new detection rules.
- Maintain and extend internal detection tools like Slurper and Netwatch.
- Integrate and tune security scanners (SAST, SCA) in CI pipelines with performance SLAs.
- Track abuse trends, measure detection effectiveness, and adapt defenses.
Requirements
- 8+ years of experience in security engineering, anti-abuse, trust & safety, or fraud detection.
- Strong programming skills in Python and/or TypeScript.
- Experience with SQL and large-scale data analysis (BigQuery, Snowflake, or similar).
- Experience building or fine-tuning ML/LLM-based classifiers for security or abuse detection.
- Familiarity with prompt injection, jailbreaking, and other LLM-specific attack vectors.
- Ability to investigate complex abuse patterns and translate findings into automated defenses.
- Familiarity with common attack patterns (phishing, account takeover, credential stuffing, resource abuse).
- Clear communication skills for cross-functional collaboration.
Benefits
- Competitive Salary & Equity
- 401(k) Program with a 4% match (US Only)
- Health, Dental, Vision and Life Insurance
- Short Term and Long Term Disability
- Paid Parental, Medical, Caregiver Leave
- Flexible Time Off (FTO) + Holidays
- Commuter Benefits (In-Office Only)
- Monthly Wellness Stipend
- Autonomous Work Environment
- In Office Set-Up Reimbursement (In-Office Only)
- Quarterly Team Gatherings
- In Office Amenities (In-Office Only)