Principal Backend Engineer (Forward Deployed)
$250k - $300k • New York - Hybrid; San Francisco Bay Area - Hybrid
Posted 1d ago
About the job
As a Principal Forward Deployed Engineer at Invisible, you will lead high-impact, AI-powered solutions that reshape how clients operate their most critical workflows. You will drive the strategy, architecture, and execution of end-to-end systems, working directly with clients. This role is a hybrid of AI engineering, software building, and technical consulting, ideal for someone who wants to be hands-on with models and close to their impact. The team is increasingly focused on applied AI work, building and deploying LLM-powered and agentic systems in production, making strong ML engineering instincts a valuable addition to architecture and leadership experience.
Responsibilities
- Partner with stakeholders to scope, design, and lead implementation of AI-driven solutions, including LLM and agentic systems.
- Identify transformational opportunities in complex workflows and develop repeatable systems.
- Lead architecture design discussions considering performance, scalability, cost, and reliability for AI/ML systems like RAG pipelines.
- Own projects from initial discovery through full deployment, including client delivery, internal coordination, and post-launch iteration.
- Build shared infrastructure, reusable components, and internal playbooks, including MLOps practices.
- Coach and mentor mid-level engineers and contribute to the culture of forward-deployed AI engineering.
Requirements
- 10+ years of software engineering experience, with significant time building data, ML, or backend systems.
- Deep Python proficiency and hands-on experience with Hugging Face, LangChain, OpenAI, Pinecone, LLMs, and AI Agents.
- Skilled in full-stack and API deployment patterns using Docker, FastAPI, Kubernetes, and cloud environments (GCP, AWS, Azure).
- Experience with workflow orchestration libraries, pub/sub systems (Kafka), and schema governance.
- Expertise in data governance and operations, including Unity Catalog, data contracts, Delta/ETL pipelines, and replay processes.
- Experience with MLOps frameworks and best practices for model versioning, monitoring, and lifecycle management.
- Strong product and system design instincts, translating business needs into technical architecture.
- Experience building systems from messy data and ambiguous requirements.
- Excellent communication and client-facing skills, with experience leading technical and non-technical stakeholder conversations.
- Proven experience owning projects from scoping through deployment in ambiguous, high-stakes environments.
- Bachelor's degree in Data Science, Computer Science, or equivalent professional experience.
- Willingness to be on-call for customers.
- Willingness to travel for on-site client engagements.
- Available to work from the office a few days per week, depending on team needs.
Benefits
- Competitive pay reflecting market conditions and team member value.
- Salary structure accounting for regional cost of living differences.
- Annual salary range: $250,000 - $300,000 USD.