AI Infrastructure Engineer Jobs in New York

Harvey is transforming legal and professional services by combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise. This role is an opportunity to help build a generational company at an inflection point, with strong product-market fit and investor support. The team moves fast, takes ownership, and is deeply committed to the mission, operating with intensity and pushing for excellence. As a Member of Technical Staff, Infrastructure, you will build the systems that power how we adapt and improve leading open-weight models for legal work, focusing on scaling post-training efforts and owning the infrastructure behind distributed training, RL rollouts, agent environments, and internal tooling for data management, evals, and job tracking.

$230k - $340k

Turing

Turing's mission is to accelerate superintelligence to drive real economic progress. The company works with frontier AI labs to generate high-quality datasets, reinforcement learning environments, and frontier research benchmarks. Turing also partners with Fortune 500 enterprises to build and deploy end-to-end agentic AI systems within mission-critical workflows. This role involves leading the Turing Enterprise AI Platform technical team, which is being built out for a late 2025 SDK release and a Q1 2027 tenant onboarding. The goal is to grow the team to eight to ten members and collaborate with the Chief AI Architect on client-facing architecture and evaluation.

Salary not listed

Turing

Enterprise AI is building the platform that enables reliable deployment of AI systems across enterprise environments. As a Platform Engineer, you will build and operate the core of Turing’s Enterprise AI platform, including platform services, the model gateway, multi-tenant infrastructure, CI/CD, infrastructure as code, and observability. The central challenge is creating repeatable deployments across environments Turing may not control, including client cloud accounts, dedicated instances, and sovereign installations. This role requires hands-on software engineering and operational ownership, with a focus on building production software and contributing deeply to platform services.

Salary not listed

Scale AI

Scale GP is building the infrastructure that makes enterprise AI seamless. We are looking for a Senior Infrastructure Engineer to act as a primary technical lead, engineering the 'paved road' for our knowledge retrieval and inference engines. You won't just be managing resources; you’ll be defining the deployment standards for Agentic workflows at scale. Your mission is to bridge the gap between complex AI orchestration and world-class infrastructure, ensuring our platform remains the most reliable destination for enterprise agents. The ideal candidate thrives in a fast-paced environment, has a passion for both deep technical work and mentoring, and is capable of setting a long-term technical strategy for a critical domain while maintaining a strong, hands-on delivery focus. You will architect and implement solutions across multiple cloud providers (GCP, Azure, AWS) for customers in diverse, highly-regulated industries like healthcare, telecom, finance, and retail.

$216k - $270k

Anthropic

Anthropic is seeking a Staff+ Software Engineer to enhance the vector search infrastructure powering Claude. This role is crucial for Claude's ability to access information, impacting web search, conversation memory, and model training. You will join a small, foundational team responsible for the architecture, performance, scalability, and cost of the vector database layer. As a hands-on engineer, you will contribute to coding and make key architectural decisions, shaping the future evolution of this critical system within the Databases organization.

$320k - $485k

LangChain is seeking an Engineering Manager to lead the team responsible for building LangSmith, our observability and evaluation platform for LLM applications. This role involves defining the technical direction, nurturing a talented engineering team, and collaborating with product and design to deliver features that empower developers to build and deploy trustworthy AI systems. You will play a key part in shaping the future of AI observability at scale, contributing to a company that has raised $125M and is experiencing rapid growth.

$200k - $250k

Reflection ai

Reflection is a research lab dedicated to making intelligence open and accessible. We build open models that empower users to control their intelligence and shape the future of AI. In this role, you will develop the critical evaluation infrastructure that researchers rely on to understand model capabilities, inform training decisions, and assess readiness for release. Your work will encompass evaluation authoring, distributed execution, and results analysis, with a specific emphasis on pre-training. You will collaborate closely with researchers to enhance the speed, reliability, and ease of use of evaluations, ensuring that the outcomes accurately reflect the models being assessed.

Salary not listed

Percepta

Percepta is seeking an AI Infrastructure Engineer to take ownership of the infrastructure, deployment, and operational reliability for Percepta's AI systems, including autonomous agents. This role involves hardening existing infrastructure, strengthening deployment pipelines, and defining new SRE practices for operating autonomous systems. You will be instrumental in bridging the gap between AI research and production, contributing to the foundational decisions that shape the future of agentic infrastructure.

$180k - $400k

Faire is a technology wholesale platform connecting independent retailers globally. We are seeking a Staff Engineer to own and evolve our data plumbing, ensuring data is accessible, accurate, and queryable for various teams including Product Engineering, Data Science, and Machine Learning. This hands-on role involves designing and building the next generation of our data ingestion and processing systems, and serving as the go-to expert for data movement at Faire.

$247k - $339k

At WRITER, our mission to expand human capacity with superintelligence relies on a foundational truth: our platform must be available, performant, and reliable, 24/7. As an Infrastructure engineer, you'll be at the heart of making this a reality, impacting every enterprise customer who trusts us with their AI-powered workflows. This isn't just about keeping the lights on; it's about pushing the boundaries of what's possible, proactively identifying and solving complex systemic challenges, and laying the groundwork for our rapid growth and the evolving demands of enterprise generative AI. You'll build resilient systems, automate across the stack, and champion reliability best practices, directly enabling our ambitious product roadmap and ensuring our customers always have access to the powerful tools they need.

$155k - $240k

Databricks

Databricks is seeking a Staff Software Engineer to join their AI Research Infrastructure team. In this role, you will be responsible for developing and operating the research stack that powers Databricks' AI research efforts. You will design and build services for scheduling, orchestrating, and observing large-scale training and inference experiment workloads across thousands of GPUs. Your work will involve improving developer tooling to ensure researchers can iterate quickly while maintaining reliability, efficiency, and security. You will collaborate closely with research scientists, ML engineers, and platform teams to transform experimental workloads into robust, repeatable pipelines and push the boundaries of the infrastructure's capabilities.

$199k - $270k

Databricks

At Databricks, we are passionate about enabling data and AI teams to solve the world's toughest problems. We build and run the world's best data and AI infrastructure platform. As part of the AI Platform team, you'll build the substrate that powers everything from data apps, AI agents, model training, model serving, and vector search. You'll be joining a high-agency, high-visibility team operating at the frontier of AI infrastructure with deep ties to research, product, and real-world enterprise use cases. Databricks Mosaic AI is one of our fastest-growing businesses helping thousands of our customers democratize AI within their organizations. We’re building the infrastructure that powers the next generation of AI.

$165k - $220k

Anthropic

We are seeking software engineers to join our Safeguards team, focusing on building robust safety and oversight mechanisms for our AI systems. This role involves developing distributed and scalable systems to monitor models, prevent misuse, and ensure user well-being. You will work to detect unwanted model behaviors and prevent disallowed use, applying your technical skills to uphold principles of safety, transparency, and oversight while enforcing terms of service and acceptable use policies.

$320k - $485k