Senior, ML Engineer - VLM
$177k - $213k • Remote • Ann Arbor, MI, Remote - US • Full-time
Posted 3mo ago
This job has been open for more than 60 days. It may no longer be actively hiring, so check with the company before putting a lot into an application.
Job Location
Ann Arbor, MI, Remote - US
Tech Stack
Remote Work Policy
Fully remote
Employment Type
Full-time
Categories
Machine Learning Engineer
About the job
Torc is seeking a Senior ML Engineer to join their Offline Perception team, focusing on creating high-quality, semantically rich training data for end-to-end Vision-Language-Action (VLA) models. This role involves owning the dataset layer that these models learn from, transforming petabytes of logged multi-modal fleet data into VLM/VLA-ready datasets. You will design, implement, and deploy cloud-based pipelines for geometric and semantic annotations, build VLM-assisted auto-labeling systems, and generate reasoning-grounded labels to support model training and explainable driving behavior. The position also requires mining and curating the long tail of rare scenarios, closing the data flywheel by defining schemas and quality metrics, and partnering closely with the end-to-end model team to ensure continuous dataset delivery. You will scale these processes on cloud infrastructure using distributed, reproducible pipelines and disciplined software practices.
Responsibilities
- Own and develop cloud-based pipelines for converting multi-sensor data into VLM/VLA training datasets, including geometric and semantic annotations.
- Develop VLM-assisted auto-labeling systems using foundation models for tasks like open-vocabulary detection and dense captioning.
- Generate language-grounded reasoning and chain-of-causation style annotations aligned with ego-motion and trajectories.
- Mine and curate datasets focusing on rare, difficult, and high-uncertainty scenarios to improve downstream model metrics.
- Define dataset schemas, quality metrics, and validation processes, and manage the data flywheel loop.
- Collaborate with VLM/VLA model developers to define dataset specifications and ensure quality and delivery cadence.
- Build distributed, reproducible pipelines using columnar data formats and distributed compute on cloud infrastructure.
- Lead projects, mentor junior engineers, conduct design reviews, and set coding and annotation standards.
- Stay updated on advances in multimodal models, auto-labeling, and autonomous driving, translating research into production systems.
Requirements
- Bachelor's Degree in Computer Science, Robotics, Electrical Engineering, or related technical field with 6+ years of experience, or Master's Degree with 3+ years of experience.
- Proficiency in Computer Vision & Deep Learning, including model training and at least two of: 2D/3D Object Detection, Tracking, Sensor Fusion, Semantic Segmentation, BEV, Depth Estimation.
- Hands-on experience with Multimodal / VLM, including open-vocabulary or zero-shot recognition, dense captioning, or semantic embeddings/search.
- Experience in Model Data Curation, building datasets that improve model performance, and large-scale Parquet data processing.
- Familiarity with Distributed ML & data frameworks like PyTorch, Lightning, Ray, or Spark.
- Experience with Scaled MLOps & Tooling, including experiment tracking, model registry, MLflow/Weights & Biases, and ML metrics/evaluation.
- Strong Python software development skills, experience with cloud-based development environments, CI systems (e.g., GitHub Actions), and Docker.