Applied ML Engineer - Edge Devices
$155k - $245k • Remote • USA - Remote • FullTime
Posted 1mo ago
Job Location
USA - Remote
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Engineer
About the job
Deepgram is seeking an Applied ML Engineer to focus on deploying their advanced speech models to a variety of edge devices and non-NVIDIA accelerators. This role involves adapting existing models to run efficiently within diverse hardware constraints, including different inference runtimes and operator sets. You will be responsible for optimizing models through techniques like quantization and precision adjustments, validating performance on real hardware, and building robust deployment pipelines for edge targets. This is a hands-on engineering position, distinct from research or cloud-focused roles, aimed at ensuring Deepgram's cutting-edge voice AI can be deployed effectively across a wide range of customer hardware.
Responsibilities
- Port Deepgram speech models to non-NVIDIA and edge platforms, adapting model structure and parameters to fit target operator sets, runtimes, and kernels with minimal modification.
- Manage serving-side model decisions for edge targets, including quantization, precision choices, operator substitution, graph rewrites, and architecture tweaks to meet device constraints while maintaining accuracy and latency.
- Validate all model ports on real hardware by building accuracy, latency, throughput, and memory benchmarks per platform.
- Develop the deployment path for edge targets, encompassing model packaging, conversion pipelines, versioning, and automated delivery for repeatable model shipping.
- Collaborate with Embedded AI Engineers to adapt models for custom kernels when standard options are insufficient.
- Engage with platform and silicon vendors to integrate their runtimes and toolchains, ensuring Deepgram deployments adhere to expected model formats and operator conventions.
- Provide feedback on edge constraints to Research and Impeller teams to facilitate easier future model porting.
- Address adjacent production concerns at the edge as the team grows, such as automated deployment, model security, integrity, and fleet-level observability.
Requirements
- Hands-on experience deploying ML models to edge or non-NVIDIA hardware in production; cloud-only or GPU-only serving experience is not sufficient.
- Working knowledge of quantization and precision tradeoffs (e.g., INT8, FP16, mixed precision, calibration) and their impact on accuracy and latency on real targets.
- Experience with at least one edge or vendor inference runtime and its conversion toolchain (e.g., ONNX Runtime, TFLite, ExecuTorch, OpenVINO, Qualcomm AI Engine, or a vendor NPU SDK).
- Ability to modify a model to fit a platform's constraints.