Embedded AI Engineer, On-Device Models
$219k - $274k • USA - Remote • FullTime
Posted 3mo ago
Job Location
USA - Remote
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Deepgram is seeking an Embedded AI Engineer to work on optimizing their speech models for a wide range of hardware, including non-NVIDIA accelerators, embedded SoCs, mobile processors, and specialized devices with tight resource constraints. This role operates at the lowest layer of the edge stack, focusing on writing and optimizing custom kernels and operators, collapsing models onto device-specific execution units, and performing target-side quantization and assembly-level tuning. The engineer will collaborate with Applied ML Engineers to ensure Deepgram's models can run efficiently on new hardware platforms, making them viable for edge deployment. This position is ideal for senior embedded engineers with a passion for AI or staff-level engineers looking to shape the future of on-device AI model deployment.
Responsibilities
- Write and optimize custom kernels and operators in C, C++, Rust, and platform assembly for various accelerators and embedded systems.
- Perform target-side optimization, including quantization, operator fusion, memory layout, and architecture-specific compilation to meet performance budgets.
- Integrate with vendor NPU/DSP toolchains and edge inference runtimes, extending them with custom operators as needed.
- Deliver reusable kernels and runtime components for Applied ML Engineers.
- Build performance-critical runtime code for embedded Linux, bare-metal, and RTOS targets.
- Establish per-platform benchmarking and validation for key performance metrics.
- Partner with silicon and platform vendors on SDK integration and low-level performance tuning.
- Provide feedback on hardware constraints to Applied ML and Research teams to facilitate model design for constrained targets.
Requirements
- Experience delivering production systems on resource-constrained hardware (embedded systems, mobile, edge AI, low-power devices).
- Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments.
- Hands-on experience with model optimization for on-device deployment (quantization, pruning, knowledge distillation, architecture-specific compilation).
- Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.
- Strong understanding of hardware-software interaction, including CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management.