Senior Deep Learning Engineer, Accuracy Evaluation
$375k - $650k • Remote • Poland, Remote • Full-time
Posted 12h ago
Remote Work Policy
Fully remote
Employment Type
Full-time
Categories
LLM Engineer
About the job
We are seeking senior engineers to pioneer new methodologies for accurately assessing the performance and capabilities of ground-breaking deep learning models, including LLMs, RAG, agents, and vision models. You will collaborate across the organization to bring the latest flagship models from our community and partners to life. This role offers an outstanding opportunity to craft the future of AI at a fast-growing company at the forefront of the AI revolution. Join our team of world-class software engineers and partners to deliver the most advanced models with lightning-fast inference. You'll work on the most powerful, enterprise-grade GPU clusters capable of hundreds of PetaFLOPS and gain early access to unreleased hardware, making a direct impact on NVIDIA's roadmap and the broader AI landscape!
Responsibilities
- Design and build evaluation environments for frontier models, producing auditable accuracy signals.
- Research and develop novel evaluation methodologies for emerging model families and capability domains.
- Build and operate evaluation infrastructure and pipelines, including benchmark environments and statistical analysis tooling.
- Partner with research, training, and customer teams to translate evaluation signals into concrete decisions for model releases and training direction.
Requirements
- BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related field.
- 6+ years of hands-on experience with LLMs, designing and running evaluations for large language models or multimodal AI systems.
- Strong statistical foundations, including experimental design and significance testing.
- Proven experience building evaluation infrastructure and reproducible CI systems.
- Clear, precise communicator capable of translating quantitative results into decisions for various stakeholders.
Benefits
- Access to enterprise-grade GPU clusters.
- Early access to unreleased hardware.
- Base salary range: 375,000 PLN – 650,000 PLN (Poland)