ML inference Jobs

Anyscale is seeking a Distributed LLM Inference Engineer to develop systems and optimizations that push the boundaries of performance for large-scale inference. This critical role will contribute to Anyscale's market-leading position in AI infrastructure. You will work closely with product teams to deliver end-to-end solutions for batch and online inference, integrating Ray Data and LLM engines to achieve cost-effective, large-scale ML inference. You will also integrate with open-source software like vLLM, contribute to Anyscale's solutions, and improve open-source projects. Staying abreast of the latest advancements in open-source and research is key to implementing and extending best practices.

$170k - $245k