Senior Machine Learning Engineer, Accelerated Apache Spark

About the job

Responsibilities

• Design and implement ML solutions for predicting and optimizing performance of GPU-accelerated enterprise Apache Spark workloads.
• Develop advanced algorithms and adaptive systems to continuously improve Spark workload performance on GPUs.
• Create AI-based agents and tools to aid in system issue resolution and application optimization.
• Collaborate with partners and customers on deploying complex ML solutions in diverse environments.
• Maintain expertise in the latest ML systems and algorithm advancements.
• Provide technical mentorship and leadership in data science and machine learning to the engineering team.

Requirements

• BS, MS, or PhD or equivalent experience in Machine Learning, Data Science, Computer Science, or a related field.
• 12+ years of professional experience in designing, implementing, and productionizing high-quality ML/DL solutions.
• 3+ years of experience as a technical lead in ML model development.
• 2+ years of proven hands-on experience with large-scale data processing platforms like Apache Spark.
• Demonstrated ability to use modern tooling and sound techniques for ML model development, deployment, and maintenance.
• Excellent programming skills in Python and Python data science libraries (numpy, pandas, scikit-learn, scipy, pytorch, tensorflow).
• Deep experience with sophisticated ML methodologies, including LLM/GenAI, reinforcement learning, and adaptive, on-line ML systems.
• Strong expertise in feature engineering, feature importance assessment, and developing boosted tree model solutions (e.g., XGBoost).

Benefits

• Equity
• Health insurance
• Base salary range: 224,000 USD - 356,500 USD