[AI Research Div.] Foundation Model Evaluation Engineer - 독자 AI 파운데이션 모델 (2년 이상 / 인턴)
Salary not listed • Seoul • Internship
Posted 3mo ago
This job has been open for more than 60 days. It may no longer be actively hiring, so check with the company before putting a lot into an application.
Remote Work Policy
On-site
Employment Type
Internship
Categories
AI Research Engineer
About the job
The Foundation Model Evaluation team is responsible for comprehensively diagnosing our foundation models and global frontier LLMs to provide data-driven direction for model development. Recognizing that benchmarks have diverse measurement axes and scores alone are insufficient to determine a model's practical capabilities, we aim to build an evaluation system that deeply analyzes where models excel and falter, and connects these findings to model improvements. We track model trends by operating both standard benchmarks and proprietary evaluation sets that reflect actual usage contexts. The evaluation results are shared with the model and data teams to directly contribute to the technical roadmap and data strategy.
Responsibilities
- Define the capabilities and targets our models should focus on, guiding the overall direction of model development.
- Design and build new benchmarks that measure a model's practical capabilities without being affected by data contamination.
- Diagnose frontier models and our own models to quickly assess the suitability of new models and support technical decision-making.
- Collaborate closely with data and model teams to identify and communicate weaknesses in data and areas for improvement in model design.
- Develop and manage evaluation pipelines and platforms for stable execution of large-scale evaluations, centralizing results and insights.
- Publish research findings in top-tier academic conferences such as NeurIPS, ICLR, ICML, ACL, and EMNLP in the form of technical reports or papers/workshops.
Requirements
- Master's or Ph.D. in Deep Learning or a related field, or equivalent research experience.
- Experience publishing papers in top-tier AI/ML conferences (NeurIPS, ICLR, ICML, ACL, EMNLP, etc.) on AI model evaluation/analysis or ML/NLP topics.
- Ability to meticulously examine data and derive insights.
- Strong communication skills to clearly document and share complex experimental setups and results.
- Adaptability to new domains.
- Proficiency in business communication.
- No disqualifications for overseas business trips.