About this role
- Design, implement, and maintain comprehensive evaluation protocols for large language models, including both automated and human-in-the-loop assessments.
- Develop and curate high-quality datasets and benchmarks to measure model performance, safety, fairness, and robustness across a variety of tasks and modalities.
- Analyze model outputs to identify strengths, weaknesses, and failure modes, and provide actionable insights to research and engineering teams.
- Collaborate with researchers, engineers, and cross-functional partners to define evaluation goals, communicate findings, and drive improvements in model quality.
- Develop tools and infrastructure to streamline and scale evaluation processes, including dashboards, annotation platforms, and reporting systems.
- Stay up-to-date with the latest research in LLM evaluation, benchmarking, and responsible AI, and incorporate best practices into Meta’s workflows.
- Disseminate evaluation results through internal reports, presentations, and, when appropriate, external publications.
- Contribute to the development of evaluation methodologies that can be applied to Meta product development and deployment.
- Currently has or is in the process of obtaining a Ph.D. degree in Computer Science, Artificial Intelligence, Generative AI, or a relevant technical field
- Must obtain work authorization in the country of employment at the time of hire and maintain ongoing work authorization during employment
- Experience with Python, C++, C, Java or other related languages
- Experience building systems based on machine learning and/or deep learning methods
- Intent to return to the degree program after the completion of the internship/co-op
- Proven track record of achieving significant results as demonstrated by grants, fellowships, patents, as well as first-authored publications at leading workshops or conferences such as NeurIPS, ICML, ICLR, ACL, EMNLP, CVPR, ICCV, ECCV, or similar
- Experience working and communicating cross functionally in a team environment
- Experience in advancing AI techniques, including core contributions to open source libraries and frameworks in computer vision
- Publications or experience in machine learning, AI, computer vision, optimization, computer science, statistics, applied mathematics, or data science
- Experience solving analytical problems using quantitative approaches
- Experience setting up ML experiments and analyzing their results
- Experience manipulating and analyzing complex, large scale, high-dimensionality data from varying sources
- Experience in utilizing theoretical and empirical research to solve problems
- Experience with deep learning frameworks
$7,650/month to $12,134/month + benefits
Other facts
About Meta
Meta's mission is to build the future of human connection and the technology that makes it possible.
Our technologies help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology.
To help create a safe and respectful online space, we encourage constructive conversations on this page. Please note the following:
• Start with an open mind. Whether you agree or disagree, engage with empathy.
• Comments violating our Community Standards will be removed or hidden. Please treat everybody with respect.
• Keep it constructive. Use your interactions here to learn about and grow your understanding of others.
• Our moderators are here to uphold these guidelines for the benefit of everyone, every day.
• If you are seeking support for issues related to your Facebook account, please reference our Help Center (https://www.facebook.com/help) or Help Community (https://www.facebook.com/help/community).
For a full listing of our jobs, visit https://www.metacareers.com
Meta's mission is to build the future of human connection and the technology that makes it possible.
Our technologies help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology.
To help create a safe and respectful online space, we encourage constructive conversations on this page. Please note the following:
• Start with an open mind. Whether you agree or disagree, engage with empathy.
• Comments violating our Community Standards will be removed or hidden. Please treat everybody with respect.
• Keep it constructive. Use your interactions here to learn about and grow your understanding of others.
• Our moderators are here to uphold these guidelines for the benefit of everyone, every day.
• If you are seeking support for issues related to your Facebook account, please reference our Help Center (https://www.facebook.com/help) or Help Community (https://www.facebook.com/help/community).
For a full listing of our jobs, visit https://www.metacareers.com
What you'll do
- Design and implement evaluation protocols for large language models, including both automated and human assessments. Collaborate with cross-functional teams to define evaluation goals and improve model quality.
Ready to join Meta?
Take the next step in your career journey