Research Scientist

Vals AI is at the forefront of developing benchmarks and evaluation methodologies for large language models. You will help shape how the most advanced AI systems are tested and validated before deployment in enterprise environments, working closely with engineering, AI labs, and enterprise customers.

What youll do

• Evaluate new AI models as they're released (such as DeepSeek, Gemini, etc.)
• Create new benchmarks from scratch, including hiring labelers, constructing datasets, and writing white papers
• Improve underlying methods for auto-evaluation of generated text
• Work closely with engineering to implement and scale evaluation methodologies
• Collaborate with top AI labs and enterprise customers to understand evaluation needs

What Vals AI is looking for

• 3+ years of AI/ML research and evaluation experience, with a track record in benchmarking or language-model research
• Proficiency in Python; experience with Django or other Python-based HTTP servers (e.g., Flask)
• Experience with deep learning frameworks (PyTorch, TensorFlow) and familiarity with LLM infrastructure and language modeling
• Experience building new benchmarks and evaluation methodologies; touchpoints with NLP or benchmarking research and published papers
• Experience working at AI startups or research labs (e.g., Anthropic, DeepMind) or similar enterprise-facing research environments
• Strong CS fundamentals; ability to translate research insights into scalable, practical evaluation methods
• Excellent collaboration skills to work with engineering on implementing and scaling evaluation methodologies and to understand customer needs