Research Scientist

About AfterQuery

AfterQuery builds the training data and evaluation infrastructure that frontier AI labs use to improve their models. They work with the world's leading labs to design high-signal datasets and run rigorous evaluations that go beyond static benchmarks. Post Series A, $30M raised at a $300M valuation, backed by Y Combinator and BoxGroup, with investors from Lightspeed, Warburg Pincus, Silver Lake, and Index Ventures, and a founding team from Jane Street, Meta, Citadel Securities, Google, Goldman Sachs, and Stanford AI Lab. Small, early team where individual contributors have a direct impact on how the next generation of models learn and improves.

About the Role

AfterQuery is hiring Research Scientists to design the datasets and evaluation frameworks that shape how frontier models are trained and measured. You will work directly with research teams at top AI labs, experiment with data collection strategies, diagnose model failure modes, and develop the metrics that determine whether a model is actually getting better.

This is hands-on, high-leverage work. You go from hypothesis to live experiment quickly, and your output directly influences model training runs at scale. The ideal candidate is an undergraduate or master's researcher who has not yet done a PhD, is obsessed with how data structure and quality drive model behavior, and favors building over theorizing.

Compensation is performance-driven. If you design the next HLE-level benchmark, you can make over $1.5M in your first year. Average performance on this work generates around $400K in cash per year.

Key Responsibilities

Design data shapes that expose meaningful model failure modes across domains like finance, code, and enterprise workflows

Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines

Model annotator behavior and run experiments to improve different model capabilities

Develop quantitative frameworks for measuring dataset quality, diversity, and downstream impact on model alignment and capability

Partner with lab research teams to translate their training objectives into concrete data and evaluation specifications

Requirements

Must-Have

Undergrad or master's research background, strong preference for candidates who have not yet done a PhD

Prior experience with evals or benchmarking, either at a competing data company, through academic research on model evaluation, or via internship at an RL environment company

Genuine obsession with how data structure, selection, and quality drive model behavior

Ability to design lightweight experiments, move fast, and extract actionable insights from messy results

Strong quantitative instincts and familiarity with LLM training pipelines, RLHF, and RLVR, or evaluation methodology

Comfort working across domains, including finance, software engineering, and policy

SWE experience is a significant plus; technical depth matters here

Bias toward building over theorizing

Nice-to-Have

Prior internship or work experience at an RL environment company, AI safety org, or benchmarking organization (METR, Artificial Analysis, Hellclimb, Idler AI, or equivalent)

Academic research on model evaluation or benchmarking

Experience with AI for data or spreadsheet automation companies

$110, 000 to $150, 000 base + up to 100% bonus + $100, 000 to $150, 000 equity over four years