Wrocław, Lower Silesian Voivodeship, Poland · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Buenos Aires, Argentina · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Japan · Remote Solely
$50/hr
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Czechia · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Lithuania · Remote Solely
$40/hr
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Latvia · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Estonia · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Pune, Maharashtra, India · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Ahmedabad, Gujarat, India · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Colombia · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Uruguay · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Peru · Remote Solely
$40/hr
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Costa Rica · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Chile · Remote Solely
$40/hr
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Mexico · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Belo Horizonte, Minas Gerais, Brazil · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Rio de Janeiro, Southeast, Brazil · Remote Solely
$30/hr
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
São Paulo, São Paulo, Brazil · Remote Solely
$30/hr
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Brazil · Remote Solely
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
South Korea · Remote Solely
$50/hr
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving…
Skills: Python, FastAPI, JavaScript, TypeScript, React
Wrocław, Lower Silesian Voivodeship, Poland • Remote Solely
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
You will design challenging tasks and evaluation criteria within simulated developer environments to test AI coding agents. Additionally, you will write tests to verify agent solutions and iterate on tasks based on quality assurance feedback.
Candidates must have at least 5 years of software development experience and proficiency in the core tech stack including Python, JavaScript, and Docker. A Master's degree in a relevant technical field is required, or a Bachelor's degree with 5 years of professional experience.
Please submit your CV in English and indicate your level of English proficiency.
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.
We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.
You'll create challenging tasks and evaluation criteria within realistic simulated environments:
What this is NOT:
What we look for:
Why this is hard:
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.
Requirements and benefits
Educational qualifications
Academic and/or Professional Experience
Candidates should have a minimum of 3 years of professional experience in related roles or domain - specifically for QA-automation/testing or cybersecurity roles
How it works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Compensation:
Paid per accepted task. Your rate depends on the qualification tier you reach and how efficiently you complete tasks — up to the equivalent of $40/hr. Because payment is per task, a faster pace raises your effective hourly rate.
Why this freelance opportunity might be a great fit for you?
Be the “I” in AI with Mindrift. Work from anywhere on groundbreaking AI projects
Mindrift (mindrift.ai) is the expert-sourcing platform built by Toloka, a Nebius Group (Nasdaq: NBIS) company and a global leader in data services for AI since 2014. Mindrift’s CEO and co-founder, Olga Megorskaya, launched the platform to connect domain experts — from software engineers and physicians to linguists and lawyers — with AI training projects for the world's leading technology companies. Through Mindrift, over 10,000 professionals across 50+ domains and 15+ languages contribute to supervised fine-tuning, reinforcement learning from human feedback (RLHF), model evaluation, and red-teaming for frontier AI models developed by leading global technology companies. Operated by Toloka from Amsterdam, Netherlands, Mindrift offers flexible, fully remote, project-based opportunities — empowering experts to shape the future of generative AI, one task at a time.
Be the “I” in AI with Mindrift. Work from anywhere on groundbreaking AI projects
Mindrift (mindrift.ai) is the expert-sourcing platform built by Toloka, a Nebius Group (Nasdaq: NBIS) company and a global leader in data services for AI since 2014. Mindrift’s CEO and co-founder, Olga Megorskaya, launched the platform to connect domain experts — from software engineers and physicians to linguists and lawyers — with AI training projects for the world's leading technology companies. Through Mindrift, over 10,000 professionals across 50+ domains and 15+ languages contribute to supervised fine-tuning, reinforcement learning from human feedback (RLHF), model evaluation, and red-teaming for frontier AI models developed by leading global technology companies. Operated by Toloka from Amsterdam, Netherlands, Mindrift offers flexible, fully remote, project-based opportunities — empowering experts to shape the future of generative AI, one task at a time.
