
Senior Coding, ML & Security Experts - AI Benchmarking
Listing checked November 7, 2026 · pay as published by Braintrust
Key details
- Pay
- $1,350 fixed
- Location
- Remote · United States; Canada; Argentina +22 moreArgentina, Australia, Austria, Belgium, Canada +20 more
- Job type
- contract
- Openings
- 100
- Posted
- November 7, 2026
Overview
Outlier wants senior practitioners in software engineering, machine learning, or cybersecurity to build benchmark tasks that today's strongest AI agents cannot solve. You pull hard, meaningful problems from your own field and shape them into self-contained challenges an agent must work through from a Linux terminal. Every task gets run against three leading models, three attempts per model, and only survives if the models fail it. The engagement is contract, remote, and paid per accepted task.
What You'll Do6
- Find difficult, worthwhile problems inside your own area of expertise.
- Turn each problem into a task an AI agent can attempt inside a Linux terminal.
- Design challenges sharp enough to expose where advanced agents break down.
- Test each task against three leading models, three attempts apiece, and confirm the models fail it.
- Rework or drop any task that the models solve too easily.
- Submit finished tasks that hold up under grading and repeated evaluation.
Requirements6
- Eight or more years of professional experience in coding, machine learning, or cybersecurity.
- A completed master's degree or higher. A finished bachelor's degree plus 10 or more years of relevant experience and a publication also qualifies.
- Degrees must be complete, and you cannot be enrolled in a degree program right now.
- At least one academic or professional publication.
- Practical comfort working inside a Linux terminal.
- Residence in Argentina, Mexico, Puerto Rico, New Zealand, Canada, the United Kingdom, the United States, Australia, Peru, Colombia, Austria, Ireland, Spain, Luxembourg, Italy, France, Germany, Belgium, the Netherlands, Sweden, Finland, Singapore, Denmark, Norway, or Switzerland.
Who Should Apply
Senior engineers, ML researchers, and security specialists with 8+ years of hands-on work, a completed graduate degree, and at least one publication. You need to spot genuinely hard problems in your field and translate them into terminal-based tasks. Comfort in a Linux shell is a must.
Salary Insight
Outlier pays up to $1,350 per task, and rates can go higher based on task quality and how many tasks you submit. Budget six to seven hours for an average task, with the first few running longer while you learn the process.
Pay and demand for AI Training roles
Market estimates from live NearSkill listings — separate from this job's published terms.
This role · as published
$1,350 fixed
Taken from the listing itself.
Typical pay · estimated
$70/hour
Median of 1,346 similar roles that publish pay · 302 publish only a top rate; those count at the rate they gave
Most AI Training roles pay $40–$95 per hour.
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 1,584
- Listed in last 30 days
- 813
- Remote
- 96%
Hiring most right now: micro1 (430) · SME Careers (254) · Mercor (232)
Most requested skills · share of roles
- ai evaluation13%
- llm evaluation12%
- python11%
- data annotation11%
Figures from AI Training roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesRequired skills
How applying works
1. Pre-screen on NearSkill
A quick application on NearSkill checks your fit against this role.
2. Finish on the employer’s site
We hand you to Braintrust’s official application page to complete the form.
3. Wait for a response
The employer’s hiring team reviews applications and gets back to you.
Similar jobs
Open roles that match this listing's skills and level.
BraintrustSenior Systems & Hardware Experts - AI Benchmarking
TuringSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
BraintrustSenior Software Engineer - AI Code Quality
TuringAI Benchmark & Red-Teaming Specialist
TuringSenior Software Engineer - AI Coding Agent Evaluation
AfterQuery
