
Senior Systems & Hardware Experts - AI Benchmarking
Listing checked November 7, 2026 · pay as published by Braintrust
Key details
- Pay
- $1,350 fixed
- Location
- Remote · United States; Canada; Argentina +22 moreArgentina, Australia, Austria, Belgium, Canada +20 more
- Job type
- contract
- Openings
- 100
- Posted
- November 7, 2026
Overview
Outlier asks seasoned systems and hardware engineers to build the hard problems that top AI agents still fail at. You bring the tough questions from your own field and shape them into tasks an agent must work through from scratch inside a Linux terminal. Every task goes through three leading models with three attempts apiece, and it only counts if the models mostly fail. The engagement runs on a contract basis for roughly the next few weeks, and the work sits fully remote.
What You'll Do6
- Spot genuinely difficult problems inside your systems or hardware specialty, the kind that resist easy answers.
- Reshape each problem into a self-contained task that a model can attempt using only a Linux terminal.
- Write the task so a capable AI agent must work it out from a blank starting point rather than pattern-match.
- Pressure-test your designs against three leading models, which each get three tries, and refine until most attempts break down.
- Document the expected path to a solution so graders can judge outcomes against your intent.
- Complete two onboarding courses plus a graded assessment, and attend a live webinar before tasking begins.
Requirements6
- Nine or more years of hands-on professional work in systems engineering or hardware engineering.
- A completed master's degree or doctorate, with transcripts closed out and no current enrollment in any degree program.
- At least one publication, academic or professional, tied to your field.
- Comfort navigating a Linux terminal as a daily working tool rather than a rare visit.
- Current residence in one of the eligible countries listed for this project, including the United States, Canada, the United Kingdom, Germany, France, Australia, New Zealand and others across Europe, Latin America and Asia.
- Bachelor's degree holders can qualify with ten or more years of relevant experience plus a publication.
Who Should Apply
Systems and hardware engineers with close to a decade or more of practice who enjoy framing messy, real-world problems into precise challenges. A publication record and steady Linux fluency matter here. You should hold a finished graduate degree and live in one of the listed countries.
Salary Insight
Each accepted task pays up to $1,350, and the rate can climb higher depending on task quality and how many you submit. Budget roughly six to seven hours per task, with the first ones running longer while you learn the format.
Pay and demand for Hardware & Embedded Systems roles
Market estimates from live NearSkill listings — separate from this job's published terms.
This role · as published
$1,350 fixed
Taken from the listing itself.
Typical pay · estimated
$70/hour
Median of 1,359 similar roles that publish pay · 303 publish only a top rate; those count at the rate they gave
Most Hardware & Embedded Systems roles pay $40–$95 per hour.
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 1,586
- Listed in last 30 days
- 821
- Remote
- 97%
Hiring most right now: micro1 (434) · SME Careers (255) · Mercor (237)
Most requested skills · share of roles
- ai evaluation13%
- llm evaluation12%
- python11%
- data annotation11%
Figures from Hardware & Embedded Systems roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesRequired skills
How applying works
1. Pre-screen on NearSkill
A quick application on NearSkill checks your fit against this role.
2. Finish on the employer’s site
We hand you to Braintrust’s official application page to complete the form.
3. Wait for a response
The employer’s hiring team reviews applications and gets back to you.
Similar jobs
Open roles that match this listing's skills and level.
BraintrustSenior Coding, ML & Security Experts - AI Benchmarking
TuringAI Benchmark & Red-Teaming Specialist
Handshake AISoftware Engineer - AI Evaluation
TuringSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
Handshake AISoftware Engineer
Handshake AI
