Software Engineer - AI Agent Code Review
Listing checked October 6, 2026 · pay as published by Terac
Overview
Terac pays software engineers to review coding tasks and evaluation harnesses used for benchmarking AI agents. You will inspect programming challenges, determine whether they mirror real software engineering problems, and verify logic, test cases, and environment structure. The remote session combines a guided conversation with screen-sharing exercises, so you explain your reasoning as you analyze code. Engineers in the USA, Canada, and France can take part.
What You'll Do6
- 1Examine programming tasks to judge their accuracy and difficulty.
- 2Check the logic inside supplied evaluation harnesses.
- 3Confirm that test cases match the intended challenge and produce correct outcomes.
- 4Determine whether each coding environment measures software engineering skills in a useful way.
- 5Talk through your analysis of complex code structures while sharing your screen.
- 6Explain how you reach conclusions about code quality and task design during a guided conversation.
Requirements6
- 1Current professional software engineering experience.
- 2Track record building or maintaining test suites and evaluation harnesses.
- 3Ability to review code and describe technical ideas out loud.
- 4Knowledge of complex software architecture used in production systems.
- 5Comfort with screen-sharing and guided technical discussion.
- 6Residency in the United States, Canada, or France.
Who Should Apply
Practicing software engineers who build and test complex systems will fit this study well. Backend engineers, full-stack developers, machine learning engineers, and software architects with hands-on test suite or evaluation harness work are the target. The role suits someone who can inspect code and narrate their reasoning during a screen share. Candidates without active coding work, or those who only write application code without testing or evaluation experience, tend to score lower. A common rejection reason is treating code review as a silent exercise rather than explaining the logic, test cases, and architecture choices out loud.
Salary Insight
The listing does not state a pay rate. Terac labels the work as paid, so ask about compensation during the screening or scheduling step.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$75/hour
This role
Pay not disclosed
Most Machine Learning & AI roles pay $50–$100 per hour.
Based on 586 similar roles that publish pay · 170 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 676
- Listed in last 30 days
- 471
- Remote
- 96%
Hiring most right now: micro1 (234) · SME Careers (143) · Mercor (83)
Most requested skills · share of roles
- python18%
- ai training17%
- llm evaluation17%
- trainer feedback12%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
Undisclosed by Terac
Comparable roles pay
586 roles$50 – $100/ hr
Not this role’s pay. Terac has not published a range.
Confirmed during Terac profile screening.
Required Skills
Application Tip
When you apply, name the test frameworks, evaluation harnesses, or code review systems you have built and quantify the scale, such as number of test cases, services covered, or review volume. Mention your willingness to explain technical reasoning during a screen-share session.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
Terac
VerifiedSoftware Engineer - AI Evaluation (Brazil/Argentina)
Terac runs a paid study on how well coding tasks measure an AI agent's software abilities. The project focuses on AI evaluation environments, a suite of programming challenges that mirror real engineering work. You join a live session, share your screen, and inspect proposed tasks plus the evaluation harnesses behind them. Your code review and technical discussion help the team judge if each task is realistic, accurate, and calibrated for testing agents.
Terac
VerifiedSoftware Engineer - AI Coding Evaluation (South Asia)
Terac wants experienced software engineers across South Asia to test the quality of environments used to score AI agents on coding work. You will join an AI-moderated session, inspect proposed programming tasks, and judge how well they match real engineering demands. Your notes will cover task structure, difficulty, and realism, plus where the coding evaluation harnesses need fixes. The project is remote, paid, and open to engineers in India and the Philippines.
Turing
VerifiedSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
This contract role places an experienced software engineer inside an evaluation workflow for frontier AI models. You will review AI-generated code for correctness, efficiency, and maintainability, validate solutions against real engineering tasks, and debug failures across different environments. You will also help build and refine evaluation datasets, benchmarks, and grading rubrics. The assignment runs for one month, requires at least 4 hours per day and 20 hours per week with a 4-hour overlap with PST, and is open only to candidates in the US. Work is fully remote.
Terac
VerifiedSoftware Engineer - AI Coding Benchmarks & RL
Terac builds coding tasks that train and test frontier AI coding agents. Experienced software engineers design realistic full-stack engineering problems, write tests that judge correctness, and push beyond the UI into services, APIs, and backend systems. The codebase for these tasks is TypeScript with React, Node, SQLite, and Playwright, and it grows as each task builds on the last. Contractors work 10 to 20+ hours per week, paid each week at $45 to $75 per hour, with US and Canada timezone overlap preferred. You can use AI tools as much as you like while writing code and tests.
Handshake AI
VerifiedSoftware Engineer - AI Evaluation
Handshake AI seeks practicing software engineers for an ongoing part-time contract. You will create expert-level problems rooted in system design, software architecture, and real implementation work, then judge AI answers for technical accuracy and practical feasibility. The role runs remote and asynchronous, so you choose your own hours with no minimum. Your structured written feedback helps models reason more like engineers who build production systems.
Micro1
VerifiedSenior Software Engineer for AI Code Evaluation and Model Review
micro1 is hiring senior software engineers to work with a leading AI lab. Your main tasks involve evaluating AI-generated code for correctness and design, writing challenging coding problems with reference solutions and test cases, and reviewing model reasoning on architecture, debugging, and system design. You will provide detailed technical feedback to help improve model performance on real-world software tasks. This is a remote contractor role with a pay range of $245 to $280 per hour.