Software Engineer - AI Coding Evaluation (South Asia)
Listing checked October 6, 2026 · pay as published by Terac
Overview
Terac wants experienced software engineers across South Asia to test the quality of environments used to score AI agents on coding work. You will join an AI-moderated session, inspect proposed programming tasks, and judge how well they match real engineering demands. Your notes will cover task structure, difficulty, and realism, plus where the coding evaluation harnesses need fixes. The project is remote, paid, and open to engineers in India and the Philippines.
What You'll Do5
- 1Examine proposed coding tasks for realism against the kind of work professional engineers handle.
- 2Inspect the programming environments paired with those tasks for gaps, mismatches, or missing context.
- 3Assess the structure and difficulty of each evaluation harness and call out where it fails to measure agent skill.
- 4Walk the moderator through your own method for verifying complex code challenges from start to finish.
- 5Recommend concrete changes that make the test suites stronger, clearer, and harder to game.
Requirements6
- 1Professional software engineer based in India or the Philippines.
- 2Hands-on work building or reviewing realistic programming tasks in a professional setting.
- 3Familiarity with coding evaluation harnesses, test automation, or both.
- 4Experience across backend, full-stack, or test automation engineering.
- 5A track record of verifying complex code challenges, not just solving them.
- 6Comfort discussing technical workflows and decisions in a live interview.
Who Should Apply
The strongest candidates are working software engineers in India or the Philippines who have built or reviewed production code and can explain what makes a coding task realistic. Backend developers, full-stack engineers, and test automation specialists often fit because they already judge code quality, task difficulty, and test coverage. The role is a poor match for engineers whose experience is limited to academic exercises or puzzle sites, since Terac needs feedback from people who have shipped and reviewed real systems. Candidates also score low when they cannot describe their verification process in detail or when they treat every coding challenge as either good or bad without explaining why. If you have never inspected an evaluation harness or worked with automated tests, expect this study to be a stretch.
Salary Insight
The listing does not publish a rate. Terac calls the work a paid study, so compensation details should arrive during screening or session scheduling. For comparable AI evaluation sessions, pay often depends on seniority and session length, yet no number appears here.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$75/hour
This role
Pay not disclosed
Most Machine Learning & AI roles pay $50–$100 per hour.
Based on 629 similar roles that publish pay · 176 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 732
- Listed in last 30 days
- 515
- Remote
- 95%
Hiring most right now: micro1 (247) · SME Careers (145) · Mercor (94)
Most requested skills · share of roles
- python17%
- ai training16%
- llm evaluation15%
- trainer feedback11%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
Undisclosed by Terac
Comparable roles pay
629 roles$50 – $100/ hr
Not this role’s pay. Terac has not published a range.
Confirmed during Terac profile screening.
Required Skills
Application Tip
Show your fit by naming the evaluation harnesses and test automation tools you have used, then describe one coding task you reviewed that missed the mark. Mention your backend, full-stack, or test automation background and quantify the codebases you have verified, because Terac screens for hands-on review experience, not just coding ability.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
Terac
VerifiedSoftware Engineer - AI Evaluation (Brazil/Argentina)
Terac runs a paid study on how well coding tasks measure an AI agent's software abilities. The project focuses on AI evaluation environments, a suite of programming challenges that mirror real engineering work. You join a live session, share your screen, and inspect proposed tasks plus the evaluation harnesses behind them. Your code review and technical discussion help the team judge if each task is realistic, accurate, and calibrated for testing agents.
Terac
VerifiedSoftware Engineer - AI Agent Code Review
Terac pays software engineers to review coding tasks and evaluation harnesses used for benchmarking AI agents. You will inspect programming challenges, determine whether they mirror real software engineering problems, and verify logic, test cases, and environment structure. The remote session combines a guided conversation with screen-sharing exercises, so you explain your reasoning as you analyze code. Engineers in the USA, Canada, and France can take part.
Handshake AI
VerifiedSoftware Engineer - AI Evaluation
Handshake AI seeks practicing software engineers for an ongoing part-time contract. You will create expert-level problems rooted in system design, software architecture, and real implementation work, then judge AI answers for technical accuracy and practical feasibility. The role runs remote and asynchronous, so you choose your own hours with no minimum. Your structured written feedback helps models reason more like engineers who build production systems.
Turing
VerifiedSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
This contract role places an experienced software engineer inside an evaluation workflow for frontier AI models. You will review AI-generated code for correctness, efficiency, and maintainability, validate solutions against real engineering tasks, and debug failures across different environments. You will also help build and refine evaluation datasets, benchmarks, and grading rubrics. The assignment runs for one month, requires at least 4 hours per day and 20 hours per week with a 4-hour overlap with PST, and is open only to candidates in the US. Work is fully remote.
Terac
VerifiedSoftware Engineer - AI Coding Benchmarks & RL
Terac builds coding tasks that train and test frontier AI coding agents. Experienced software engineers design realistic full-stack engineering problems, write tests that judge correctness, and push beyond the UI into services, APIs, and backend systems. The codebase for these tasks is TypeScript with React, Node, SQLite, and Playwright, and it grows as each task builds on the last. Contractors work 10 to 20+ hours per week, paid each week at $45 to $75 per hour, with US and Canada timezone overlap preferred. You can use AI tools as much as you like while writing code and tests.
Micro1
VerifiedSenior Software Engineer for AI Code Evaluation and Model Review
micro1 is hiring senior software engineers to work with a leading AI lab. Your main tasks involve evaluating AI-generated code for correctness and design, writing challenging coding problems with reference solutions and test cases, and reviewing model reasoning on architecture, debugging, and system design. You will provide detailed technical feedback to help improve model performance on real-world software tasks. This is a remote contractor role with a pay range of $245 to $280 per hour.