Software Engineer - AI Evaluation (Brazil/Argentina)
Listing checked October 6, 2026 · pay as published by Terac
Overview
Terac runs a paid study on how well coding tasks measure an AI agent's software abilities. The project focuses on AI evaluation environments, a suite of programming challenges that mirror real engineering work. You join a live session, share your screen, and inspect proposed tasks plus the evaluation harnesses behind them. Your code review and technical discussion help the team judge if each task is realistic, accurate, and calibrated for testing agents.
What You'll Do6
- 1Review programming tasks that test AI agents.
- 2Check whether each challenge reflects realistic software work.
- 3Assess the structure and accuracy of evaluation harnesses.
- 4Talk through your logic as you verify code validity.
- 5Flag flaws in the environments during a screen share.
- 6Suggest changes that raise task complexity and realism.
Requirements6
- 1Professional background as a software engineer or developer.
- 2Residence in Brazil or Argentina.
- 3Hands-on work building or verifying coding tasks.
- 4Hands-on work with test environments or harnesses.
- 5Ability to discuss technical architectures in detail.
- 6Ability to discuss evaluation frameworks in detail.
Who Should Apply
Engineers based in Brazil or Argentina with professional software development experience fit this study best. Terac welcomes backend developers, full-stack engineers, and QA automation specialists who have built, reviewed, or tested evaluation harnesses and programming tasks. The work does not suit developers outside those countries or people without hands-on code review or test environment experience. Applications often score low when the candidate cannot explain their reasoning during a live code walkthrough. Another common miss: claiming familiarity with AI evaluation but showing no concrete examples of tasks or harnesses they have verified.
Salary Insight
The listing does not share a rate or salary range. Terac calls the study paid, and the team discusses compensation during the application or screening step.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$75/hour
This role
Pay not disclosed
Most Machine Learning & AI roles pay $50–$100 per hour.
Based on 601 similar roles that publish pay · 172 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 696
- Listed in last 30 days
- 481
- Remote
- 96%
Hiring most right now: micro1 (240) · SME Careers (145) · Mercor (87)
Most requested skills · share of roles
- python18%
- ai training16%
- llm evaluation16%
- trainer feedback11%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
Undisclosed by Terac
Comparable roles pay
601 roles$50 – $100/ hr
Not this role’s pay. Terac has not published a range.
Confirmed during Terac profile screening.
Required Skills
Application Tip
Name the evaluation harnesses or test environments you have built or verified, and add one concrete example of a coding task you reviewed and improved. State your country and years of software engineering experience near the top so Terac can confirm your fit for the Brazil/Argentina cohort.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
Terac
VerifiedSoftware Engineer - AI Agent Code Review
Terac pays software engineers to review coding tasks and evaluation harnesses used for benchmarking AI agents. You will inspect programming challenges, determine whether they mirror real software engineering problems, and verify logic, test cases, and environment structure. The remote session combines a guided conversation with screen-sharing exercises, so you explain your reasoning as you analyze code. Engineers in the USA, Canada, and France can take part.
Terac
VerifiedSoftware Engineer - AI Coding Evaluation (South Asia)
Terac wants experienced software engineers across South Asia to test the quality of environments used to score AI agents on coding work. You will join an AI-moderated session, inspect proposed programming tasks, and judge how well they match real engineering demands. Your notes will cover task structure, difficulty, and realism, plus where the coding evaluation harnesses need fixes. The project is remote, paid, and open to engineers in India and the Philippines.
Handshake AI
VerifiedSoftware Engineer - AI Evaluation
Handshake AI seeks practicing software engineers for an ongoing part-time contract. You will create expert-level problems rooted in system design, software architecture, and real implementation work, then judge AI answers for technical accuracy and practical feasibility. The role runs remote and asynchronous, so you choose your own hours with no minimum. Your structured written feedback helps models reason more like engineers who build production systems.
Terac
VerifiedSoftware Engineer - AI Coding Benchmarks & RL
Terac builds coding tasks that train and test frontier AI coding agents. Experienced software engineers design realistic full-stack engineering problems, write tests that judge correctness, and push beyond the UI into services, APIs, and backend systems. The codebase for these tasks is TypeScript with React, Node, SQLite, and Playwright, and it grows as each task builds on the last. Contractors work 10 to 20+ hours per week, paid each week at $45 to $75 per hour, with US and Canada timezone overlap preferred. You can use AI tools as much as you like while writing code and tests.
Turing
VerifiedSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
This contract role places an experienced software engineer inside an evaluation workflow for frontier AI models. You will review AI-generated code for correctness, efficiency, and maintainability, validate solutions against real engineering tasks, and debug failures across different environments. You will also help build and refine evaluation datasets, benchmarks, and grading rubrics. The assignment runs for one month, requires at least 4 hours per day and 20 hours per week with a 4-hour overlap with PST, and is open only to candidates in the US. Work is fully remote.
Micro1
VerifiedAI Evaluation Specialist
As an AI Evaluation Specialist, you'll help train next-generation AI systems by designing and executing hands-on evaluation tasks. Your insights will directly shape how models learn, reason, and perform on practical computer-based workflows. This is a fully remote contract role where meticulous observation and clear documentation are key.