Software Engineer - AI Code Evaluation (Remote)
Listing checked September 18, 2026 · pay as published by Handshake AI
Overview
Handshake AI brings experienced software engineers into part-time contract projects that judge AI-generated code and technical explanations. Contributors review programming outputs, score them against engineering standards, and write structured critiques that help models handle system design and coding tasks better. The program runs year-round and places people on projects as client needs arise, so you can keep your main job while taking assignments. Most contributors log about 5 to 20 hours per week during active projects, and the schedule stays flexible. Pay reaches $65.00 per hour.
What You'll Do8
- 1Review AI-written code for logic errors, style problems, security gaps, and mismatches with the original prompt.
- 2Compare model answers with accepted practices in software engineering, then note where an answer falls short.
- 3Write clear critiques that explain why a snippet fails, what a better solution looks like, and which tradeoffs matter.
- 4Assess technical explanations of programming concepts, data structures, APIs, and system design.
- 5Mark unsafe, misleading, or invented claims in model output.
- 6Apply detailed client guidelines to every review so scores stay consistent.
- 7Complete tasks on your own schedule and submit feedback by agreed project deadlines.
- 8Debug or reason through production-level code when a project requires deeper verification.
Requirements6
- 1Bring 4 or more years of professional software engineering experience, with internships excluded.
- 2Show hands-on coding skill in at least one major language such as Python, Java, C++, JavaScript, or Go.
- 3Point to a track record of writing, reviewing, and debugging code that runs in production.
- 4Work through detailed technical guidelines on your own.
- 5Communicate in writing with close attention to detail.
- 6Check F-1 work authorization: CPT or OPT may qualify you, while STEM OPT does not apply.
Who Should Apply
Active software engineers with four or more years of professional experience fit this work best, including backend, frontend, full-stack, systems, or application developers who still write and review production code. The role suits people who can explain technical flaws in writing and follow a detailed rubric without hand-holding. Candidates who need a full-time job with benefits, or who only have internship experience, will find this less suitable. Applications often score low when the candidate cannot show four years of non-internship engineering work or when their written feedback stays vague and undocumented. F-1 students who need STEM OPT should not apply because the program does not support that authorization.
Salary Insight
Pay goes up to $65.00 per hour for this independent contractor role. That rate sits at the high end for part-time AI evaluation projects and often matches engineers with several years of production coding experience. Handshake AI pays by the hour, not by project, and the listing does not mention benefits or a fixed weekly minimum.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$74/hour
This role
up to $65/hr
Most Machine Learning & AI roles pay $50–$96 per hour. This role's pay falls inside that range.
Based on 656 similar roles that publish pay · 168 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 733
- Listed in last 30 days
- 382
- Remote
- 97%
Hiring most right now: micro1 (295) · SME Careers (139) · Mercor (91)
Most requested skills · share of roles
- llm evaluation16%
- python15%
- ai training15%
- trainer feedback11%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
Up to $65/hr
Required Skills
Application Tip
Lead your application with four or more years of professional engineering work, and name the languages you use in production, such as Python, Java, or Go. Add one short example of a code review or debugging fix you wrote in plain language, because structured written feedback and attention to detail drive placement decisions.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
Turing
VerifiedSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
This contract role places an experienced software engineer inside an evaluation workflow for frontier AI models. You will review AI-generated code for correctness, efficiency, and maintainability, validate solutions against real engineering tasks, and debug failures across different environments. You will also help build and refine evaluation datasets, benchmarks, and grading rubrics. The assignment runs for one month, requires at least 4 hours per day and 20 hours per week with a 4-hour overlap with PST, and is open only to candidates in the US. Work is fully remote.
Micro1
VerifiedSenior Software Engineer for AI Code Evaluation and Model Review
micro1 is hiring senior software engineers to work with a leading AI lab. Your main tasks involve evaluating AI-generated code for correctness and design, writing challenging coding problems with reference solutions and test cases, and reviewing model reasoning on architecture, debugging, and system design. You will provide detailed technical feedback to help improve model performance on real-world software tasks. This is a remote contractor role with a pay range of $245 to $280 per hour.
Turing
VerifiedSenior Software Engineer - AI Coding Agent Evaluation
Turing seeks experienced software engineers to assess how well AI coding agents handle real repositories. You will treat the agent like a colleague whose pull requests you review: checking whether it read the task, picked a sound design, and produced working code. The work spans Python, TypeScript/JavaScript, and Go codebases, plus the evaluation rubrics and data pipelines that sharpen model behavior. Researchers rely on your written judgment to turn one-off observations into repeatable quality signals. This is a freelance contract, roughly three months long, with 40 hours per week preferred.
Micro1
VerifiedAI Engineer
micro1 hires software engineers to build evaluation environments that push AI systems into realistic engineering workflows. As a remote contractor, you set your own schedule at roughly 15 hours per week and focus on one thing: creating reinforcement learning environments that force an AI model to use Model Context Protocol (MCP) tools while solving bugs, implementing features, or refactoring code. Each environment needs deterministic verification and a golden reference solution so results are measurable and repeatable. Prior AI experience isn't required; your coding background and judgment are what the project needs.
AfterQuery
VerifiedStudent Software Engineering Expert - Remote Contract
AfterQuery builds training content for AI systems, and this contract asks college software engineers to write and review coding challenges. You will create questions that push models through algorithm design, code debugging, system architecture, API design, testing strategies and similar topics. The work runs remote and asynchronous at 10 hours per week, so it can fit around a class schedule. A YC-backed startup, AfterQuery uses this content to help AI models handle real software development tasks with technical accuracy.
Micro1
VerifiedAI Software Engineering Domain Expert
Remote, part-time contract work with micro1 where you apply deep software engineering knowledge to help train next-generation AI systems. You’ll review and polish AI-generated technical content, refine prompts, and judge model outputs using rubrics to ensure accuracy and quality. The role centers on drafting and editing technical docs, architecture plans, RFCs, and design specs that feed AI training, plus independent research and data annotation. Strong writing and a solid engineering background are essential, and you’ll collaborate asynchronously with project leads to meet deliverables.