Senior Software Engineer – LLM Evaluation (US/Canada/WEU based)
Listing checked August 25, 2026 · pay as published by Turing
Overview
Turing, a San Francisco-based research accelerator, is hiring a contract software engineer to evaluate AI-generated code and build datasets for large language models. You will curate code examples and write precise corrections in Python, JavaScript/ReactJS, C/C++, Rust, plus Java and Go. The role includes scoring model outputs for efficiency, scalability, and reliability, designing verification mechanisms, and partnering with researchers to strengthen enterprise coding solutions. This remote engagement runs one month, offers 10 to 40 flexible hours per week, and accepts candidates in the US, Canada, and Western Europe.
What You'll Do6
- 1Curate code examples and develop precise solutions in Python, JavaScript/ReactJS, C/C++, Java, Rust, and Go for AI training datasets.
- 2Review and correct AI-generated code to improve its efficiency, scalability, and reliability.
- 3Collaborate with researchers and engineers to benchmark AI-driven coding solutions against industry performance standards.
- 4Build software agents that inspect code quality and detect recurring error patterns.
- 5Analyze the software engineering lifecycle from prototyping and API design to production deployment and monitoring, then design experiments to test model capabilities at each stage.
- 6Design automated verification mechanisms that confirm whether a proposed solution correctly solves a given software engineering task.
Requirements4
- 1At least 3 years of professional software engineering experience.
- 2Strong record of building full-stack applications and deploying production-grade software with modern tools.
- 3Deep understanding of software architecture, design, development, debugging, and code quality review.
- 4Excellent written and verbal communication skills for producing clear, structured evaluation rationales.
Who Should Apply
The ideal candidate has at least 3 years of software engineering experience and has built and shipped full-stack applications in production. You should be comfortable reviewing code line by line, writing clear evaluation rationales, and switching between Python, JavaScript/ReactJS, C/C++, Rust, and other languages. This role is less suitable for engineers who need full-time employment or prefer building features over assessing AI output. Rejections often happen when candidates present vague debugging examples or write explanations that lack structure and evidence.
Pay and demand for Software Engineering roles
AggregatedTypical pay
$72/hour
This role
Pay not disclosed
Most Software Engineering roles pay $42–$100 per hour.
Based on 1,086 similar roles that publish pay · 302 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 1,252
- Listed in last 30 days
- 482
- Remote
- 97%
Hiring most right now: micro1 (365) · SME Careers (253) · Mercor (158)
Most requested skills · share of roles
- ai training13%
- llm evaluation11%
- ai evaluation11%
- data annotation9%
Figures from Software Engineering roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
Undisclosed by Turing
Comparable roles pay
1,086 roles$42 – $100/ hr
Not this role’s pay. Turing has not published a range.
Confirmed during Turing profile screening.
Required Skills
Application Tip
Practice a three-minute response about a time you debugged or reviewed AI-generated code, and structure it as problem, diagnosis, fix, and measurable result. Your evaluation rationale will be scored on clarity, so avoid rambling and quantify the outcome with concrete numbers.
How NearSkill helps you apply
1. Check your job match
We compare your resume with the job requirements to help you understand how well your experience fits.
2. Continue to the employer’s website
We’ll direct you to the employer’s application portal, where you can complete the application.
3. Get updates by email
If NearSkill is handling follow-up for this role, we’ll email you when there’s an update on your screening.
Want to check your fit first?
See how your resume matches this role before applying.
Check my job matchSimilar open positions
Explore active roles that match your skills and interests.
Turing
VerifiedSenior Software Engineer – LLM Evaluation
In this contract role, you will build and refine training datasets that help large language models improve their coding skills. You will write, correct, and evaluate code in Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go, collaborating with researchers and cross-functional teams. Your daily work includes assessing AI-generated code for efficiency, scalability, and reliability, plus building verification agents that catch error patterns. The one-month engagement runs 10-40 hours per week with partial PST overlap.
Turing
VerifiedSenior Software Engineer - AI Coding Agent Evaluation
Turing seeks experienced software engineers to assess how well AI coding agents handle real repositories. You will treat the agent like a colleague whose pull requests you review: checking whether it read the task, picked a sound design, and produced working code. The work spans Python, TypeScript/JavaScript, and Go codebases, plus the evaluation rubrics and data pipelines that sharpen model behavior. Researchers rely on your written judgment to turn one-off observations into repeatable quality signals. This is a freelance contract, roughly three months long, with 40 hours per week preferred.
Turing
VerifiedSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
This contract role places an experienced software engineer inside an evaluation workflow for frontier AI models. You will review AI-generated code for correctness, efficiency, and maintainability, validate solutions against real engineering tasks, and debug failures across different environments. You will also help build and refine evaluation datasets, benchmarks, and grading rubrics. The assignment runs for one month, requires at least 4 hours per day and 20 hours per week with a 4-hour overlap with PST, and is open only to candidates in the US. Work is fully remote.
Turing
VerifiedSenior Software Engineer – C++ (LLM Evaluation & Repository Validation)
Turing is assembling a team to build LLM evaluation and training datasets that teach models to solve real software engineering tasks. This project constructs verifiable SWE problems by mining public repository histories and using a synthetic, human-in-the-loop approach. A senior C++ engineer in this role will analyze trending GitHub issues, triage bugs, and evaluate unit test quality across open-source libraries. The work includes setting up repositories with Docker, running and modifying codebases locally, and partnering with researchers to design challenges that stretch LLM capabilities. This is a fully remote contractor assignment with required PST overlap.
Turing
VerifiedSenior Software Engineer – Python (LLM Evaluation & Repository Validation)
GitHub repository histories become training ground for LLM evaluation at this contract role. You will create verifiable software engineering tasks through a synthetic, human-in-the-loop pipeline that broadens dataset coverage across programming languages and difficulty levels. Day-to-day work involves triaging issues in trending open-source libraries, configuring Docker environments, and measuring unit test quality. You will run and modify local codebases to see how well LLMs handle real bug-fixing scenarios, then share findings with researchers. There is also room to take on a team lead role with junior engineers.
Micro1
VerifiedCoding Research Technical Staff
This full-time remote role centers on advancing how frontier coding agents are evaluated and improved. You will work where AI research, software engineering, and model evaluation meet, creating benchmarks, scoring methods, and data systems that shape how next-generation coding models are measured. The position involves close collaboration with researchers, engineers, and applied AI teams. Python or C++ skills will be central, along with familiarity with LLMs and coding agents.