Senior Software Engineer – LLM Evaluation
Listing checked August 25, 2026 · pay as published by Turing
Overview
In this contract role, you will build and refine training datasets that help large language models improve their coding skills. You will write, correct, and evaluate code in Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go, collaborating with researchers and cross-functional teams. Your daily work includes assessing AI-generated code for efficiency, scalability, and reliability, plus building verification agents that catch error patterns. The one-month engagement runs 10-40 hours per week with partial PST overlap.
What You'll Do6
- 1Curate code examples and build reference solutions across multiple languages, including Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go.
- 2Evaluate AI-generated code and refine it to improve efficiency, scalability, and reliability.
- 3Partner with cross-functional teams to strengthen AI-driven coding tools against industry performance benchmarks.
- 4Develop automated agents that check code quality and surface common error patterns.
- 5Map out stages of the software engineering cycle, from prototyping to monitoring, and assess how well models handle each stage.
- 6Design verification methods that confirm a solution's correctness for a software engineering task.
Requirements4
- 1Several years of software engineering experience, including at least 2 years of continuous full-time work at a top-tier product company such as Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, or IBM Research.
- 2Strong full-stack development skills and a track record of deploying scalable, production-grade software with modern languages and tools.
- 3Deep knowledge of software architecture, design, debugging, and code quality assessment.
- 4Clear written and verbal communication skills to produce structured evaluation rationales.
Who Should Apply
The ideal candidate has several years of software engineering experience, including 2+ years at a top-tier product company, and can move across full-stack development and code review. This role suits someone who enjoys evaluating and correcting code more than writing features from scratch, and who can communicate evaluation reasoning in a clear, structured way. It is less suitable for engineers who prefer long-term product ownership or who are not comfortable with short-term contract work and variable hours. Candidates often get rejected when they lack experience with production-grade systems at scale, or when their rationale for scoring AI-generated code is vague and unstructured.
Pay and demand for Software Engineering roles
AggregatedTypical pay
$70/hour
This role
Pay not disclosed
Most Software Engineering roles pay $40–$98 per hour.
Based on 1,092 similar roles that publish pay · 302 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 1,259
- Listed in last 30 days
- 483
- Remote
- 97%
Hiring most right now: micro1 (371) · SME Careers (253) · Mercor (158)
Most requested skills · share of roles
- ai training13%
- ai evaluation11%
- llm evaluation11%
- data annotation9%
Figures from Software Engineering roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
Undisclosed by Turing
Comparable roles pay
1,092 roles$40 – $98/ hr
Not this role’s pay. Turing has not published a range.
Confirmed during Turing profile screening.
Required Skills
Application Tip
Prepare a sample evaluation of AI-generated code in Python or JavaScript, showing how you assess correctness, efficiency, and scalability. Mention your experience at a top-tier product company and the languages you have used in production. Quantify your impact, such as the scale of systems you have worked on.
How NearSkill helps you apply
1. Check your job match
We compare your resume with the job requirements to help you understand how well your experience fits.
2. Continue to the employer’s website
We’ll direct you to the employer’s application portal, where you can complete the application.
3. Get updates by email
If NearSkill is handling follow-up for this role, we’ll email you when there’s an update on your screening.
Want to check your fit first?
See how your resume matches this role before applying.
Check my job matchSimilar open positions
Explore active roles that match your skills and interests.
Turing
VerifiedSenior Software Engineer – LLM Evaluation (US/Canada/WEU based)
Turing, a San Francisco-based research accelerator, is hiring a contract software engineer to evaluate AI-generated code and build datasets for large language models. You will curate code examples and write precise corrections in Python, JavaScript/ReactJS, C/C++, Rust, plus Java and Go. The role includes scoring model outputs for efficiency, scalability, and reliability, designing verification mechanisms, and partnering with researchers to strengthen enterprise coding solutions. This remote engagement runs one month, offers 10 to 40 flexible hours per week, and accepts candidates in the US, Canada, and Western Europe.
Turing
VerifiedSenior Software Engineer - AI Coding Agent Evaluation
Turing seeks experienced software engineers to assess how well AI coding agents handle real repositories. You will treat the agent like a colleague whose pull requests you review: checking whether it read the task, picked a sound design, and produced working code. The work spans Python, TypeScript/JavaScript, and Go codebases, plus the evaluation rubrics and data pipelines that sharpen model behavior. Researchers rely on your written judgment to turn one-off observations into repeatable quality signals. This is a freelance contract, roughly three months long, with 40 hours per week preferred.
Turing
VerifiedSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
This contract role places an experienced software engineer inside an evaluation workflow for frontier AI models. You will review AI-generated code for correctness, efficiency, and maintainability, validate solutions against real engineering tasks, and debug failures across different environments. You will also help build and refine evaluation datasets, benchmarks, and grading rubrics. The assignment runs for one month, requires at least 4 hours per day and 20 hours per week with a 4-hour overlap with PST, and is open only to candidates in the US. Work is fully remote.
Turing
VerifiedSenior Software Engineer – LLM Evaluation & Repository Validation
This contract role centers on building LLM evaluation datasets from public repository histories. You will triage GitHub issues, set up Docker environments, and run codebases on your own machine to score how well language models fix real bugs. The work targets popular repositories with 500+ stars and spans languages such as Python, JavaScript, and Go. The project uses a human-in-the-loop approach to expand task coverage across difficulty levels and programming languages.
Micro1
VerifiedCoding Research Technical Staff
This full-time remote role centers on advancing how frontier coding agents are evaluated and improved. You will work where AI research, software engineering, and model evaluation meet, creating benchmarks, scoring methods, and data systems that shape how next-generation coding models are measured. The position involves close collaboration with researchers, engineers, and applied AI teams. Python or C++ skills will be central, along with familiarity with LLMs and coding agents.
Turing
VerifiedSenior Software Engineer – Python (LLM Evaluation & Repository Validation)
GitHub repository histories become training ground for LLM evaluation at this contract role. You will create verifiable software engineering tasks through a synthetic, human-in-the-loop pipeline that broadens dataset coverage across programming languages and difficulty levels. Day-to-day work involves triaging issues in trending open-source libraries, configuring Docker environments, and measuring unit test quality. You will run and modify local codebases to see how well LLMs handle real bug-fixing scenarios, then share findings with researchers. There is also room to take on a team lead role with junior engineers.