Senior Software Engineer – LLM Evaluation
Overview
In this contract role, you will build and refine training datasets that help large language models improve their coding skills. You will write, correct, and evaluate code in Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go, collaborating with researchers and cross-functional teams. Your daily work includes assessing AI-generated code for efficiency, scalability, and reliability, plus building verification agents that catch error patterns. The one-month engagement runs 10-40 hours per week with partial PST overlap.
What You'll Do6
- 1Curate code examples and build reference solutions across multiple languages, including Python, JavaScript (ReactJS), C/C++, Java, Rust, and Go.
- 2Evaluate AI-generated code and refine it to improve efficiency, scalability, and reliability.
- 3Partner with cross-functional teams to strengthen AI-driven coding tools against industry performance benchmarks.
- 4Develop automated agents that check code quality and surface common error patterns.
- 5Map out stages of the software engineering cycle, from prototyping to monitoring, and assess how well models handle each stage.
- 6Design verification methods that confirm a solution's correctness for a software engineering task.
Requirements4
- 1Several years of software engineering experience, including at least 2 years of continuous full-time work at a top-tier product company such as Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, or IBM Research.
- 2Strong full-stack development skills and a track record of deploying scalable, production-grade software with modern languages and tools.
- 3Deep knowledge of software architecture, design, debugging, and code quality assessment.
- 4Clear written and verbal communication skills to produce structured evaluation rationales.
Who Should Apply
The ideal candidate has several years of software engineering experience, including 2+ years at a top-tier product company, and can move across full-stack development and code review. This role suits someone who enjoys evaluating and correcting code more than writing features from scratch, and who can communicate evaluation reasoning in a clear, structured way. It is less suitable for engineers who prefer long-term product ownership or who are not comfortable with short-term contract work and variable hours. Candidates often get rejected when they lack experience with production-grade systems at scale, or when their rationale for scoring AI-generated code is vague and unstructured.
Location
Required Skills
Application Tip
Prepare a sample evaluation of AI-generated code in Python or JavaScript, showing how you assess correctness, efficiency, and scalability. Mention your experience at a top-tier product company and the languages you have used in production. Quantify your impact, such as the scale of systems you have worked on.
See NearSkill jobs more often in your search
How your application is processed
1Application received
Your resume and details are logged the moment you apply.
2ATS + eligibility screening
We check your profile against the role’s skills, seniority, and requirements.
3Employer sees qualified profiles only
Only candidates who clear screening move forward.
Similar open positions
Explore active roles that match your skills and interests.
Turing
VerifiedSenior Software Engineer – LLM Evaluation (US/Canada/WEU based)
Turing, a San Francisco-based research accelerator, is hiring a contract software engineer to evaluate AI-generated code and build datasets for large language models. You will curate code examples and write precise corrections in Python, JavaScript/ReactJS, C/C++, Rust, plus Java and Go. The role includes scoring model outputs for efficiency, scalability, and reliability, designing verification mechanisms, and partnering with researchers to strengthen enterprise coding solutions. This remote engagement runs one month, offers 10 to 40 flexible hours per week, and accepts candidates in the US, Canada, and Western Europe.
Turing
VerifiedSoftware Engineer – AI Code Evaluation & Benchmarking (US candidates only)
This contract role places an experienced software engineer inside an evaluation workflow for frontier AI models. You will review AI-generated code for correctness, efficiency, and maintainability, validate solutions against real engineering tasks, and debug failures across different environments. You will also help build and refine evaluation datasets, benchmarks, and grading rubrics. The assignment runs for one month, requires at least 4 hours per day and 20 hours per week with a 4-hour overlap with PST, and is open only to candidates in the US. Work is fully remote.
Turing
VerifiedSenior Software Engineer – LLM Evaluation & Repository Validation
This contract role centers on building LLM evaluation datasets from public repository histories. You will triage GitHub issues, set up Docker environments, and run codebases on your own machine to score how well language models fix real bugs. The work targets popular repositories with 500+ stars and spans languages such as Python, JavaScript, and Go. The project uses a human-in-the-loop approach to expand task coverage across difficulty levels and programming languages.
Turing
VerifiedSenior Software Engineer – Python (LLM Evaluation & Repository Validation)
GitHub repository histories become training ground for LLM evaluation at this contract role. You will create verifiable software engineering tasks through a synthetic, human-in-the-loop pipeline that broadens dataset coverage across programming languages and difficulty levels. Day-to-day work involves triaging issues in trending open-source libraries, configuring Docker environments, and measuring unit test quality. You will run and modify local codebases to see how well LLMs handle real bug-fixing scenarios, then share findings with researchers. There is also room to take on a team lead role with junior engineers.
Turing
VerifiedSenior Software Engineer – C#(LLM Evaluation & Repository Validation)
This project builds LLM evaluation and training datasets that help models solve realistic software engineering problems. The work involves creating verifiable software engineering tasks from public repository histories using a synthetic, human-in-the-loop approach. Engineers analyze GitHub issues, set up Docker-based environments, and evaluate test coverage to judge model performance in bug-fixing scenarios. The role fits a tech lead-level engineer comfortable with C#, Git, and running complex codebases locally.
Turing
VerifiedSenior Software Engineer – Go (LLM Evaluation & Repository Validation)
This role centers on building LLM evaluation datasets that teach models to solve realistic software engineering problems. You will triage GitHub issues from popular open-source repositories, set up Docker containers and code environments, and run modified codebases in local environments to measure how well models handle bug-fixing tasks. The work is hands-on and includes evaluating unit test coverage, collaborating with researchers to identify challenging tasks, and the option to lead junior engineers. Projects draw on public repository histories with a human-in-the-loop, synthetic approach to expand task coverage across Go and other programming languages.