Senior Domain Expert - AI Research Evaluation
Listing checked September 16, 2026 · pay as published by AfterQuery
Overview
AfterQuery builds a cohort of senior specialists across seven tracks: software and systems engineering, formal methods and computational science, ML inference and GPU kernels, enterprise operations, security, hardware design, and creative technology. The work centers on judging how frontier AI models handle hard expert-level problems, not on shipping production code. You define what counts as correct and what counts as excellent inside your domain, then turn that standard into scenarios, reference answers, and grading rubrics. The engagement runs remote and asynchronous on a contract basis, with about 4 hours of commitment each week and pay from $100.00 to $170.00 per hour.
What You'll Do5
- 1Design realistic technical scenarios and problem sets that reflect the hard parts of your specialist track.
- 2Write expert-level reference solutions that show the reasoning and domain judgment behind each answer.
- 3Build grading rubrics that separate correct, partial, and incorrect AI outputs.
- 4Review AI-generated responses for correctness, depth, and the quality of technical reasoning.
- 5Produce written explanations and structured feedback that other reviewers can use to apply your standard.
Requirements11
- 1Meet the experience and education bar for at least one track in the full program description.
- 2Show current or past hands-on work in your field, not just academic-adjacent experience.
- 3Write technical explanations and structured feedback at a high level.
- 4Work on your own schedule in a remote, asynchronous setup.
- 5Preferred: published research, open-source contributions, or recognized work product in your domain.
- 6Preferred: experience grading, reviewing, or editing others' technical work.
- 7Preferred signal for ML track: SGLang, vLLM, Mamba/Mamba2, TensorRT-LLM, or GPU kernel projects.
- 8Preferred signal for science track: Lean/Coq theorem proving, computational biology or genomics, condensed-matter physics, or quantum physics.
- 9Preferred signal for security track: reverse engineering, cryptanalysis, or CTF experience.
- 10Preferred signal for hardware track: RTL/HDL, CAD, or robotics work.
- 11Preferred signal for media track: audio engineering, linguistics, or multimodal design.
Who Should Apply
The ideal candidate brings deep hands-on practice in one of the seven tracks, plus a record that shows real work product: a paper, repository, certification, or shipped system. You should enjoy writing because the contract asks for reference solutions, rubrics, and structured critique, not just problem solving. This role fits experts who want a small remote side engagement of about 4 hours per week. Candidates who only have academic proximity without active practice often score low because the program needs current field judgment. Hiring reviewers also reject weak writers: if you cannot explain why an AI answer fails, you cannot build a rubric that others can follow.
Salary Insight
AfterQuery lists $100.00 to $170.00 per hour for this contract. The rate scales with your specialist track and seniority, so the top of the band goes to experts with rare domain depth and a strong public track record. At 4 hours per week, the engagement works as a side project rather than a full-time income. The listing does not break down pay by track, so expect that conversation during the hiring process.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$75/hour
This role
$100–$170/hr
Most Machine Learning & AI roles pay $50–$98 per hour. This role's pay sits above that range.
Based on 675 similar roles that publish pay · 169 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 753
- Listed in last 30 days
- 399
- Remote
- 96%
Hiring most right now: micro1 (296) · SME Careers (140) · Mercor (97)
Most requested skills · share of roles
- python16%
- llm evaluation15%
- ai training14%
- trainer feedback10%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
$100–170/hr
Required Skills
Application Tip
Name your track in the first line of your application and back it with one concrete artifact: a paper, repo, code review, CTF result, or hardware design. For the ML track, point to work with SGLang, vLLM, or GPU kernels. For science, security, hardware, or media, cite the matching bonus signal from the listing. Then add a short sample of your technical writing, such as a rubric or review comment, because the role hinges on authoring explanations and structured feedback.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
AfterQuery
VerifiedResearch Scientist - Formal Methods (Remote)
AfterQuery runs a research cohort of senior domain experts who probe how frontier AI models handle hard, expert-level problems. The STEM track needs people active in formal methods and computational science, whether that means theorem proving in Lean or Coq, genomics and computational biology, or condensed-matter and quantum physics. The work leans toward evaluation and research rather than production engineering: you define what a correct or excellent answer looks like inside your own specialty. Expect roughly 4 hours a week on a remote, asynchronous contract that fits beside a research post or an industry job.
Micro1
VerifiedAi Domain Expert Focused on Domain Knowledge and Evaluation
Remote, part-time contractor role focused on guiding AI systems through real-world input. You’ll review AI outputs for accuracy, craft prompts that test reasoning, and provide precise written feedback to improve model performance. Your domain knowledge matters most, even if you’re not required to have AI prior experience. You’ll join a global, distributed team and help shape how models learn and reason. Bolded areas reflect core competencies like data annotation, prompt engineering, and ethical awareness in AI.
Micro1
VerifiedAI Software Engineering Domain Expert
Remote, part-time contract work with micro1 where you apply deep software engineering knowledge to help train next-generation AI systems. You’ll review and polish AI-generated technical content, refine prompts, and judge model outputs using rubrics to ensure accuracy and quality. The role centers on drafting and editing technical docs, architecture plans, RFCs, and design specs that feed AI training, plus independent research and data annotation. Strong writing and a solid engineering background are essential, and you’ll collaborate asynchronously with project leads to meet deliverables.
Mercor
VerifiedEngineering & Software Domain Expert
Frontier AI models improve only when someone with real engineering experience checks their work. This role puts you inside a leading AI lab's GenAI team, where you will review engineering knowledge tasks, write instruction specs and golden solutions, and build benchmarks that measure model progress. You will work in the client's own tools, on-site in the Bay Area several days each week, and your output directly shapes how the model reasons about software and systems. Employment is W-2 through Cincinnatus LLC, with client-issued accounts and equipment.
AfterQuery
VerifiedCreative Technologist - Audio Design UX Remote
AfterQuery seeks senior creative technology experts for a research cohort that tests how frontier AI models handle hard audio, design, and UX problems. The contract calls for four hours per week and runs remote, so you can keep other commitments while contributing to model evaluation. Your work centers on research and assessment rather than production engineering. You decide what counts as a correct or excellent answer in your domain, then use that standard to judge AI outputs.
Micro1
VerifiedAi Consulting Domain Expert Focused on AI Output Evaluation
A remote contractor role focused on evaluating and enhancing AI outputs for real-world business use. You’ll help train next-generation systems by refining responses, writing and reviewing technical documentation, and improving prompts for large language models. Your domain knowledge in strategy and operations matters, even without prior AI experience. You’ll work with high-quality input to influence how models learn and perform.