Domain Contributor - Evaluation Specialist
Listing checked October 21, 2026 · pay as published by micro1
Overview
micro1 seeks Domain Contributor - Evaluation Specialists to join an internal project that evaluates and improves advanced AI models. You will design challenging tasks that expose reasoning gaps, verify model outputs against primary sources, and document findings. This is a remote contractor role, about 15 hours per week, with pay per task. Experts in history, law, finance, journalism, or tech are needed.
What You'll Do6
- 1Create complex, domain-specific tasks that test the limits of AI models and reveal reasoning flaws.
- 2Write questions and scenarios that require deep subject knowledge, not just surface-level understanding.
- 3Check AI-generated answers for accuracy using authoritative primary sources in your field.
- 4Explain in writing where models fall short, citing missed reasoning, source errors, or overlooked nuances.
- 5Apply detailed task guidelines and evaluation rubrics to every submission.
- 6Deliver completed work independently on a rolling basis, meeting project standards for clarity.
Requirements7
- 1Hands-on experience with AI model training, annotation, RLHF, or evaluation.
- 2Deep expertise in one core domain: history, archival or library science, law (paralegal, legal academic, compliance), finance (equity research, PE/VC, forensic accounting), journalism, or technology/industry history.
- 3Advanced research skills and a track record of verifying complex claims against authoritative sources.
- 4Strong written and verbal communication skills to articulate findings and feedback.
- 5Proven ability to follow detailed instructions and evaluation frameworks precisely.
- 6Background in adversarial or 'stomp the model' task design is a plus.
- 7Graduate-level research, Fortune 100 experience, or professional fluency in a non-English language is advantageous.
Who Should Apply
The ideal candidate has deep domain knowledge in history, law, finance, journalism, or tech, plus prior experience with AI training or evaluation. You should be skilled at researching primary sources and explaining reasoning clearly. If you can design tasks that challenge models and spot subtle errors, this role fits you.
Salary Insight
Compensation is $30 to $60 per hour, paid per task that meets project specifications. The time to complete tasks varies with experience and workflow. Minimum weekly submissions apply.
Pay and demand for AI Training roles
AggregatedTypical pay
$70/hour
This role
$30–$60/hr
Most AI Training roles pay $40–$95 per hour. This role's pay falls inside that range.
Based on 1,035 similar roles that publish pay · 302 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 1,172
- Listed in last 30 days
- 462
- Remote
- 98%
Hiring most right now: micro1 (345) · SME Careers (253) · Handshake AI (143)
Most requested skills · share of roles
- ai training13%
- ai evaluation13%
- llm evaluation12%
- data annotation10%
Figures from AI Training roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
$30–60/hr
Required Skills
Application Tip
Highlight your experience with AI evaluation or RLHF and specify your domain expertise. Include examples of verifying claims against primary sources and designing tasks that test model reasoning. Mention any adversarial task design or advanced research you have done.
How NearSkill helps you apply
1. Check your job match
We compare your resume with the job requirements to help you understand how well your experience fits.
2. Continue to the employer’s website
We’ll direct you to the employer’s application portal, where you can complete the application.
3. Get updates by email
If NearSkill is handling follow-up for this role, we’ll email you when there’s an update on your screening.
Want to check your fit first?
See how your resume matches this role before applying.
Check my job matchSimilar open positions
Explore active roles that match your skills and interests.
micro1
VerifiedEvaluation Specialist/Recent Grad
micro1 is hiring Evaluation Specialists to support a high-impact project for a customer at the frontier of artificial intelligence. You will create original question-and-answer pairs that test the limits of advanced AI models across diverse subjects. The role involves deep research, source triangulation, and iterative testing to push model reasoning. No prior AI experience is required; your domain knowledge and research skills are what matter. This is a remote contractor position with output-based pay.
micro1
VerifiedResearch Evaluation Specialist (PhD or Masters / Researcher / Professor)
micro1 seeks Research Evaluation Specialists with a PhD or Masters to build rigorous test material for advanced AI model evaluation on a customer project. You will craft original, high-difficulty question-and-answer pairs at the frontier of your discipline, then verify and document answers using primary sources and authoritative references. The role is remote and worldwide, about 15 hours per week, with output-based pay. Prior experience in AI training, data labeling, or model evaluation is required.
micro1
VerifiedAI Evaluation Specialist
As an AI Evaluation Specialist, you'll help train next-generation AI systems by designing and executing hands-on evaluation tasks. Your insights will directly shape how models learn, reason, and perform on practical computer-based workflows. This is a fully remote contract role where meticulous observation and clear documentation are key.
Turing
VerifiedDomain Expert- Politics
Turing seeks a Politics Domain Expert to evaluate and improve Large Language Models with a focus on political science, governance, elections, public policy, and international relations. The role involves designing challenging prompts that test factual accuracy, reasoning, completeness, and nuance in AI-generated responses. You will build benchmark datasets and adversarial test cases to expose hallucinations, logical inconsistencies, outdated information, and edge cases. Work happens remotely on a contractor basis for 8 weeks at 40 hours per week with a 4-hour PST overlap.
Turing
VerifiedDomain Expert - Art
The role centers on evaluating and improving Large Language Models (LLMs) using deep knowledge of art history, visual arts, architecture, and design. You will design prompts that range from simple to advanced and assess model answers for factual accuracy, reasoning, completeness, and nuance. Each day involves comparing multiple AI outputs, ranking them, and explaining correct or incorrect responses with reliable references. You will also create benchmark datasets and adversarial test cases to expose hallucinations and logical gaps. The position is an 8-week contractor role requiring 40 hours per week with at least 4 hours of PST overlap.
Turing
VerifiedDomain Expert - TV Show & Movies
Turing is hiring a TV Shows & Movies Domain Expert for an 8-week contractor assignment focused on evaluating and improving large language models. The day-to-day work centers on creating prompts of varying difficulty across entertainment topics, then comparing and ranking AI responses. The expert will judge responses for factual accuracy, reasoning, completeness, and nuance, and will flag hallucinations, outdated information, and edge cases. This remote role is open to US-based candidates who can commit 40 hours per week with 4 hours of daily overlap with PST.