Legal Expert - AI Legal Reasoning Evaluation (Remote)
Listing checked September 15, 2026 · pay as published by AfterQuery
Overview
AfterQuery recruits legal experts to teach and test how AI models handle legal reasoning. Your work can range from composing hard legal prompts to checking model answers for accuracy, sound logic, and hidden mistakes such as invented citations or rules used in the wrong context. You may also draft reference answers and explanations, or build grading rubrics that separate strong legal analysis from weak output. This contract role runs remote at 40 hours per week, with pay from $50 to $200 per hour.
What You'll Do7
- 1Write demanding legal prompts that probe how an AI model applies doctrine, procedure, or statutory rules.
- 2Review model outputs for correct legal conclusions, accurate citations, and sound chains of reasoning.
- 3Flag subtle flaws, including fabricated case citations, misread authority, and rules applied to the wrong facts.
- 4Draft model answers and explanations that show the expected legal analysis.
- 5Build rubrics that define what counts as a strong, weak, or failing response.
- 6Compare multiple model responses against the same legal question and assign scores.
- 7Document recurring errors so the project can improve training data.
Requirements6
- 1Active bar admission, as requested during application.
- 2Strong legal reasoning across at least one practice area.
- 3Skill in legal research and legal writing.
- 4Ability to catch fabricated citations, misapplied rules, and faulty logic in legal arguments.
- 5Comfort drafting clear explanations of legal concepts.
- 6Capacity for roughly 40 hours per week during the contract.
Who Should Apply
Candidates who hold an active bar license and enjoy close reading of cases, statutes, and regulations will fit this work. You should feel at home drafting legal analysis and grading someone else's reasoning, because the role centers on AI model evaluation rather than client representation. Applicants who lack bar admission or cannot commit roughly 40 hours each week tend to score poorly in review. The assessment also trips up people who submit vague legal writing or miss citation errors, so precision matters more than volume.
Salary Insight
Pay ranges from $50 to $200 per hour for this contract. That spread usually maps to how much legal specialization and evaluation experience you bring, with the higher end for senior attorneys who can design prompts and rubrics for complex domains. The hiring team sets the exact rate during the process.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$80/hour
This role
$50–$200/hr
Most Machine Learning & AI roles pay $55–$105 per hour. This role's pay sits above that range.
Based on 575 similar roles that publish pay · 94 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 634
- Listed in last 30 days
- 285
- Remote
- 97%
Hiring most right now: micro1 (297) · Mercor (92) · SME Careers (64)
Most requested skills · share of roles
- python15%
- technical writing8%
- llm evaluation7%
- data annotation6%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
$50–200/hr
Required Skills
Application Tip
Complete the assessment with identifier N7AQjnsLZTWiWwoievVN before review and include your Bar Admission details. In your sample answers, show how you catch a fabricated citation or a rule applied to the wrong facts, because that skill sits at the core of the evaluation work.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
Mercor
VerifiedLegal Expert - AI Evaluation and Review (Remote)
Mercor keeps an open pool for legal professionals who want part-time, remote work on AI evaluation and training. Research teams behind major AI models turn to that pool for legal judgment they cannot generate on their own, and your standards shape the rubric that grades model output. Assignments can involve writing a legal problem from a matter you handled, complete with facts, forum, governing law, and a reference answer. You might also review model-drafted provisions, privilege calls, or citations for accuracy and persuasive force. Projects open on short notice, so the listing stays live even when no matching engagement runs this week.
SME Careers
VerifiedLegal Expert for AI Data Training, Remote Contract
Remote contract role focused on guiding AI data training by reviewing legal analyses and producing precise explanations. You will assess reasoning quality, identify gaps in legal methodology, and provide structured feedback that maps facts to valid elements and defenses. Citations, rules, and jurisdictional relevance are checked for accuracy, with clear notes for improvement. This assignment supports SME Careers' AI data services by delivering reliable, well-reasoned content to improve model performance.
Mercor
VerifiedLegal Expert (Remote) - AI Model Evaluation
Mercor maintains an open pool of licensed lawyers who supply legal judgment to the research teams behind top AI models. Work arrives as project-based tasks: drafting a motion or brief from a fact pattern, then writing the reference answer that grades a model's attempt. Other assignments involve scoring model-drafted contract and discovery work, checking whether a privilege call holds, or deciding if a citation supports the claim it backs. The listing stays open even when no project matches your practice area, and new engagements start on short notice. Pay for past projects in this field has ranged from $100 to $150 an hour, with scope and depth setting the exact figure.
AfterQuery
VerifiedAttorney - Remote Contract, Legal AI Content
AfterQuery hires Juris Doctor graduates to build legal reasoning challenges for AI models. You will craft questions and case studies in areas such as contract law, constitutional law, and intellectual property. The contract runs 20 hours per week, remote and asynchronous, with pay between $75 and $200 per hour. You also review model answers and flag gaps in logic or legal accuracy. A YC-backed team pairs legal experts with technical staff to sharpen content difficulty.
Handshake AI
VerifiedLawyers - Remote AI Evaluation for Legal Experts
Handshake AI hires lawyers with active or recent practice experience to review AI output for legal soundness. The work centers on legal analysis, statutory interpretation, case law reasoning, and drafting standards. You design job-related prompts and critique model responses for accuracy, clarity, and relevance to real legal practice. This remote contract role runs alongside a primary legal job and is not a law firm or in-house counsel position. The program operates year-round, placement depends on current project needs, and no AI or machine learning background is required.
AfterQuery
VerifiedParalegal Expert - Remote Contract Work
AfterQuery, a YC-backed startup, hires paralegals to build and grade the legal challenges that teach AI models how to handle real legal support work. You design scenarios around legal research, citation formats, document drafting, discovery management and trial preparation, then judge how well a model answers them. The contract runs about 10 hours a week, fully remote and asynchronous, so you set your own hours around other commitments. Pay sits at $50 to $60 per hour for this specialized legal content work.