Legal & Compliance Specialist Rubric Design
Listing checked September 29, 2026 · pay as published by SME Careers
Overview
As an hourly, remote Legal & Compliance SME, you will craft challenging scenarios that test expert judgment in legal analysis and regulatory interpretation. You will design detailed rubrics that articulate rubric design and ai evaluation tasks, specifying issues, standards, reasoning steps, exceptions, and risks. Your analyses will guide model benchmarking and data annotation in privacy, contracts, and financial regulation. This role supports SME Careers by delivering precise evaluation materials that elevate professional quality across AI systems.
What You'll Do9
- 1Develop realistic legal and compliance tasks spanning contracts, privacy, financial regulation, and corporate matters.
- 2Design granular rubrics that clearly define relevant issues, applicable standards, reasoning steps, exceptions, and risks for each task.
- 3Evaluate AI-generated analyses for accuracy, completeness, and professional rigor using ai evaluation criteria.
- 4Assess whether AI responses identify material legal considerations and demonstrate issue spotting.
- 5Identify gaps in reasoning, flag unsupported conclusions, and call out missing exceptions.
- 6Critique peer rubrics for ambiguity, redundancy, or misaligned expectations and provide constructive feedback.
- 7Develop reference content including gold-standard analyses and structured answer frameworks.
- 8Support AI training workflows by producing questions, rubrics, evaluations for RLHF and SFT.
- 9Maintain professional quality by ensuring content distinguishes factual analysis from interpretation and judgment.
Requirements11
- 1holder of a jd, llb, llm, law degree, compliance qualification, or equivalent professional background relevant to the specialty.
- 2professional legal qualification or bar admission strongly preferred for attorney-focused work.
- 3strong professional proficiency in English, minimum c1.
- 43+ years of professional experience in law, regulatory compliance, legal research, policy, risk, privacy, financial compliance, contracts, or related fields.
- 5demonstrated expertise in a clearly defined legal, regulatory, or compliance domain.
- 6ability to create challenging professional scenarios requiring issue spotting, interpretation, analysis, and judgment.
- 7strong ability to design rubrics defining issues, standards, reasoning steps, relevant exceptions, risks, and critical omissions.
- 8ability to distinguish between legitimate alternative legal interpretations and objectively deficient analysis.
- 9comfortable reviewing rubrics developed by other legal or compliance experts.
- 10prior experience with ai evaluation, legal ai, data annotation, rl hf, model benchmarking, or rubric-based scoring is strongly preferred.
- 11experience with legal education, examination writing, peer review, legal editing, quality assurance, policy drafting, compliance testing, or supervisory review is strongly preferred.
Who Should Apply
Ideal candidates hold a law degree and 3+ years in regulatory compliance or related fields, with proven rubrics or exam-writing experience. Strong English (C1+) and comfort with remote, contract work are essential. Experience in ai evaluation, data annotation, RLHF, or model benchmarking is highly valued. This role may be less suitable for individuals without legal qualification or those lacking rubric-design experience or the ability to deliver precise materials on tight timelines.
Salary Insight
Pay is $120/hr. This hourly rate reflects specialized consultancy work in legal compliance and rubric design, typical for contract SME engagements requiring expert judgment and precise output.
Pay and demand for Legal roles
AggregatedTypical pay
$70/hour
This role
up to $120/hr
Most Legal roles pay $35–$100 per hour. This role's pay sits above that range.
Based on 1,051 similar roles that publish pay · 301 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 1,199
- Listed in last 30 days
- 420
- Remote
- 98%
Hiring most right now: micro1 (340) · SME Careers (253) · AfterQuery (148)
Most requested skills · share of roles
- ai training13%
- ai evaluation12%
- llm evaluation11%
- data annotation9%
Figures from Legal roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
Up to $120/hr
Required Skills
Application Tip
Prepare a 1-page sample rubric draft for a privacy/regulatory scenario that demonstrates issue spotting, standards alignment, and a clear scoring rationale; include RLHF considerations and a brief evaluation of an AI-generated answer.
How NearSkill helps you apply
1. Check your job match
We compare your resume with the job requirements to help you understand how well your experience fits.
2. Continue to the employer’s website
We’ll direct you to the employer’s application portal, where you can complete the application.
3. Get updates by email
If NearSkill is handling follow-up for this role, we’ll email you when there’s an update on your screening.
Want to check your fit first?
See how your resume matches this role before applying.
Check my job matchSimilar open positions
Explore active roles that match your skills and interests.
SME Careers
VerifiedLegal Expert for AI Data Training, Remote Contract
Remote contract role focused on guiding AI data training by reviewing legal analyses and producing precise explanations. You will assess reasoning quality, identify gaps in legal methodology, and provide structured feedback that maps facts to valid elements and defenses. Citations, rules, and jurisdictional relevance are checked for accuracy, with clear notes for improvement. This assignment supports SME Careers' AI data services by delivering reliable, well-reasoned content to improve model performance.
SME Careers
VerifiedCompliance and Regulatory Expert for AI Training Content
Remote, hourly contract for a Compliance and Regulatory SME focused on AI training data. You will review AI-generated compliance outputs and craft expert content such as policies, control descriptions, monitoring plans, and risk assessments. Your assessments anchor regulatory interpretation in practical controls and clear documentation, evaluated against defensible criteria. Expect collaboration across time zones, using GRC tooling, privacy concepts and ISO-style controls to ground your work.
Mercor
VerifiedLegal Expert - AI Evaluation and Review
Mercor keeps an open pool for legal professionals who want part-time, remote work on AI evaluation and training. Research teams behind major AI models turn to that pool for legal judgment they cannot generate on their own, and your standards shape the rubric that grades model output. Assignments can involve writing a legal problem from a matter you handled, complete with facts, forum, governing law, and a reference answer. You might also review model-drafted provisions, privilege calls, or citations for accuracy and persuasive force. Projects open on short notice, so the listing stays live even when no matching engagement runs this week.
AfterQuery
VerifiedRegulatory Compliance Officer Expert
AfterQuery, a YC-backed startup, needs a compliance practitioner to build and review regulatory compliance challenges that test AI models. You will craft questions and case studies that probe how models interpret rules, assess risk, handle audits and run investigations across ethics programs and reporting duties. The contract runs remote and async, with a commitment of about 10 hours each week. Your work pairs real compliance judgment with model evaluation, shaping how AI handles practical regulatory problems.
SME Careers
VerifiedLegal QA Lead AI Training Content Remote Contract
This remote, hourly contract places you as a Legal QA Lead overseeing quality across AI training programs for legal content. You’ll review AI-generated legal explanations, contract reviews, and compliance guidance, checking accuracy against project rubrics and flagging issues like unsupported citations or jurisdictional gaps. Deliver precise feedback and guide trainers and QAs via Discord, while maintaining up-to-date style guides, FAQs, and calibration tasks. You’ll help onboard contributors, document processes, and ensure outputs are well-reasoned, cautious, and aligned with client expectations.
SME Careers
VerifiedAttorney AI Training Specialist for Legal Content Review
Remote, hourly contract role for an attorney who reviews AI-generated legal outputs and creates expert legal content. You assess reasoning quality and perform step-by-step issue analyses, delivering precise written feedback. You fact-check legal concepts and, when needed, provide citations to primary authorities to ensure accuracy and clarity. This remote contract position supports training data programs for major AI labs.