Data Analysis Expert - Remote AI Evaluation
Listing checked September 16, 2026 · pay as published by Mercor
Overview
Mercor maintains a standing pool for data analysis experts who want to train and evaluate frontier AI models. Research teams bring their hardest data judgment calls here, and your standards help build the rubrics that grade model answers. The pool spans business intelligence, data engineering, statistics, and machine learning, with part-time project work that starts on short notice. A matching project may arrive within a week or take several months, depending on client needs. The listing stays open even when no project is active.
What You'll Do6
- 1Turn an analysis you shipped into a test problem, complete with the dataset, the core question, the constraint, and a reference answer for grading a model's response.
- 2Review model-written analyses and pipelines for logic, asking whether a query answers the actual question, whether a join fans out, and whether a metric means what the model claims.
- 3Spot results that look valid but would mislead a decision-maker because of selection effects, data leakage, or a metric that fails to track the outcome.
- 4Write clear reasoning for each judgment so project teams understand why a model response passes or fails.
- 5Flag ambiguous instructions or missing context before they corrupt a grading task.
- 6Build rubrics and answer keys that other experts can apply to similar model outputs.
Requirements6
- 1Professional experience in analytics, data engineering, statistics, or machine learning.
- 2A track record of shipped analyses that someone used to make a real decision.
- 3Strong written communication, because most tasks require you to explain your reasoning in detail.
- 4Comfort with ambiguous briefs and the judgment to ask questions when instructions lack clarity.
- 5Depth in at least one area: business intelligence, data pipelines, statistical modeling, or machine learning.
- 6Ability to identify selection effects, leakage, and metric-outcome mismatches in analytical work.
Who Should Apply
Data practitioners with a few years of professional analytics, data engineering, statistics, or machine learning work fit this pool best, including those who can point to analyses that shaped a business or product decision. Clear writers thrive here because grading model output depends on explaining why an answer succeeds or fails. The role suits people who handle vague prompts without freezing and who call out unclear instructions instead of guessing. Candidates who need tasks defined from the start, dislike writing rationales, or lack shipped analysis work tend to score low. A common rejection reason is a resume that lists tools but shows no analysis with a concrete decision attached. Another is treating the short AI interview as a formality rather than a chance to demonstrate reasoning.
Salary Insight
Mercor lists $70 to $120 per hour for this data analysis expert pool. That range sits at the higher end for part-time AI evaluation and data judgment work, where rates reflect specialized domain skill. The top of the band matches candidates with deep experience in machine learning, statistics, or complex data engineering, while the lower end fits solid analytics generalists. The specific project listing names the rate, hours, and client before any hiring decision, so final pay ties to that brief.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$75/hour
This role
$70–$120/hr
Most Machine Learning & AI roles pay $55–$100 per hour. This role's pay falls inside that range.
Based on 578 similar roles that publish pay · 97 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 677
- Listed in last 30 days
- 313
- Remote
- 97%
Hiring most right now: micro1 (294) · Mercor (90) · Turing (69)
Most requested skills · share of roles
- python16%
- technical writing8%
- llm evaluation7%
- data annotation7%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
$70–120/hr
Required Skills
Application Tip
During the short AI interview, walk through one shipped analysis from question to decision, naming the tools and the constraint you faced. Add that same example to your resume summary, with a number or outcome if you have one. Complete the domain expert interview and join every network you qualify for, since each one raises your odds of a match. Confirm your work location when you apply.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
Mercor
VerifiedScience Research Expert - AI Evaluation (Remote)
Mercor builds pools of scientific experts who help frontier AI teams judge model work in domains those teams cannot cover on their own. This listing is a standing application for part-time, remote research work, not a single opening. Accepted experts join a science pool, and Mercor invites matching people to specific projects that name the client, rate, hours, and hiring decision. Projects in life, physical, social sciences, math, and policy research have paid $60 to $120 per hour, with scope and depth setting the exact rate. Your role centers on research design, statistical reasoning, and written judgments that become grading rubrics for AI evaluation.
Mercor
VerifiedLegal Expert - AI Evaluation and Review (Remote)
Mercor keeps an open pool for legal professionals who want part-time, remote work on AI evaluation and training. Research teams behind major AI models turn to that pool for legal judgment they cannot generate on their own, and your standards shape the rubric that grades model output. Assignments can involve writing a legal problem from a matter you handled, complete with facts, forum, governing law, and a reference answer. You might also review model-drafted provisions, privilege calls, or citations for accuracy and persuasive force. Projects open on short notice, so the listing stays live even when no matching engagement runs this week.
Micro1
VerifiedData Scientist for AI Model Evaluation and Quality Assurance
micro1 is hiring data scientists to work with a leading AI lab on a contract basis. Your job is to bring quantitative rigor to how AI models handle statistics, machine learning, and experimentation. You will review model outputs, spot methodological flaws, and build expert-level prompts and datasets. This is a remote role for someone with a strong analytical background and clear technical writing skills.
Mercor
VerifiedMachine Learning Expert (Remote, Part-Time)
Mercor runs a standing pool for machine learning practitioners who provide the expert judgment that frontier AI research teams cannot produce on their own. Your standards become the rubric that grades model outputs. Projects are remote and part-time, and clients set rates from $70 to $120 per hour based on scope and depth. Work can involve writing a modeling problem from a system you shipped, checking training and evaluation code that models produce for loss-objective alignment and eval leakage, or catching an approach that would train without error but fail in production because of distribution shift, label noise, or a metric that rewards the wrong behavior.
Mercor
VerifiedGeneralist AI Evaluation Expert (Remote)
Mercor runs a standing pool for people who want part-time work in AI training and model evaluation. The team connects generalists to projects that need careful human judgment, from writing prompts and reference answers to grading how well a model handles general reasoning. Contracts pay $40 to $70 per hour and open on short notice, so the listing stays active even when no project is live. You apply once, complete a short interview, and join a network that clients draw from when a matching task appears.
Mercor
VerifiedRemote Survey Research Expert for AI Evaluation
Mercor is connecting you with a leading AI research organization that needs seasoned survey professionals to evaluate its surveys for methodological quality. You will examine question wording, response scales, and sampling approaches, and articulate why certain designs yield unreliable or biased data. This remote, hourly role suits practitioners from market research, consumer insights, behavioral science, and related fields who have designed and fielded surveys in real projects. Your critical eye will help improve how AI systems think about research design and evidence quality. Candidates with hands-on experience at research agencies or internal insights teams will feel at home.