Arts and Design AI Evaluation Expert - Remote
Listing checked September 16, 2026 · pay as published by Mercor
Overview
Mercor keeps a standing pool of creative professionals who grade and guide frontier AI models. You might write design briefs from past work, then use those briefs to judge model output on hierarchy, type choices, and colour decisions. Projects span graphic design, UX/UI, visual arts, music, theatre, film, photography, fashion, textiles, writing, and editing. A portfolio and the ability to explain why a piece works carry more weight than a credential. Apply with a resume, confirm where you work from, and finish a short AI interview that takes about 20 minutes.
What You'll Do6
- 1Turn delivered creative work into a brief that names the audience, limits, and quality bar, plus a reference answer for grading model attempts.
- 2Score model-generated design, writing, and imagery on craft: check whether hierarchy holds, type and colour choices look deliberate, and prose length earns its place.
- 3Point out outputs that look competent on the surface but would fail to ship because of brand rules, accessibility, licensing, or medium constraints.
- 4Explain your reasoning in writing so researchers can see the standard behind each judgment.
- 5Flag unclear instructions instead of guessing at the intended task.
- 6Bring professional judgment from your discipline to evaluation rubrics and model comparisons.
Requirements5
- 1Professional experience in design, writing, or a creative production field.
- 2Ability to write clear explanations of why a piece works or fails.
- 3Comfort with open-ended tasks and changing project details.
- 4Willingness to ask for clarification when a brief leaves gaps.
- 5A portfolio that shows finished work and the thinking behind it.
Who Should Apply
The best fit is a working designer, writer, or production professional who can point to a portfolio and explain craft choices in plain language. You should enjoy judging work against a stated standard, not just making new work. This pool suits people less if they want a fixed schedule or a single named client from day one. Candidates often score low when their written reasoning stays vague, such as saying a layout feels off without naming hierarchy, type, or accessibility issues. Another common miss is refusing to flag ambiguous instructions, which makes grading unreliable.
Salary Insight
Mercor lists creative roles in this pool at $60 to $80 an hour, and sets the exact rate per project by scope and depth. That range sits at a senior freelance level for design and writing evaluation, so candidates with deep domain expertise and clear written analysis tend to land toward the upper end.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$75/hour
This role
$60–$80/hr
Most Machine Learning & AI roles pay $50–$100 per hour. This role's pay falls inside that range.
Based on 543 similar roles that publish pay · 95 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 613
- Listed in last 30 days
- 277
- Remote
- 96%
Hiring most right now: micro1 (273) · Mercor (87) · SME Careers (66)
Most requested skills · share of roles
- python16%
- technical writing8%
- llm evaluation8%
- data annotation7%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
$60–80/hr
Required Skills
Application Tip
When you apply, include a portfolio link and, next to two pieces, write a short note on the audience, constraint, and standard each piece had to meet. That mirrors the brief-writing and grading work and gives reviewers proof you can explain craft, not just produce it.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
AfterQuery
VerifiedGraphic Designer - Brand & AI Image Evaluation (Remote)
AfterQuery contracts working designers to support the image generation programs run by frontier AI labs. You write brand and layout briefs, then judge what the models produce against real design standards: type rendering, logo fidelity, and composition. The work sits inside graphic design and brand identity, so your craft decides how you are measured rather than how many images you push through. Hours are yours to set around a 10 hour weekly minimum, and pay lands weekly through Stripe at a rate matched to your experience.
Mercor
VerifiedGeneralist AI Evaluation Expert (Remote)
Mercor runs a standing pool for people who want part-time work in AI training and model evaluation. The team connects generalists to projects that need careful human judgment, from writing prompts and reference answers to grading how well a model handles general reasoning. Contracts pay $40 to $70 per hour and open on short notice, so the listing stays active even when no project is live. You apply once, complete a short interview, and join a network that clients draw from when a matching task appears.
Micro1
VerifiedDesign Expert (Multidisciplinary)
This is a contract role for a seasoned multidisciplinary designer who wants to apply their craft in a unique way — helping to train the next generation of AI systems. You'll evaluate and refine design-related materials, from UX/UI flows to visual identity assets, providing feedback that directly improves how AI models understand great design. No prior AI experience is needed; your domain expertise is the key ingredient.
Micro1
VerifiedCreative & Design Specialist (Remote)
micro1 is looking for experienced creative professionals to support a customer project that trains next-generation AI systems. Your day-to-day work will involve using tools like Figma, Adobe After Effects, and DaVinci Resolve to complete structured tasks, evaluate AI-generated media, and provide detailed feedback. No prior AI experience is needed, but you must have paid experience with human data work for AI models, such as data labeling or RLHF. This is a chance to apply your design and multimedia skills in a way that directly shapes how AI learns and performs.
AfterQuery
VerifiedCreative Technologist - Audio Design UX Remote
AfterQuery seeks senior creative technology experts for a research cohort that tests how frontier AI models handle hard audio, design, and UX problems. The contract calls for four hours per week and runs remote, so you can keep other commitments while contributing to model evaluation. Your work centers on research and assessment rather than production engineering. You decide what counts as a correct or excellent answer in your domain, then use that standard to judge AI outputs.
Micro1
VerifiedAI Evaluation Specialist
As an AI Evaluation Specialist, you'll help train next-generation AI systems by designing and executing hands-on evaluation tasks. Your insights will directly shape how models learn, reason, and perform on practical computer-based workflows. This is a fully remote contract role where meticulous observation and clear documentation are key.