Generalist AI Evaluation Expert

What You'll Do7

• Write prompts and reference answers for everyday questions with a correct response, plus the context a strong answer needs.
• Grade model output on general reasoning, checking whether the answer is right, follows the instruction given, and avoids filler.
• Mark cases where a model sounds sure but states something false or clashes with an earlier claim in the same answer.
• Flag responses that answer a question the user did not ask.
• Explain your verdicts in writing so project teams can follow your logic.
• Call out unclear instructions or ambiguous prompts instead of guessing at intent.
• Apply each project's rubric to every prompt and output you review.

Requirements5

• Careful reading habits and sound judgment, backed by a professional or academic background in any field.
• Clear written communication, since most tasks ask you to justify your reasoning.
• Comfort with ambiguous tasks and a willingness to flag unclear instructions.
• Mercor does not require a specific credential. What matters is that you can say why an answer is wrong.
• Ability to confirm your work location and complete a short AI interview, about 20 minutes.

Who Should Apply

Generalists who read with care, write clear explanations, and can defend a judgment about a model's answer fit this pool well. The work suits people from any professional or academic field who want part-time AI evaluation projects and are open to ambiguous prompts. Candidates who want a single fixed project, a set schedule, or a decision on this listing will find a poor fit, because Mercor treats it as a standing pool and reaches out when a client project matches. Applicants often score low when they summarize an answer without naming the exact flaw, or when they ignore the instruction and grade against their own preferences. A missing resume or an incomplete interview also keeps you out of the matching pool.

Salary Insight