Legal Expert (Remote) - AI Model Evaluation

What You'll Do7

• Draft motions, briefs, or other legal documents from a fact pattern, then create the reference answer that graders use to score a model's version.
• Review model-written contracts and discovery materials, judging whether each provision achieves what it claims.
• Assess privilege calls in model output and determine if the legal reasoning holds.
• Check citations in model answers to confirm each source supports the proposition it backs.
• Identify where a model states the law in a sound way but would still lose because of the forum, the standard of review, or facts the argument cannot carry.
• Write clear explanations of your grading decisions so project leads can see the basis for each score.
• Flag unclear instructions or gaps in a fact pattern before work begins.

Requirements5

• Hold a J.D., LL.B., or LL.M. and a current license to practice.
• Bring professional experience in litigation, transactional work, or regulatory practice.
• Explain legal reasoning in writing that another lawyer can follow.
• Handle ambiguous tasks and flag unclear instructions instead of guessing.
• Show direct drafting and analysis work you completed, recent enough to discuss why you reached your conclusion.

Who Should Apply

Licensed attorneys with hands-on litigation, transactional, or regulatory backgrounds fit this pool best. The work suits lawyers who drafted motions, briefs, contracts, or discovery responses and can walk through the choices they made. Judicial clerks, in-house counsel, and enforcement lawyers often match, though firm prestige matters less than your own drafting and analysis. Candidates who lack a current license, have not written legal analysis in years, or want a steady full-time employer will find this setup a poor match. Applications often score low when a resume lists practice areas without showing specific documents the candidate wrote or when the AI interview answer stays vague about forum, standard of review, or privilege analysis.

Salary Insight