Mathematics Expert - AI Evaluation Project

What You'll Do5

• Design expert-level mathematics problems and scenarios that reflect genuine proof, formalization, and computation practice.
• Evaluate AI-generated responses for technical accuracy, mathematical rigor, and fit with how researchers solve problems.
• Write structured feedback that points out errors, gaps, and reasoning flaws in model outputs.
• Work at your own pace without a fixed schedule or minimum weekly hours.
• Use your sub-domain depth to pressure-test model reasoning on problems that fall outside current AI capability.

Requirements12

• PhD in Mathematics, pure or applied.
• First-author publications and a record of formalizing proofs that AI models struggle with.
• Depth in proof and formal methods such as Lean, Coq, or Isabelle.
• Depth in symbolic computation with SageMath or Mathematica, optimization, or numerical analysis.
• Depth in a core subfield such as algebra, topology, number theory, or combinatorics.
• Meeting two of the three main qualifications also qualifies you to apply.
• Familiarity with theorem provers and symbolic computation tools.
• Experience formalizing proofs or tackling problems at the edge of current AI ability.
• US work authorization.
• Strong written communication and attention to detail.
• Ability to explain complex mathematical concepts in clear terms.
• Ability to work on your own in a remote, asynchronous environment.

Who Should Apply

The strongest applicant holds a PhD in pure or applied mathematics and can point to first-author publications. They also formalize proofs in Lean, Coq, or Isabelle, or show depth in symbolic computation, optimization, numerical analysis, or a core subfield such as algebra or topology. The role suits someone who can explain a tricky proof in plain English and write precise, structured critiques of model output. The role fits less well if you lack US work authorization or have no hands-on proof formalization to show. Applicants often score low when their application lists math coursework but no first-author research, or when they cannot describe a specific problem where current AI models fail.

Salary Insight