AI Benchmark & Red-Teaming Specialist

What You'll Do8

• Investigate niche subjects to surface ground-truth facts that simple search queries cannot reach.
• Verify each fact against primary or highly credible sources, note conflicting evidence, and record exact source URLs.
• Draft clear, unambiguous benchmark prompts built to expose structural gaps in how models research.
• Run tasks against designated SOTA models to confirm they trigger the intended failures.
• Review model run logs, search logs, and intermediate outputs to diagnose the root cause of each failure.
• Separate genuine model limitations from vague or ambiguous task wording.
• Apply validated model-breaking research patterns to new domains.
• Review benchmark tasks authored by peer trainers when performance warrants.

Requirements10

• At least 1 year of experience in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
• Hands-on experience authoring, testing, or reviewing tasks for benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE).
• Intermediate-to-advanced web search and OSINT skills, including comfort with academic archives, local records, and obscure databases.
• Ability to inspect model reasoning chains, search logs, and source selection to find failure root causes.
• Precision writing: clear, unambiguous prompts backed by primary-sourced answer keys.
• Bachelor's degree or higher in a relevant engineering domain.
• Preferred: experience designing prompt-injection or reasoning-trap tasks that break SOTA models.
• Preferred: advanced OSINT practice or published investigative/academic research with deep source verification.
• Preferred: niche expertise in record-rich fields such as local government filings, regional sports history, legal archives, or historical patents.
• Preferred: prior lead reviewer, benchmark auditor, or quality owner role on AI evaluation projects.

Who Should Apply

Candidates with at least a year in investigative research, fact-checking, or AI benchmark authoring and a bachelor's degree in engineering. The role suits someone who knows benchmarks like GAIA or BrowseComp, works comfortably with OSINT and obscure databases, and can write precise prompts with primary-sourced answer keys. Prior red-teaming or prompt-injection task design is a plus.