Red-Teaming QA Lead for AI Safety and Evaluation

What You'll Do10

• Conduct ongoing quality monitoring of red-teaming submissions and provide actionable feedback to contributors.
• Evaluate adversarial prompts, model outputs, risk classifications, and safety analyses for accuracy and usefulness.
• Update trainers and QA reviewers on updated guidelines and project changes via Discord messages and trackers.
• Respond to questions about risk categories, adversarial strategies, policy boundaries, and rubric interpretation with clear, timely answers.
• DM inactive contributors to encourage re-engagement and track follow-ups.
• Create and maintain red-teaming project documentation, including style guides, FAQs, calibration tasks, and onboarding materials.
• Lead onboarding and training calls to align teams on project expectations and review standards.
• Ensure consistent application of red-teaming and safety-review guidelines as projects evolve.
• Flag unsafe, low-quality, or poorly documented items for remediation.
• Identify recurring quality gaps and propose scalable QA improvements.

Requirements10

• bachelor’s, master’s, or professional experience in computer science, cybersecurity, ai safety, trust & safety, public policy, psychology, linguistics, law, security studies, risk analysis, or related field.
• strong English to follow guidelines, communicate with teams, and provide clear written feedback.
• 3+ years in ai safety, red-teaming, cybersecurity, trust & safety, content policy, risk analysis, adversarial testing, model evaluation, content moderation, or related workflows.
• deep understanding of ai risk categories, adversarial prompting, jailbreak patterns, harmful-content taxonomies, misuse scenarios, policy interpretation, model behavior, and safety evaluation principles.
• ability to evaluate red-teaming content against rubrics and identify issues such as weak adversarial design, unrealistic scenarios, poor risk categorization, policy misinterpretation, unsafe outputs, or superficial vulnerability testing.
• familiarity with prompt injection, social engineering, cybersecurity abuse, fraud, self-harm safety, extremist content, misinformation, privacy risk, illicit behavior, bias, and model refusal behavior is preferred.
• experience leading or supporting remote teams of red-teamers, reviewers, policy analysts, annotators, researchers, or QAs is strongly preferred.
• comfortable working in fast-moving remote environments using Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
• highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, calibration tasks, and documentation.
• experience with ai training, llm evaluation, safety evaluations, content moderation qa, policy qa, or rubric-based review is a strong plus.

Who Should Apply

Candidates with 3+ years in ai safety or red-teaming and a track record leading remote QA or policy-review work fit best. You should deeply understand ai risk categories and how to apply detailed rubrics to assess adversarial prompts. You thrive in a fully remote, fast-moving setting and can coordinate distributed contributors using Discord and Google Sheets. This role is less suitable for applicants who prefer on-site work or lack experience coordinating remote teams and policy interpretation.

Salary Insight