Red-Teaming QA Lead for AI Safety and Evaluation
Overview
As a remote Red-Teaming QA Lead, you guide quality and consistency across AI red-teaming and safety-evaluation work performed by a distributed contractor team. You review ai red-teaming outputs, assess adversarial prompts and risk classifications, and provide precise written feedback to keep guidelines aligned. You’ll maintain project rubrics, coordinate updates for trainers and QAs, and manage documentation across a fast-moving remote workflow using Discord, Google Sheets, and dashboards. This hourly contract role supports SME Careers’ AI data services and helps improve safety training data for leading models.
What You'll Do10
- 1Conduct ongoing quality monitoring of red-teaming submissions and provide actionable feedback to contributors.
- 2Evaluate adversarial prompts, model outputs, risk classifications, and safety analyses for accuracy and usefulness.
- 3Update trainers and QA reviewers on updated guidelines and project changes via Discord messages and trackers.
- 4Respond to questions about risk categories, adversarial strategies, policy boundaries, and rubric interpretation with clear, timely answers.
- 5DM inactive contributors to encourage re-engagement and track follow-ups.
- 6Create and maintain red-teaming project documentation, including style guides, FAQs, calibration tasks, and onboarding materials.
- 7Lead onboarding and training calls to align teams on project expectations and review standards.
- 8Ensure consistent application of red-teaming and safety-review guidelines as projects evolve.
- 9Flag unsafe, low-quality, or poorly documented items for remediation.
- 10Identify recurring quality gaps and propose scalable QA improvements.
Requirements10
- 1bachelor’s, master’s, or professional experience in computer science, cybersecurity, ai safety, trust & safety, public policy, psychology, linguistics, law, security studies, risk analysis, or related field.
- 2strong English to follow guidelines, communicate with teams, and provide clear written feedback.
- 33+ years in ai safety, red-teaming, cybersecurity, trust & safety, content policy, risk analysis, adversarial testing, model evaluation, content moderation, or related workflows.
- 4deep understanding of ai risk categories, adversarial prompting, jailbreak patterns, harmful-content taxonomies, misuse scenarios, policy interpretation, model behavior, and safety evaluation principles.
- 5ability to evaluate red-teaming content against rubrics and identify issues such as weak adversarial design, unrealistic scenarios, poor risk categorization, policy misinterpretation, unsafe outputs, or superficial vulnerability testing.
- 6familiarity with prompt injection, social engineering, cybersecurity abuse, fraud, self-harm safety, extremist content, misinformation, privacy risk, illicit behavior, bias, and model refusal behavior is preferred.
- 7experience leading or supporting remote teams of red-teamers, reviewers, policy analysts, annotators, researchers, or QAs is strongly preferred.
- 8comfortable working in fast-moving remote environments using Discord, Google Sheets, Google Docs, trackers, dashboards, and project management systems.
- 9highly detail-oriented and organized, with the ability to maintain style guides, FAQs, trackers, onboarding materials, calibration tasks, and documentation.
- 10experience with ai training, llm evaluation, safety evaluations, content moderation qa, policy qa, or rubric-based review is a strong plus.
Who Should Apply
Candidates with 3+ years in ai safety or red-teaming and a track record leading remote QA or policy-review work fit best. You should deeply understand ai risk categories and how to apply detailed rubrics to assess adversarial prompts. You thrive in a fully remote, fast-moving setting and can coordinate distributed contributors using Discord and Google Sheets. This role is less suitable for applicants who prefer on-site work or lack experience coordinating remote teams and policy interpretation.
Salary Insight
Pay is $100 per hour. This hourly rate aligns with mid-senior level remote contract roles focused on AI safety QA and red-teaming.
Location
Required Skills
Application Tip
Include a concise red-teaming feedback sample with a rubric-based scoring example and a brief policy interpretation note to demonstrate your precision and scope.
See NearSkill jobs more often in your search
How your application is processed
1Application received
Your resume and details are logged the moment you apply.
2ATS + eligibility screening
We check your profile against the role’s skills, seniority, and requirements.
3Employer sees qualified profiles only
Only candidates who clear screening move forward.
Similar open positions
Explore active roles that match your skills and interests.
SME Careers
VerifiedComputer Science Team Lead for QA in Remote Contract
This hourly, remote contractor role leads quality assurance across computer science and IT training programs, software content, and system workflows. You will review AI training content and LLM evaluation outputs, assessing explanations, code snippets, and troubleshooting guidance for accuracy against rubrics. Provide precise written feedback and keep trainers and QAs aligned through updates on Discord about guidelines and quality standards. A strong foundation in CS/IT, excellent written English, and meticulous attention to detail ensure effective coordination of remote contributors.
SME Careers
VerifiedR QA Lead for Remote Data-Analysis Projects Contract
This remote, hourly contractor role places you in charge of R quality assurance for AI training initiatives focused on data-analysis workflows. You will review R code, assess statistical reasoning, and verify reproducible workflows against client guidelines. Collaborate with remote trainers, analysts, and QA peers, delivering precise feedback and maintaining onboarding materials. Your guidance helps ensure clear documentation and standardized outputs that meet client expectations.
Mercor
VerifiedAI Safety Red Teamer
Join a team focused on strengthening the safety of advanced AI systems by uncovering their hidden weaknesses. As an AI Safety Red Teamer, you'll design clever prompts to stress-test models, spot dangerous behaviors, and help improve how these systems handle tricky real-world scenarios. This remote contractor role lets you work alongside top researchers while earning $70–$84 per hour.
SME Careers
VerifiedEnglish QA Lead for AI Training Data Projects
As a remote, hourly English QA Lead, you oversee quality across AI training projects, reviewing ai training data content and coaching a distributed team of trainers and QAs. You assess accuracy, clarity, tone, and instruction compliance against project rubrics, delivering precise, written feedback that guides improvements. You’ll maintain style guides and onboarding materials, and keep guidelines current for a fast-moving remote team via Discord and collaboration tools. This role partners with SME Careers and the SuperAnnotate ecosystem to elevate quality for leading AI models.
SME Careers
VerifiedLegal QA Lead AI Training Content Remote Contract
This remote, hourly contract places you as a Legal QA Lead overseeing quality across AI training programs for legal content. You’ll review AI-generated legal explanations, contract reviews, and compliance guidance, checking accuracy against project rubrics and flagging issues like unsupported citations or jurisdictional gaps. Deliver precise feedback and guide trainers and QAs via Discord, while maintaining up-to-date style guides, FAQs, and calibration tasks. You’ll help onboard contributors, document processes, and ensure outputs are well-reasoned, cautious, and aligned with client expectations.
SME Careers
VerifiedData Science Team Lead for AI Training QA Remote
This remote contract role places you as a data science QA Lead guiding quality and consistency across AI training programs. You review AI-generated content and trainer outputs, applying quality standards and providing precise written feedback to uphold guidelines. You assess statistics and model evaluation to detect issues such as data leakage or non-reproducible code. You support onboarding, maintain documentation, and help activate contributors who drift from guidelines.