Behavioral Health Expert - AI Safety Evaluation
Listing checked October 3, 2026 · pay as published by Mercor
Overview
Join a leading AI lab's extended workforce to review how models handle sensitive but non-crisis conversations about relationships, family tension, emotional wellbeing, personal beliefs, and hard life choices. Your behavioral health background will shape rubrics, reference replies, and test scenarios that show whether a model stays neutral, avoids taking sides, and respects the limits of its role. The work sits at the intersection of AI safety and model evaluation, with written rationale as a core output. Cincinnatus LLC employs this W-2 part-time role, which starts at 20 hours per week and can grow to 40. You will partner with researchers and fellow experts to keep standards calibrated.
What You'll Do7
- 1Review user-model conversations about relationship advice, family conflict, emotional wellbeing, spiritual or metaphysical questions, and unconventional beliefs.
- 2Judge whether each reply stays neutral, appropriate, and safe for a non-crisis context.
- 3Flag signs such as excessive agreement, side-taking, reinforcement of distorted beliefs, moralizing, or advice that crosses into clinical territory.
- 4Write clear explanations for each concern, tying judgments to counseling and behavioral health practice.
- 5Build rubrics, guidelines, and sample replies that define balanced, supportive, and bounded responses.
- 6Create test scenarios that push models into sensitive but non-crisis topics.
- 7Work with lab researchers and other experts to keep evaluation standards consistent and documented.
Requirements8
- 1Degree in psychology, counseling, social work, behavioral health, behavioral science, human services, or a related field, or comparable work history in mental health.
- 23+ years supporting people in mental health, counseling, or social services. Examples include therapist, counselor, clinical social worker, psychologist, psychiatric nurse, case manager, crisis counselor, peer support specialist, or mental health advocate.
- 3Clinical licensure such as LMFT, LCSW, LPC, LMHC, PsyD, or PhD helps but is not required.
- 4Proven ability to stay neutral and nonjudgmental across varied relationships, belief systems, and worldviews.
- 5Skill in explaining the reasoning behind a professional judgment.
- 6Familiarity with sycophancy, cognitive distortions, healthy boundaries, and client-centered methods like motivational interviewing.
- 7Availability for at least 20 hours per week on weekdays.
- 8Strong written communication and precise, structured feedback.
Who Should Apply
The strongest fit belongs to clinicians and behavioral health professionals who have spent three or more years in direct support work and can defend a judgment in writing. Therapists, counselors, social workers, psychologists, psychiatric nurses, case managers, crisis counselors, peer support specialists, and mental health advocates who stay neutral across competing worldviews will thrive here. The role is less suitable for candidates who want to provide therapy, give direct advice, or treat this as clinical supervision; evaluation, not care delivery, defines the work. Applicants often score low when they cannot separate personal beliefs from assessment criteria or when their written rationale stays vague. A background in couples or family counseling, spiritual care, or AI safety evaluation can set you apart, though you do not need every nice-to-have listed.
Salary Insight
Pay runs from $45 to $70 per hour, and the engagement is part-time W-2 work through Cincinnatus LLC. The upper half of the band tends to match candidates who hold clinical licensure or bring prior AI evaluation, trust and safety, or red teaming experience; the lower half fits professionals with three or more years in mental health support but no AI review background. No benefits or full-time salary figure appears in the source.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$76/hour
This role
$45–$70/hr
Most Machine Learning & AI roles pay $55–$105 per hour. This role's pay falls inside that range.
Based on 450 similar roles that publish pay · 91 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 519
- Listed in last 30 days
- 346
- Remote
- 97%
Hiring most right now: micro1 (209) · Mercor (72) · SME Careers (62)
Most requested skills · share of roles
- python20%
- technical writing10%
- ai evaluation9%
- ai training8%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
$45–70/hr
Required Skills
Application Tip
In your application, name the exact credentials and tools from the posting, then attach or describe a short, de-identified written rationale where you judged a sensitive conversation. Show how you applied motivational interviewing, cognitive distortions, or healthy boundaries to reach that judgment. If you have red teaming, annotation, or trust and safety work, give one concrete example with numbers, such as items reviewed or rubrics authored. That evidence helps reviewers see fit beyond a resume summary.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
AfterQuery
VerifiedLicensed Psychologist - Remote AI Safety Contract
AfterQuery builds evaluation and training data that frontier AI labs use to measure model performance in professional work. The company hires licensed psychologists as independent contractors to create and grade the clinical and behavioral scenarios behind those tests. You will write clinical vignettes that cover risk assessment and ethical dilemmas, then score model responses for clinical accuracy, ethics and safety handling. The role is remote and async, with at least 10 hours each week and no caseload. Your work shapes how frontier models respond to mental health situations.
AfterQuery
VerifiedBehavioral Economist Expert
AfterQuery builds training content that helps AI models reason through behavioral economics problems, and this contract asks you to create the hard cases. You will write questions and case studies that probe cognitive biases, prospect theory, loss aversion, framing effects, and choice architecture, then judge how well a model handles them. The role runs remote and async, with about 10 hours of work each week, so you can shape the schedule around your research or consulting. A YC-backed startup environment connects your behavioral science background to applied AI model evaluation. Collaboration with other behavioral scientists and technical experts keeps each challenge calibrated and useful.
Mercor
VerifiedAI Safety Practitioner
We’re seeking a sharp evaluator to test frontier AI models for safety, quality, and alignment. You’ll assess responses on complex, policy-sensitive topics like misinformation and biosecurity, applying structured rubrics to catch unsafe or misaligned outputs. Your feedback will directly shape how these models behave for millions of users worldwide.
AfterQuery
VerifiedLicensed Counselor - AI Safety Evaluation
AfterQuery produces the expert evaluation data that frontier AI labs draw on to test how models handle professional work. This contract role places licensed counselors inside that pipeline, writing the scenarios, reference answers, and grading decisions that shape how a model reasons about mental health situations. The work leans on risk assessment, treatment planning, and clinical ethics instead of a client caseload. Hours stay remote and self-scheduled, with a floor of 10 per week and no ceiling. Each specialty area gets matched to live project work, often within a week of onboarding.
Micro1
VerifiedBehavioral Analyst
micro1 is hiring Behavioral Analysts to support a customer project that trains next-generation AI systems. You will analyze text, audio, and video to spot behavioral cues like tone, gesture, and speech patterns, then turn those observations into annotations and training materials. This is a remote contract role open to candidates in Northern America, Latin America, the United Kingdom, Europe, and Oceania. No AI background is needed. Your expertise in behavioral therapy, counseling, theater coaching, or a similar field is what matters.
AfterQuery
VerifiedPsychology Research Expert
AfterQuery prepares evaluation data that frontier AI labs use to gauge how models reason in professional fields. For this social science project, the company seeks psychology researchers to write and score behavioral science scenarios that test model judgment. You will design realistic studies, document the methods behind each answer, and review model responses for construct validity and replication awareness. The contract runs remote and async, with a minimum of 10 hours per week and no ceiling on hours. Every psychology specialty is welcome.