Linguistics Expert - AI Model Evaluation
Listing checked September 22, 2026 · pay as published by AfterQuery
Overview
AfterQuery seeks linguists to craft language analysis scenarios that frontier AI labs use to measure how models handle syntax, semantics, and phonology. The work stays inside your subfield, and the team judges it on analytical rigor instead of volume. You set your own schedule, commit at least 10 hours each week, and can adjust that number as your availability changes. Pay runs weekly through Stripe at a rate between $50 and $100 per hour. Your reference analyses and grading decisions shape how future AI systems interpret language.
What You'll Do4
- 1Design linguistic scenarios that probe model reasoning across syntax, semantics, and phonology.
- 2Compose reference analyses that lay out the argument for every judgment you make.
- 3Evaluate model output for soundness of analysis and how it handles the underlying data.
- 4Identify outputs that sound plausible but misread the data or attach the wrong label to a phenomenon.
Requirements8
- 1Advanced degree in linguistics or a field that sits close to it.
- 2Two or more years of work in research, teaching, or applied linguistics.
- 3Deep knowledge of one subfield, such as syntax, semantics, phonology, or sociolinguistics.
- 4Clear written English for documenting linguistic analysis.
- 5Availability of at least 10 hours each week.
- 6A PhD along with peer reviewed publications.
- 7Background in computational linguistics or corpus methods.
- 8Experience with fieldwork or documentation of understudied languages.
Who Should Apply
The ideal candidate holds an advanced degree in linguistics, brings at least two years of research or teaching experience, and has real depth in one subfield like syntax or phonology. You write clear English and can explain the reasoning behind a linguistic judgment. This role suits someone who wants part-time, remote work that stays inside their specialty and who can commit 10 hours each week. Less suitable candidates include generalists without a focused subfield or people who cannot document their analysis in precise prose. Applicants often score low when they submit generic writing samples or fail to show the argumentation behind their judgments. Another common rejection reason is lacking the stated two years of experience or an advanced degree.
Salary Insight
The listing sets pay between $50 and $100 per hour, with the exact number based on your experience and background. For contract linguistics work focused on AI evaluation, that range sits on the higher end and matches candidates with a PhD, peer reviewed publications, or computational linguistics skills. Weekly pay goes through Stripe.
Pay and demand for Machine Learning & AI roles
AggregatedTypical pay
$58/hour
This role
$50–$100/hr
Most Machine Learning & AI roles pay $30–$88 per hour. This role's pay falls inside that range.
Based on 735 similar roles that publish pay · 184 publish only a top rate; those count at the rate they gave
Rates shown per hour. Yearly and monthly pay converted; one-time fees and non-USD pay are not included.
- Live similar roles
- 828
- Listed in last 30 days
- 434
- Remote
- 98%
Hiring most right now: micro1 (325) · SME Careers (157) · Mercor (114)
Most requested skills · share of roles
- llm evaluation12%
- localization11%
- ai training10%
- translation quality review10%
Figures from Machine Learning & AI roles live on NearSkill when this page loaded. A role can close before you apply, so check the listing itself.
Compare your resume against these rolesLocation
Compensation
$50–100/hr
Required Skills
Application Tip
When you apply, show your subfield depth with a focused writing sample, such as a reference analysis that includes the argument behind each judgment. Point to any corpus methods or fieldwork experience if you have it, and state your weekly availability since the role requires at least 10 hours.
See NearSkill jobs more often in your search
Application & verification flow
1Instant rubric match
Your resume is scanned against this role’s requirements to check qualification fit.
2Screened before the employer sees it
Only profiles that clear screening are passed on.
3Outcome by email
We notify you at the address on your resume once the screening is reviewed.
Similar open positions
Explore active roles that match your skills and interests.
Handshake AI
VerifiedLinguistics Expert - Remote Fellowship
Handshake AI hires experienced linguists and computational linguists for flexible, part-time contract projects that improve how AI models handle language. You bring hands-on work in linguistic analysis, corpus methods, or NLP tooling to judge AI-generated answers and write feedback that reflects how a working linguist reasons. Assignments run on an ongoing, project basis, so you can add them alongside another role. There is no fixed schedule and no minimum hour target. The fellowship pays $75 per hour and requires US work authorization.
Handshake AI
VerifiedLinguistics Expert - Remote India Contract
Handshake AI seeks linguists and computational linguists for ongoing, project-based work that fits around a current role. You will create expert-level problems and judge AI responses in areas such as morphology, syntax, corpus methods, and phonetics. The position is remote, part-time, and open to candidates with Indian work authorization. Your feedback gives models a clearer picture of how practicing linguists analyze language. No set schedule or minimum hours, so you choose when to contribute.
Micro1
VerifiedLinguistics Specialist for AI Quality Evaluation
Remote, contractor role for Linguistics Specialists focused on AI quality and evaluation. You’ll apply linguistic analysis to assess written content and AI responses, ensuring accuracy, clarity, and coherence. The work emphasizes precise grammar, usage, and meaning against detailed guidelines, with a four-week initial sprint and potential extension. U.S. English is essential, and the project welcomes specialists who can dedicate at least 20 hours weekly.
Mercor
VerifiedEnglish Language Expert - Remote AI Project
Mercor needs language specialists for an AI research effort with a top research lab. The work centers on large language models: you check how well they summarize, hold dialogue, correct grammar, rewrite text, and reason in English. You also build and refine English linguistic data that improves what these models understand and generate. Your edits, annotations, and error reports go straight into model training.
Mercor
VerifiedSpanish Linguistic Expert (Remote, AI Lab Project)
Mercor is building a Spanish-language dataset and evaluation set for a large language model created with a leading research lab. You will judge model output in tasks such as translation, summarization, dialogue, and grammar correction, then improve Spanish text so it reads as natural and meets cultural expectations. The project also calls for linguistic annotation of grammar, syntax, and meaning, plus written notes that help AI researchers see where the model fails. Work happens remote and pays by the hour, so you can fit the project around other commitments. Your feedback shapes how the model handles Spanish across regions and contexts.
AfterQuery
VerifiedContent Writer & AI Evaluation Specialist - Remote
AfterQuery seeks writers who can both produce polished copy and judge model output across language, audio and video projects. The contract runs 40 hours a week for 3 to 6 months, with remote work and pay between $35 and $50 per hour. You will craft prompts, responses and explanatory text that train and test AI systems, then assess those systems for fluency, factuality, creativity and bias. AfterQuery is a YC-backed startup and works with a leading AI partner, so your writing shapes how models handle nuanced language. Strong prompt engineering skill and a background in English, linguistics or computer science will help you succeed.