About this position
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: LLM Red Team Specialist — Failure Modes & Edge Cases
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week
Role Responsibilities
- Probe models to explore how frontier AI models handle coding, ML, and analysis tasks. Identify spots where models quietly fail.
- Design challenges by turning identified weaknesses into well-crafted tasks that are challenging for models but fair to grade.
- Document findings clearly with reproducible evidence and steps.
- Strengthen tasks by collaborating with task authors to close loopholes, shortcuts, and grading gaps.
- Work as a team by sharing insights with researchers and fellow experts to continually improve the benchmark.
Qualifications
Must-Have
- MSc or PhD in a STEM field or equivalent practical experience.
- 1+ years of experience in research, research-engineering, security, or AI-evaluation roles.
- Ability to identify vulnerabilities, edge cases, or failure modes in LLMs or ML systems.
- Proficiency in Python and Git for scripting probes and analyses.
- Familiarity with LLM capabilities, limitations, and evaluation techniques.
- Ability to engage reliably for approximately 35 hours/week.
Preferred
- Experience in AI training, model evaluation, or benchmark/task authoring.
- High attention to detail, creativity, and strong written communication skills.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
- For any help or support, reach out to: support@mercor.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Location and commute
San Francisco, CA .
Plan your route with Google Maps. Travel times and transportation options vary; confirm the work location with the employer before commuting.
How to apply
Review the vacancy and complete the application on the employer or recruitment partner’s website. Requisition ID: WJ-3006645347.