AI training & evaluation
Law Expert
Mercor
Mercor is hiring practicing attorneys to build and review evaluation tasks for AI agents.
- Work arrangement & location
- Remote
- Compensation
- $120/hour
Track this job
- Engagement
- hourly
Employer description and requirements
Role Overview
- Mercor is hiring practicing attorneys to build and review evaluation tasks for AI agents.
- Each task is set in a realistic simulated firm or legal department, with the case files, contracts, correspondence and matter records lawyers really work from.
- You make sure each task reads like a real assignment, the reference answer holds up against the record and the governing law, and the grading rubric checks what a senior attorney would check.
Key Responsibilities
- Refine AI-drafted task prompts so they read like a real assignment from a partner, general counsel or client, without giving the answer away.
- Check the reference answer against the world's files and the applicable law, and correct what is wrong.
- Write and tighten grading rubrics: pass/fail criteria tied to specific facts and rules, plus the judgment calls a non-lawyer would miss.
- Read how leading AI models attempt each task and confirm they fail for fair, defensible reasons.
- Work in your own practice area. Current areas include corporate and M&A, commercial contracts, civil litigation and arbitration, employment, real estate, intellectual property, estate planning, family, criminal and appellate, administrative and regulatory, antitrust, privacy, securities, environmental, health, and insurance coverage.
Ideal Qualifications
- 3+ years of practice as an attorney. Major-firm or in-house experience at a large company is a plus, but time in practice matters more.
- JD or an international-equivalent law degree.
- Active bar admission (or an international equivalent such as Barrister or Solicitor with advocacy rights).
- US bar admission preferred.
- Prior experience on a Mercor RL project is a plus.
Details
- Pay: $120/hour.
- You start with a short onboarding session and a paid skills assessment. Once you pass, you can claim your first task right away.
- Strong performers can move into reviewer and world lead roles.