Sr. Code Review Specialist

Mercor
Part time
Software Development
Canada
Hiring from: Canada
About The Job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: AI Coding Assistant Evaluator

Type: Contract

Compensation: $40–$85/hour

Location: Remote

Commitment: Flexible, task-based

Role Responsibilities

  • Review long-form transcripts between users and AI coding assistants.
  • Analyze the AI's logic, execution, and stated actions in detail.
  • Score each transcript using a 10-point rubric across multiple criteria.
  • Optionally write brief justifications citing examples from the dialogue.
  • Detect mismatches between claims and actions, such as saying "I'll run tests" but not doing so.
  • Work independently and asynchronously to complete tasks on your own schedule.

Qualifications

Must-Have

  • Senior or Staff Engineers with deep code review experience and execution insight.
  • QA Engineers with strong verification and consistency-checking habits.
  • Technical Writers or Documentation Specialists skilled at comparing instructions vs. implementation.

Preferred

  • Backend or Full-Stack Developers comfortable with function calls, APIs, and test workflows.
  • DevOps or SRE professionals familiar with tool orchestration and system behavior analysis.
  • Proficiency in Python is helpful.
  • Familiarity with other languages like JavaScript, TypeScript, Java, C++, Go, Ruby, Rust, or Bash is a plus.
  • Comfort with Git workflows, testing frameworks, and debugging tools is valuable.

Compensation & Legal

  • Independent service providers classification.
  • Payments issued weekly via Stripe Connect.

Application Process (Takes 20–30 mins to complete)

  • Submit your resume to begin.
  • If selected, you’ll receive rubric documentation and access to the evaluation platform.
  • Most applicants hear back within a few business days.

Resources & Support

  • For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
  • For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

,

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Tax Professional - Work from Home
Intuit
Full time
Overview Intuit est la plateforme technologique financière mondiale qui favorise la prospérité des personnes et des communautés que nous servons. Avec environ 100 millions de clients dans le monde utilisant des produits tels que TurboImpôt, Credit Karma, QuickBooks et Mailchimp,...
Software Development
Canada
Hiring from: Canada

Audio Engineering Technician
Mercor
Part time
About The Job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey ....
Software Development
Canada
Hiring from: Canada

Senior Software Engineer - Database Internals
Tiger Data (creators of TimescaleDB)
Full time
At Tiger Data, formerly Timescale, we empower developers and businesses with the fastest PostgreSQL platform designed for transactional, analytical, and agentic workloads. Trusted globally by thousands of organizations, Tiger accelerates real-time insights, drives intelligent applications, and powers critical infrastructure at...
Software Development
Canada
Hiring from: Canada