Agency

SWE-Bench AI Task Auditor - Freelance AI Trainer Project

World Wide - Remote

hirly Pro

Get in front of the people hiring at Agency

Reach the hiring manager at Agency on LinkedIn, from your own account
hirly applies to this job for you, with unlimited applications
A resume and cover letter written for this job, unlimited
The hiring manager and recruiter at Agency, with contact details

Get Pro for this job →

$49 a month · ₹1,999 for 30 days in India · cancel any time

Get past the screening software and onto a recruiter's desk

hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.

Keywords matched to this posting
Fit score before you apply
Cover letter included

Matched against 3M live jobs from 220,000+ employers in 200+ countries.

Tailor my resume for this job →

Apply from your AI assistant

Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.

See how it works

Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.

hirly's read of this role

Role family: Finance
Seniority: Mid level
Work mode: Remote-friendly
First seen by hirly: 10 Sept 2026

Derived automatically from the posting. Upload your resume above to see how the role scores against it.

the posting

Project Overview We are sourcing experienced technical specialists in Software Engineering (SWE-Bench) to audit tasks used to train and evaluate AI systems. The objective of this project is to ensure that AI training workflows and tasks are technically rigorous, practical, and highly accurate within your specific area of expertise.

Project Deliverables & Scope

Task Evaluation: Assess whether software engineering tasks are technically accurate, realistic, solvable, reproducible, and supported by reliable tests and evaluation criteria.

Technical Auditing & Feedback: Provide clear, actionable feedback on any identified codebase integration issues, test failures, or logic errors within the tasks.

Required Expertise

Demonstrable professional experience and deep technical knowledge in software engineering (including complex codebase navigation, real-world application development, and SWE-Bench standards). (Note: Candidates will be considered for one specialty based on their experience; expertise across other domains is not required).

Strong analytical and problem-solving skills to rigorously test, troubleshoot, and evaluate complex technical scenarios.

We offer a pay rate of $60 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. As a contractor you’ll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.

Employment type: Freelance Contract

Workplace type: Remote

Original posting on Agency's site ↗

Listed on hirly, a job board. hirly is not the employer: Agency is hiring for this role.

Want this one?

Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.

Tailor my resume for this job