← Back to all jobs
A software engineer building evaluation tests on a laptop
AI Training & Data Remote — worldwide

Applied Engineer, Evaluations

Build and refine evaluations used to assess frontier AI coding agents.

Posted 20 Nov 2026Last checked 20 Nov 2026Open

About this work

Software engineers are wanted to build and refine evaluations used to assess frontier AI coding agents, including Claude Code and Codex. The work includes turning completed pull requests into engineering evaluation tasks; building and validating tests and grading criteria; running evaluations across different AI models; and identifying weaknesses in AI-generated code and improving evaluation quality. This is an independent contractor arrangement with a flexible schedule and weekly payments via Stripe or Wise.

What you need

  • Strong programming skills in languages such as Python, JavaScript/TypeScript, Go, Java, C++, Rust and others
  • Experience working with complex codebases and understanding unfamiliar repositories
  • Ability to write automated tests, edge-case checks and regression tests
  • Practical experience with Git, test runners and CI tools
  • Familiarity with AI coding tools and the ability to evaluate their outputs critically
  • Strong analytical and communication skills

Good to know

Weekly payments via Stripe or Wise.

Report a problem with this listing