
Applied Engineer, Evaluations
Build and refine evaluations used to assess frontier AI coding agents.
About this work
Software engineers are wanted to build and refine evaluations used to assess frontier AI coding agents, including Claude Code and Codex. The work includes turning completed pull requests into engineering evaluation tasks; building and validating tests and grading criteria; running evaluations across different AI models; and identifying weaknesses in AI-generated code and improving evaluation quality. This is an independent contractor arrangement with a flexible schedule and weekly payments via Stripe or Wise.
What you need
- Strong programming skills in languages such as Python, JavaScript/TypeScript, Go, Java, C++, Rust and others
- Experience working with complex codebases and understanding unfamiliar repositories
- Ability to write automated tests, edge-case checks and regression tests
- Practical experience with Git, test runners and CI tools
- Familiarity with AI coding tools and the ability to evaluate their outputs critically
- Strong analytical and communication skills
Good to know
Weekly payments via Stripe or Wise.
Report a problem with this listing