
AI Benchmark & Red-Teaming Specialist
Develop challenging web-research benchmark tasks and test advanced AI models.
About this work
An analytical researcher is wanted to develop challenging web-research benchmark tasks and test advanced AI models. The work includes discovering difficult-to-find factual information; building research benchmark tasks with verified answers; testing advanced AI models and identifying weaknesses; analysing model failures and improving evaluation tasks; and reviewing benchmark tasks and maintaining quality standards. The engagement is up to 40 hours per week with 4 hours of PST overlap, for up to 4 weeks. Selection involves a job interest form and an assessment.
What you need
- At least 1 year of experience in investigative research, fact-checking, AI benchmark creation, red-teaming or technical QA
- Familiarity with AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench or Humanity's Last Exam
- Strong web research and OSINT skills
- Ability to verify information using credible primary sources
- Strong analytical and technical writing skills
- Ability to investigate AI model failures and evaluate their reasoning and search performance
