
Senior Software Engineer — LLM Evaluation
Experienced software engineers are wanted to help train, benchmark and improve large language models through software engineering evaluations.
About this work
This role recruits experienced software engineers to train, benchmark and improve large language models through software engineering evaluations. The work includes creating and correcting code in Python, JavaScript, React, C/C++, Java, Rust and Go; evaluating AI-generated code; building verification agents; and developing benchmarks for software engineering tasks. The commitment is flexible at 10–40 hours per week, with partial overlap with Pacific Time (PST) required. The contract runs 1 month initially, with potential extensions based on performance and fit. It is a strong fit for senior engineers with substantial production experience, particularly those from established technology companies.
What you need
- Several years of software engineering experience
- At least 2 years of continuous full-time experience at a top-tier product company
- Strong full-stack development and production software deployment experience
- Deep understanding of software architecture, debugging, code quality and reviews
- Strong communication and technical evaluation skills
- Availability for partial overlap with Pacific Time (PST)


