Coding Research Technical Staff

What You'll Do8

• Create and maintain evaluation frameworks for coding agents, covering benchmark specs, scoring rubrics, and quality standards.
• Drive research projects from start to finish that measure and enhance coding model performance on varied software engineering tasks.
• Produce high-quality datasets, golden examples, and evaluation protocols for reliable assessment of frontier coding systems.
• Examine model behavior and failure modes to spot systematic weaknesses and turn those findings into concrete improvements for training and evaluation.
• Build tooling and infrastructure for large-scale experimentation, data generation, review workflows, and evaluation pipelines.
• Set best practices for coding-agent assessment that ensure methodological rigor, reproducibility, and measurement quality.
• Work with researchers, engineers, and applied AI teams to design experiments and evaluate emerging model capabilities.
• Contribute to technical reports, benchmark studies, and client-facing research that communicate model performance and insights.

Requirements8

• Strong software engineering background with expertise in Python, C++, or similar languages.
• At least 3 years in software engineering, machine learning, AI research, evaluation, or related technical fields.
• Experience designing, reviewing, or validating technical assessments, benchmarks, coding tasks, or evaluation methodologies.
• Familiarity with large language models, coding agents, reinforcement learning, model evaluation, or related AI systems.
• Proven ability to build tooling, automate workflows, and improve technical processes through systematic experimentation.
• Strong analytical skills to investigate model behavior and derive insights from complex technical systems.
• Excellent written and verbal communication, including the ability to explain technical findings to diverse audiences.
• Comfortable working in fast-moving research settings with significant ambiguity and shifting priorities.

Who Should Apply

This role suits engineers and researchers who have spent at least three years building or evaluating AI systems, especially those with hands-on experience in Python or C++ and a track record of designing benchmarks or evaluation protocols. Candidates who enjoy digging into model failure modes and turning those insights into better training data or scoring methods will find the work rewarding. The position is less ideal for someone who prefers stable, well-defined tasks or who has not worked with LLMs, coding agents, or evaluation methodologies. A common reason applicants get rejected is lacking concrete examples of building evaluation frameworks or tooling for large-scale experiments. Another is failing to show how their work directly improved model performance or measurement quality.

Salary Insight