Member of Technical Staff, Cortex Lab (applied AI research)

What You'll Do8

• Create evaluation frameworks, benchmarks, and datasets for LLM agents and complex AI workflows.
• Find, analyze, and rank model, agent, and workflow failures that show up in production.
• Run quick experiments to test hypotheses, measure performance, and boost agent reliability.
• Work with enterprise customers and researchers to evaluate, deploy, and improve AI systems in production.
• Produce high-quality training and evaluation data, task specs, and feedback loops that lift system performance.
• Build Python-based evaluation pipelines, experimentation tools, and lightweight research infrastructure.
• Team up with researchers, domain experts, and customer stakeholders to set evaluation strategies and optimization roadmaps.
• Write technical reports, benchmarks, and research insights that guide future system improvements.

Requirements8

• Solid background in applied AI research, research engineering, machine learning, or LLM-based systems.
• Hands-on experience designing and implementing evaluations for AI agents, covering tool use, multi-step workflows, and task completion.
• Strong Python skills and experience building evaluation pipelines, experimental tooling, or ML infrastructure.
• Experience developing benchmarks, datasets, evaluation metrics, and experimental methods for AI systems.
• Ability to analyze agent failures, find root causes, and turn findings into measurable performance gains.
• Understanding of LLM behavior, agentic architectures, and common reliability, robustness, and generalization challenges.
• Ability to turn ambiguous real-world problems into structured experiments and actionable technical solutions.
• Strong collaboration and communication skills for working across research, engineering, and customer teams.

Preferred Qualifications6

• Experience evaluating or optimizing AI agents in production or enterprise settings.
• Familiarity with reinforcement learning, reward modeling, or post-training techniques.
• Experience building research infrastructure, evaluation tooling, or automated experimentation frameworks.
• Prior work at a frontier AI lab or in an applied AI research environment.
• Contributions to research publications, open-source benchmarks, or AI evaluation frameworks.
• Experience collaborating directly with enterprise customers or technical stakeholders.

Who Should Apply

This role suits an applied AI researcher or research engineer with hands-on experience evaluating LLM agents and a strong Python foundation. You should be comfortable digging into agent failures, running experiments, and working with enterprise customers to improve AI systems in production.

Salary Insight