Senior Software Engineer – LLM Evaluation (US-based)

What You'll Do8

• Develop and refine code examples in Python and JavaScript (React, Node.js) for LLM training and benchmarking.
• Evaluate AI-generated code across backend and frontend contexts, checking for efficiency, scalability, and reliability.
• Build automated verification agents that detect error patterns and assess code quality in full-stack applications.
• Formulate hypotheses about stages of the software engineering cycle and test model capabilities on them.
• Design mechanisms to automatically verify solutions to software engineering tasks.
• Work with cross-functional teams to improve AI-driven coding solutions against industry benchmarks.
• Provide clear, structured written rationales for evaluation decisions.
• Correct and improve code in C/C++, Java, Rust, and Go as needed.

Requirements6

• At least 3 years of software engineering experience.
• Strong expertise in full-stack development with Python and JavaScript (React, Node.js), comfortable working across backend and frontend codebases.
• Experience deploying scalable, production-grade software using modern languages and tools.
• Deep understanding of software architecture, design, development, debugging, and code quality assessment.
• Excellent oral and written communication skills for clear evaluation rationales.
• Must be based in the United States.

Preferred Qualifications3

• Background at high-scale engineering organizations such as Google, Microsoft, Apple, Amazon, or Meta.
• Graduation from leading programs like Harvard, Columbia, Princeton, Yale, or University of Pennsylvania.
• Experience with C/C++, Java, Rust, or Go.

Who Should Apply

The role suits engineers with 3+ years of full-stack experience in Python and JavaScript who have shipped production systems at scale. Candidates should be US-based and comfortable evaluating AI-generated code, writing structured rationales, and building verification tools.