Senior Python Engineer – LLM Evaluation (US-based)

What You'll Do7

• Create high-quality code examples and solutions for LLM training and benchmarking.
• Review and correct AI-generated code across backend and frontend codebases.
• Develop agents that automatically check code quality and flag error patterns.
• Build verification mechanisms to confirm correctness of software engineering solutions.
• Assess model performance on stages like prototyping, API design, and production deployment.
• Collaborate with researchers and cross-functional teams to improve AI coding solutions.
• Write structured rationales for code evaluations and model behavior.

Requirements6

• 3+ years of software engineering experience.
• Strong full-stack skills with Python and JavaScript (React, Node.js).
• Track record of deploying production software using modern languages and tools.
• Deep knowledge of software architecture, design, debugging, and code review.
• Excellent written and verbal communication for clear evaluation write-ups.
• Must be based in the United States.

Preferred Qualifications2

• Experience at frontier AI companies such as OpenAI, NVIDIA, Databricks, Palantir, or Snowflake.
• Degree from top computer science programs like Stanford, MIT, Carnegie Mellon, UC Berkeley, or Georgia Tech.

Who Should Apply

Engineers with at least 3 years of full-stack experience in Python and JavaScript who have shipped production systems. The role suits someone who enjoys reviewing code, spotting errors, and writing clear evaluations. Background at a frontier AI lab or a top CS program is a plus but not required.