Senior Python Engineer – LLM Evaluation (US-based)

What You'll Do6

• Curate and build code examples and solutions, mainly in Python, with secondary work in JavaScript (ReactJS), C/C++, Java, Rust, and Go.
• Review and improve AI-generated code to meet standards for efficiency, scalability, and reliability.
• Collaborate with researchers and cross-functional teams to refine enterprise AI coding solutions against benchmark metrics.
• Develop agents and automated verification tools in Python that assess code quality and flag error patterns.
• Form hypotheses about stages of the software cycle, such as prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, and maintenance, and test model capabilities on them.
• Design mechanisms that automatically verify solutions to software engineering tasks.

Requirements6

• 3+ years of software engineering experience.
• Strong Python expertise with deep knowledge of frameworks, tooling, and best practices for production-grade software.
• Experience building full-stack applications and deploying scalable software with modern languages and tools.
• Solid grasp of software architecture, design, development, debugging, and code quality/review assessment.
• Excellent oral and written communication skills for clear, structured evaluation rationales.
• Candidates must be based in the United States.

Preferred Qualifications2

• Experience at frontier AI companies such as OpenAI, NVIDIA, Databricks, Palantir, or Snowflake.
• Graduation from programs with strong CS foundations, such as University of Washington, UIUC, UT Austin, University of Michigan, or Purdue.

Who Should Apply

Engineers with at least three years of software experience, deep Python skills, and a habit of writing production-grade code. The role fits someone who has built full-stack applications and can explain technical trade-offs in writing. Candidates based in the United States with an interest in evaluating and improving AI-generated code should apply.