Senior Software Engineer – LLM Evaluation (US-based)

What You'll Do7

• Curate code examples and build solutions in multiple languages, including Python, C/C++, Rust, Go, Java, and JavaScript (with ReactJS).
• Evaluate and refine AI-generated code with attention to systems-level correctness, performance, and reliability.
• Collaborate with cross-functional teams to improve AI-driven coding solutions against industry benchmarks.
• Build agents that verify the quality of systems-level and infrastructure code and detect error patterns.
• Formulate hypotheses about stages of the software engineering cycle (e.g., prototyping, architecture design, API design, production, launch, experiments, monitoring, maintenance) and assess model capabilities on them.
• Design automated verification mechanisms for software engineering tasks.
• Provide clear, structured written rationales for evaluations.

Requirements6

• 3+ years of software engineering experience.
• Strong expertise in systems programming, infrastructure, or backend development using languages such as Python, C/C++, Rust, and Go.
• Experience building and deploying scalable, production-grade software.
• Deep understanding of software architecture, design, development, debugging, and code quality assessment.
• Excellent oral and written communication skills, with the ability to produce clear evaluation rationales.
• Must be based in the United States.

Preferred Qualifications2

• Experience at frontier AI companies (e.g., OpenAI, NVIDIA, Databricks, Palantir, Snowflake).
• Graduation from leading programs such as Harvard, Columbia, Princeton, Yale, or University of Pennsylvania.

Who Should Apply

The role suits engineers with at least three years of experience in systems programming, infrastructure, or backend development, particularly those proficient in Python, C/C++, Rust, and Go. Candidates who have worked at frontier AI labs or on performance-critical systems will find this a strong fit. Exceptional skill and experience matter more than pedigree.