Senior Software Engineer – LLM Evaluation (US-based)

What You'll Do6

• Curate code examples, provide precise solutions, and correct code across Python and JavaScript (React, Node.js) stacks.
• Evaluate AI-generated code in C/C++, Java, Rust, and Go for efficiency, scalability, and reliability.
• Build agents that verify code quality and detect error patterns in full-stack applications.
• Hypothesize on steps in the software engineering cycle (prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, operational maintenance) and evaluate model capabilities on them.
• Design verification mechanisms that can automatically verify a solution to a software engineering task.
• Collaborate with researchers and cross-functional teams to improve AI-driven coding solutions against industry benchmarks.

Requirements5

• 3+ years of software engineering experience.
• Strong expertise building full-stack applications with Python and JavaScript (React, Node.js), able to work across backend and frontend codebases.
• Experience deploying scalable, production-grade software using modern languages and tools.
• Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
• Excellent oral and written communication skills for clear, structured evaluation rationales.

Who Should Apply

Candidates with 3+ years of software engineering experience, especially those who have shipped high-impact products at fast-growing companies and have strong full-stack skills in Python and JavaScript. The role suits engineers who enjoy evaluating code quality and designing verification methods for AI models, and who can commit at least 10 hours per week from the United States.