Senior Software Engineer – LLM Evaluation (US-based)

What You'll Do7

• Curate high-quality code examples and solutions in Python, C/C++, Rust, Go, Java, and JavaScript (including ReactJS) for AI training.
• Evaluate and improve AI-generated code with a focus on systems-level correctness, performance, and reliability.
• Develop agents that automatically verify the quality of systems and infrastructure code and detect error patterns.
• Formulate hypotheses about software engineering stages (prototyping, architecture, API design, implementation, launch, monitoring) and test model capabilities.
• Design automated verification mechanisms for software engineering tasks.
• Work with cross-functional teams to enhance AI-driven coding solutions against industry benchmarks.
• Provide clear, structured rationales for evaluation decisions.

Requirements6

• At least 3 years of software engineering experience.
• Strong expertise in systems programming, infrastructure, or backend development using Python, C/C++, Rust, or Go.
• Experience building and deploying scalable, production-grade software.
• Deep understanding of software architecture, design, development, debugging, and code quality assessment.
• Excellent written and verbal communication skills for explaining evaluation rationales.
• Must be based in the United States.

Preferred Qualifications2

• Experience at high-growth companies like Stripe, Airbnb, Cloudflare, Datadog, or Coinbase.
• Graduation from top computer science programs such as Stanford, MIT, Carnegie Mellon, UC Berkeley, or Georgia Tech.

Who Should Apply

The role suits engineers with 3+ years of experience in systems programming or backend development, particularly those skilled in Python, C/C++, Rust, or Go. Candidates who have built scalable production software and can articulate technical evaluations clearly will thrive here.