MLE Bench – ML Engineers

Overview

This contract role centers on benchmark-driven evaluation of real-world machine learning systems. You will work directly with production-grade codebases to build, run, and modify model training, evaluation, and inference pipelines. The work blends research and engineering, using frameworks such as PyTorch, TensorFlow, or JAX and collaborating with researchers to design challenging evaluation tasks. The position is fully remote and requires a minimum 20-hour week with a 4-hour daily overlap with PST.

What You'll Do

• Support MLE Bench-style evaluation tasks by working directly with real-world ML codebases.
• Develop, execute, and adjust model training, evaluation, and inference pipelines.
• Assemble datasets, engineer features, and define metrics for benchmarking and validation.
• Identify and fix bugs, refactor code, and enhance production-like ML systems for correctness and speed.
• Analyze model behavior, failure modes, and edge cases in benchmark scenarios.
• Produce clean, reproducible, and well-documented Python code for ML workflows.
• Take part in code reviews to ensure high engineering standards.
• Work alongside researchers and engineers to craft challenging ML engineering tasks for evaluating AI systems.

Requirements

• At least 3 years of experience as a Machine Learning Engineer or ML-focused Software Engineer.
• Advanced Python skills for machine learning and data workflows.
• Hands-on experience building and tuning model training, evaluation, and inference pipelines.
• Solid grasp of ML fundamentals, including supervised and unsupervised learning, evaluation metrics, and optimization.
• Experience with ML frameworks like PyTorch, TensorFlow, or JAX.
• Ability to read, navigate, and modify complex real-world ML codebases.
• Track record of writing readable, reusable, and maintainable production-quality code.
• Strong problem-solving and debugging abilities.
• Excellent spoken and written English communication skills.

Who Should Apply

The candidate who thrives here brings at least 3 years of ML engineering experience and can move fluidly between research ideas and production code. This role is less suitable for engineers who prefer model experimentation only and avoid dealing with messy, real-world codebases. A common reason candidates get rejected is struggling with the 60-minute live coding challenge, especially when asked to modify an existing ML pipeline under time pressure. Others score low on fit when they cannot articulate evaluation metric choices or fail to demonstrate strong debugging skills.

Required Skills

python · machine learning · pytorch · tensorflow · jax · model training · model evaluation · model inference · data pipelines · feature engineering · benchmarking · debugging · code review · supervised learning · unsupervised learning · optimization