Software Engineer (Scala + ML Systems) - US Only

What You'll Do8

• Build and maintain the orchestration layer that routes ML inference and evaluation traffic for teams across Etsy.
• Design service APIs and request-routing paths, using gRPC or queues, so product teams reach models and serving infrastructure on their own.
• Operate the platform on Kubernetes, covering deployments, scaling, resource allocation, and live debugging.
• Set up metrics, tracing, and alerting, then share on-call ownership for a latency-sensitive, high-availability system.
• Profile performance and tune batching, latency, and throughput trade-offs for production inference.
• Work with Applied Scientists, Data Enablement, model serving, LLM hosting, and product engineering partners to shape platform decisions.
• Treat the platform as a product by studying internal users and improving the path from experiment to production.
• Apply generative AI and productivity tools with intent, balancing learning against deliberate contribution.

Requirements7

• 5+ years of software engineering experience building and operating distributed, low-latency production services.
• Hands-on Scala or another functional language, with comfort in functional patterns such as immutability, composition, and effect or concurrency libraries.
• A clear grasp of production ML inference: model serving, feature retrieval, batching, latency and throughput trade-offs, and how predictive and transformer-based models are deployed and evaluated.
• Practical Kubernetes experience running services, including deployments, scaling, resource management, and production debugging.
• Experience designing service APIs plus request-routing or orchestration systems such as gRPC or queues.
• Operational depth in metrics, tracing, alerting, on-call ownership, and performance profiling.
• A product mindset and strong communication skills for working with internal users to understand their needs.

Preferred Qualifications5

• Cloud infrastructure experience, ideally GCP
• Model serving frameworks and LLM inference infrastructure such as vLLM, Triton, or KServe
• Experimentation platforms, feature stores, or online evaluation systems
• GPU capacity management and cost optimization
• Scala libraries and frameworks such as Cats Effect, ZIO, Akka/Pekko, or http4s

Who Should Apply

Back-end engineers with solid Scala skills and a record of running distributed, low-latency services on Kubernetes fit this role. You should know how ML inference behaves in production, from feature retrieval to batching, and enjoy shaping a platform that other engineers build on.

Salary Insight