Software Engineer (Scala + ML Systems)

What You'll Do8

• Build and maintain the orchestration layer that routes ML traffic for online inference and model evaluation.
• Design service APIs and request-routing paths, including gRPC endpoints and queue-based flows.
• Deploy, scale, and troubleshoot services on Kubernetes, covering resource management and live production debugging.
• Instrument the platform with metrics, tracing, and alerting, then carry on-call ownership for it.
• Profile performance and tune batching, latency, and throughput trade-offs alongside model serving partners.
• Work with Applied Scientists, Data Enablement, model serving and LLM hosting groups, and product engineering teams across Etsy.
• Shape platform interfaces so product teams onboard models and features without needing a guide.
• Weigh design decisions against the buyer and seller experience that sits downstream of every request.

Requirements7

• 5 or more years of software engineering experience building and operating distributed, low-latency production services.
• Working knowledge of Scala or another functional language, including immutability, composition, and effect or concurrency libraries.
• Clear grasp of production ML inference: model serving, feature retrieval, batching, and latency versus throughput trade-offs for predictive and transformer-based models.
• Hands-on Kubernetes work covering deployments, scaling, resource management, and debugging running services.
• Track record designing service APIs and request-routing or orchestration systems.
• Operational strength in metrics, tracing, alerting, on-call ownership, and performance profiling.
• Product mindset and clear communication with the internal users who depend on the platform.

Preferred Qualifications5

• Cloud infrastructure experience, ideally GCP.
• Model serving frameworks and LLM inference infrastructure such as vLLM, Triton, or KServe.
• Experimentation platforms, feature stores, or online evaluation systems.
• GPU capacity management and cost optimization.
• Scala libraries and frameworks such as Cats Effect, ZIO, Akka/Pekko, or http4s.

Who Should Apply

Backend engineers with 5 or more years on distributed, low-latency systems who know Scala and have run ML inference in production. Kubernetes operations and observability work should already feel routine. The role suits someone who treats a platform as a product and enjoys talking with the scientists who use it.

Salary Insight