(Senior/Junior AI Inference Reliability Engineer, Compute Infra

We're looking for a software engineer, reliability-minded but still a builder, to own what happens in production to the Kubernetes-native control plane that provisions and runs our GPU inference fleet. The platform team builds the manifest-driven API and controllers that let the inference team declare what they need, a cluster, a model deployment, a capacity change, and get it reconciled underneath. Your job is to keep that promise true in the real world: triaging InfiniBand issues when prefill and decode nodes stall exchanging KV cache over RDMA, diagnosing hero-model and LoRA-serving faults that don't show up in generic health checks, and extending the automated self-healing system so each new failure mode becomes a permanent fix instead of a recurring page.

The core value we're after is keeping the decoupling real even when things break, and keeping the fleet efficient without that efficiency becoming a new source of instability. You'll own SLOs and error budgets for inference serving, lead incident response, and turn what postmortems teach you into actual changes in the controllers and automation, not just updated runbooks. You'll also work the reliability side of capacity efficiency, since a mistuned defragmentation or bin-packing pass is itself a production risk. Strong candidates have hands-on production-reliability experience on large-scale distributed or GPU-serving systems, are comfortable troubleshooting InfiniBand/RDMA and Kubernetes controller/CRD-based platforms, and think about GPU infrastructure, and the tools that keep it reliable, as software to be engineered.

A product mindset applies: the on-call experience, the clarity of an alert, and the judgment built into an automated remediation are things you ship and iterate on.

We offer competitive compensation, startup equity, health insurance and other competitive benefits. The US base salary range for this full-time position is: $160,000 - $280,000 + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.

Please see our privacy policy at together.ai/privacy.