About the Job

Join Impala AI, an innovative startup building a fully-managed LLM-inference platform, that enables data heavy enterprises to perform any AI task at any scale without limits.

We're looking for an Experienced Software engineer to join our founding team. You’ll be responsible for building and optimizing scalable cloud infrastructure solutions tailored for AI workloads. This role offers a unique opportunity to directly shape our infrastructure strategy, improve system reliability and performance, and contribute to establishing Impala AI as a leader in adaptive AI compute management.

Join us to tackle the magic that make AI tick under the hood and build the backbone powering the AI revolution.

What You’ll Do

• Design and implement the control plane that orchestrates distributed inference across cloud GPU infrastructure.
• Own backend systems responsible for job scheduling, SLA tracking, usage metering, tenancy, auth, and data coordination.
• Build robust APIs that glue together infra automation, ML workloads, and user-facing dashboards.
• Work across multiple cloud providers and BYOC environments—build systems that expect chaos and still work.
• Collaborate with infra, ML, and product teams to abstract complex infra flows into clean interfaces.
• Help define system architecture and lay the groundwork for a platform that adapts to varied customer deployments.

What You’ll Bring

• 5+ years of experience building backend systems and APIs in cloud production environments.
• Fluency in Python, with solid experience in designing clean service boundaries.
• Experience building distributed systems or control planes—Kubernetes operators, service meshes, autoscaling logic, etc.
• Familiarity with cloud-native tooling: Terraform, Docker, Helm, Prometheus, etc.