Staff Software Engineer | AI Platform, LLM Gateway

One of the more substantial AI infrastructure roles on the board. You would build and run the LLM control plane at Tekion: smart routing across models and vendors, rate limits and quotas, failover, and token and cost tracking. On top of that sits a unified API and SDKs over REST and gRPC with normalised schemas, structured outputs, caching and full observability, plus safety and privacy enforced by default through content filtering, prompt and response validation and PII redaction. You would also own the agent runtime, including the tool registry, permissions and function calling. This is platform engineering for AI rather than model work, and the stated bar is eight or more years building large scale data, ML or platform systems.

Stated requirement is 8+ years building large scale data, ML or platform systems, with strong software engineering fundamentals in abstracted API design, concurrency and distributed systems. Technical requirements as published: production experience with Python plus one of Java, Scala or Go, and microservices and API design; MLOps at scale covering pipelines with Airflow or Kubeflow, tracking and registry with MLflow, CI/CD for models, A/B testing, shadow and canary releases, and online feature computation with Spark, Flink or Kafka; cloud and containers with AWS preferred plus Docker and Kubernetes, including performance, reliability and cost engineering in multi tenant SaaS; and practical ML knowledge across feature engineering, training, evaluation and drift detection, including experience deploying models. Responsibilities: build and operate the LLM control plane and gateway, covering smart routing, rate limits and quotas, failover, and token and cost tracking; ship a unified API and SDKs over REST and gRPC with normalised schemas, structured outputs, caching and full observability through traces, logs and metrics; enforce safety and privacy by default with content filtering, prompt and response validation and PII redaction; enable multi model and multi vendor use of LLMs with automated canarying and versioning; and own the agent runtime including the tool registry, permissions and function calling. A note on how the years were read: this posting opens by saying the automotive industry has not innovated in over 50 years. That is a statement about the industry, not a requirement, and automated parsers routinely mistake it for one. The real figure is the 8+ in the skills section. Location is Bangalore HQ. Good fit if you want to build AI infrastructure that other engineering teams consume, rather than train models.