
Posted today
MLOps / Platform Engineer — LLM Serving & Model Routing
AI Summary
Stands up and operates the model-serving platform for an Agentic AI build, running self-hosted open-source LLMs on vLLM with a LiteLLM routing layer across on-premise, private cloud and hyperscaler environments. Owns serving performance, data-residency routing, observability and cost.
About this role
SDG Group is a global enterprise data, analytics and AI consultancy. We design and build data and AI solutions for our clients. You will stand up and run the model-serving platform behind a production Agentic AI build: open-source models self-hosted on-premise with vLLM, a LiteLLM router in front, and a model garden spanning private cloud, a regional provider and a hyperscaler region, with routing driven by cost, performance and data sensitivity.
What you will do
Deploy and operate self-hosted open-source models on vLLM on-premise GPU infrastructure.
Build and run the LiteLLM routing layer and the model-garden register of approved models and the data each may receive.
Size and tune GPU serving (quantization, KV-cache and concurrency planning) so the on-premise footprint holds at load.
Wire multi-environment inference across private cloud, a regional provider and AWS Bedrock or Azure AI Foundry, with routing rules enforcing data residency.
Own observability, fallback, cost and performance of the serving layer.
What you bring
Solid MLOps or ML platform engineering, with real LLM serving experience (vLLM, TGI or equivalent).
A model gateway or router in production (LiteLLM or similar).
GPU infrastructure depth: CUDA, VRAM and KV-cache sizing, quantization trade-offs, and concurrency.
Kubernetes and containers, CI/CD, and cloud (AWS and/or Azure); monitoring and cost control.
Nice to have
Hands-on with open-weight models such as Qwen, GPT-OSS and Kimi.
Sovereign or data-residency-constrained deployments; Terraform or other infrastructure as code; telecom scale.
Why SDG Group
Own the serving and routing backbone of a large enterprise Agentic AI platform, including real on-premise GPU and multi-cloud model infrastructure. We welcome strong candidates regardless of current location, provided they can be based in Cairo and travel as the engagement requires.
Skills
Explore related jobs
More jobs at SDG Group
Similar LLM Serving jobs
- DSenior Technical Specialist - AWS/AWS Lambda/AWS Glue/AWS SNSDatavail · Bengaluru, Karnataka
- C
AWS EngineerCaxton Associates · Bengaluru, Karnataka - L
Arquitecto AWSLogicalis Spain · Madrid, Spain - S
Consultant - AWSSoftwareOne · Mumbai, India - A
Architect AWSArete Technologies, Inc. · Cambridge, MA - E
AWS DeveloperExcyl Inc · Washington, DC
Browse these categories
Market data for devops / sre / platform engineer roles
All reports →- SeriesRole reportsOne role family at a time: how many openings, what changed this week, who is hiring, what it pays.
- SeriesSalary reportsWhat employers publish in job postings, by level and workplace. Not self-reported pay.
- Market overviewState of tech hiring, September 2026: up 4.8%Tech hiring rose 4.8% month over month in September 2026, with 411,122 new listings. Customer support and account executive roles led the growth.