Uber's agentic dev infra is built on a decade of platform work

2026-10-14 Wed ⏱ 1 hr 34 min pragmaticengineer

YT :: https://www.youtube.com/watch?v=MDUYSMJmi8A
Original title :: Uber’s AI infra, from the inside

Gergely Orosz, a former Uber engineer, tours the internal developer tooling of present-day Uber with Uday Kiran Medisetty, a Distinguished Engineer who joined in 2015 and now leads much of the agentic AI work. The main message is that the coding harness is the easy part. What makes agents useful at Uber is the pre-AI infrastructure around it: cloud dev boxes, a monorepo build system, API registries, distributed tracing, test environment routing and capture/replay. Uday walks through the model gateway, MCP gateway, skills repo, context graph, the Minion background agent, Code Inbox, Slate, Capri, Shepherd/Fox and Wisdom, then talks about token spend, productivity metrics and how the engineer's role is changing.

Uber's early days and pre-AI tooling

Uday joined in May 2015 (around the 600th engineer) on the real-time API team, the mobile API gateway. At the time there were two services: dispatch (NodeJS) and API (Python). The old protocol was a 4-second ping returning the whole world state; the new one added REST plus server push. It was later rewritten in Go and is still among the highest-QPS services.
Hypergrowth and expansion to new countries forced teams to build services fast, which led to thousands of services. Around 2018-19 Uber standardized on Go and Java and platformized. Service count is now roughly the same ballpark (4.5k to 5k, per Gergely) even though the business is 10x larger. Engineering grew to a few thousand people across 12 global tech sites, all on one global tech stack.
Homegrown tools born from scale: uOwn (who owns an asset), design doc and incident systems, M3DB (500M metrics per second, open sourced), Michelangelo (ML platform, deep learning parts recently open sourced), Cadence (durable workflow engine, the ancestor of Temporal), Morpheus (config, feature flags and experiments with switchbacks, auto-generated analysis).
Cities are first-class everywhere: metrics cardinality, feature flags, sharding by city to regions with cross-region failover, logical deployment groups for some services.
Build and repo history: microrepos with several build systems, then one monorepo per language around 2017-18, with Bazel as the build system. That made upgrades tractable but laptops could not cope, so DevPods (cloud dev boxes, one flavor per monorepo) appeared in 2017.

DevPods and AI readiness

DevPods come up in seconds from warmed Kubernetes images with build caches and repos already cloned.
A "mega DevPod" holds all repos, so agents can work and open PRs across repositories. Claude Code, Codex and OpenCode are preinstalled, which also lets non-engineers start using them.
Standardized build and test commands made top-level AGENTS.md and CLAUDE.md files effective for most teams out of the box.
Theme: years of platform investment now serve agents as well as humans.

Foundations: model gateway, AIFX, spend tiers

Guiding principle: build blocks that are harness and model agnostic, since models and harnesses change quickly.
Model gateway (since early 2023-24, built on Michelangelo GenAI projects): one OpenAI-compatible or Anthropic-compatible API in front of every vendor, open or closed weights. Every model request at Uber passes through it, so it gives one place for audit logs, session traces, per user and per project attribution, cost tracking, spend tiers and possibly model routing.
AI Guard: middleware on every request and response with five fine-tuned security models (content, safety, comms policy) under strict latency limits. The security team can iterate on them centrally.
AIX (AI Foundations): wrapper around all harnesses (Claude Code, Codex and others), preinstalled on DevPods or installable locally. It has personas with default plugins and skills, centrally managed settings synced across environments, and a status line with tips and spend.
Spend tiers: everyone gets a default tier, then a simple manager approval moves you up, framed as a conversation about ROI and responsible use, not restriction. Interactive harnesses and background agents have separate tiers. A "cost check" plugin gives personalized tips on spend, context health and model sizing.

MCP gateway and skills

MCP gateway (started April 2025): one gateway for all tools. MCP specs are auto-generated from the Thrift and Protobuf IDL registry, so teams enable a service with only config and a security policy change. LLM-optimized descriptions shrink bloated tool definitions (3-4k tokens).
Custom MCPs can be proxied through it, and vendor tools (Google Workspace, Slack, Jira, Linear) are exposed the same way. Recently all MCPs were also projected as CLIs. Sharing a team with the API edge gateway made this quick.
Skills monorepo: central platform skills plus team skills, installed across all harnesses via the AIX plugin command. Quality gates run across all of them. In progress is collecting harness traces to generate improvement patches for skills.
About 30,000 skill executions per day. Switching skills from MCP to CLI patterns, done by a couple of engineers with AI, cut context load and improved token efficiency.

Context graph and Cortana

Cortana is an assistant (web, Slack, CLI) that began as a way to answer any question about any codebase. Traces showed it spent many turns finding grounding because information sits in silos (tasks, on-call, dashboards, docs).
The Uber context graph has 100+ node types and 130+ edge types: people, PRs, assets, design docs, features, incidents, on-call events, config changes, Jira items, data tables and queries. It is the default grounding for the assistant, so quality goes up while cost and latency go down on many question types.
Agents are good at navigating a graph, much as they are with a file system via bash. The graph platform team that runs the production graph helped build it, and it is small by their standards.
Cortana can produce a 360-degree health report on a service, and existing user permissions carry over, so users only see what they could already access.

Planning and the PRD/ERD tool

A single place for product docs, engineering docs, launch, experiment and production readiness reviews, with status and approvals, linked to Jira or Linear and fed into the graph. It serves as source of truth for humans and agents.
Planning matters because of product and country heterogeneity and physical-world corner cases. A combined plan that engineers and agents iterate on together is still early.

Minion, the background coding agent

Around 15% of production PRs go through Minion (v2). It is used from web or Slack, with a plan mode or a one-shot prompt, and runs on the mega DevPod across repos, producing one or many PRs.
It started with maintenance tasks and is widening as it learns to self-validate on web, mobile and backend.
In Slack you can mention it in a thread with the PM or EM and have it create a PR from the discussion. In support channels, on-call can ask it for the root cause of a bug and fix it on the spot.

Validation and code review

Inner loop: simple CI checks moved into one command (aifx validate: lint, coverage, build rules, templates). Agents can also validate visually (emulator, web) and against a staging deployment, using skills.
Self-healing CI per PR, with automations for auto review requests, auto merge and auto fix, all aimed at lower P50 and P90 cycle times. PR volume and CI jobs more than doubled year over year, so CI infrastructure was beefed up too.
Code Inbox: one page for PRs awaiting your review and your own PRs, with AI review results, sanity test screenshots and videos, build checks, and a coming risk score and blast radius.
Every PR is still merged only after a human decides it is ready, because incidents are expensive at Uber. If risk scoring and AI review can be back-tested with years of data, a "green lane" of auto-landing for low-risk PRs may follow. Priority zero is getting the right signals to reviewers.

Slate and Capri

Slate: a personal environment deploying your branch, with test account UIDs pinned to it. Everything upstream and downstream uses the usual staging services, and multi-service tests work. Routing relies on Jaeger trace baggage propagated through the service mesh, which took years to retrofit across services.
Agents were taught to create and deploy Slate environments, run agent-browser A/B tests of main versus the branch, and attach before/after reports, screenshots and video to PRs. The same works for iOS, Android, web and backend APIs.
Capri (capture and replay): one middleware captures production requests, responses and downstream calls, tagged by dimensions (line of business, marketplace type, terminal state).
MatchGuard builds the full set of live dimension combinations a platform supports and checks a PR's before and after delta pre-deploy, for example catching a break to cash trips in Brazil only. A bug can also be replayed on a Slate instance to see whether the fix works. Agents get both as tools for feedback loops. It supersedes older tools Deputy and Cerberus.
Edge Gateway: the single API entry point (gRPC, webhooks, GraphQL, external APIs) with its own monitoring, alerting and capture replay. Jaeger traces plus required/optional dependency tags help find the unhealthy node when diagnosing alerts, for both humans and agents.

Shepherd, Fox and Wisdom

Shepherd runs large-scale changes as "campaigns": a YAML policy (for example JUnit 4 to 5) creates PRs per module owner, deterministic via OpenRewrite or agentic via prompts and skills (for example fixing context propagation for tracing). Owners track status.
Fox builds on it for continuous code maintenance. Services enroll in skills (dead code cleanup, performance analysis, noisy logs) and get a PR weekly, generated around Sunday when CI load is low. It is opt-in for now, and some skills may become opt-out later. Maintenance-type PRs inform which new skills to add.
Wisdom is the employee bug-report system in dogfood apps. It auto-routes to the owning team and captures logs, telemetry, analytics, network requests and feature flags, then derives reproduction steps (screens visited), expected versus observed behavior.
A managed agent, Debug Assist, pinpoints the root cause with evidence and timeline, then hands off to Minion for a PR attached to the Jira task. The test device can replay the network responses the reporter got, to reproduce and then confirm a fix.

The changing role of the engineer and productivity

Agentic work means constantly hunting the next bottleneck, with both AI and systems engineering. Uday mentions "capability overhang": models can do more than current harnesses and tooling let companies use.
Productivity thinking moved from diff throughput, dev-experience surveys, NPS, incidents per diff and coverage to feature velocity. PRs are auto-tagged by complexity and category (feature, bug fix, maintenance, tests, docs), and the feature share is rising while maintenance falls. AI clusters and tags missing links to measure P50 feature time end to end and find whether the bottleneck is PRD, launch or rollout.
Skills that matter more: context engineering, writing specs and constraints, multitasking across sessions, scoping, tracking and fast verification. Distinguished engineers are hands-on, and Uday does far more PRs than a year ago.
The new primitive is setting up an agent to build and verify the work. Example: customer support policy conversion from runbooks to AI-agent policies, with auto-configuration, synthetic evals and human guardrails, went from weeks to days.
Personal tool pick: personal agents (such as the Claw-style agents) automating manual life tasks with a human in the loop.

Gergely's takeaways

Capri is the standout, much easier than the old Deputy and Cerberus for capturing and replaying sessions to reproduce bugs and verify fixes at thousands of services.
Code review is "slowly dying everywhere", but Uber keeps a human sign-off and invests in signals so engineers can tell which PRs need depth and which merit a quick LGTM.
Ten years of developer productivity investment (DevPods, Slate, validation and rollout systems) now pays off as agent infrastructure. Companies benefiting most from AI seem to be the ones that already had developer platform teams.

▶ YouTube Summaries