Tuesday, November 17, 2026 San Francisco Bay Area Notes by RSS
Asim Zafir

Reliability AI Serving Infrastructure Distributed Systems

Uptime on 10 PB+ HBase platforms
99.996%
Nodes under reliability ownership
50,000+
Years in SRE and infrastructure
19
Kubernetes versions operated
1.18 to 1.34

Nineteen years keeping large-scale production and AI infrastructure available, fast, and observable

A staff reliability engineer for AI serving infrastructure and large-scale distributed systems, now building infrastructure for agentic AI.

Most recently I owned production reliability for GPU training and inference serving paths, extending observability across the serving path and jumping into unfamiliar systems mid-incident to drive resolution. I reason holistically about how systems compose and where the seams are, and partner across teams as an embedded teammate rather than an outside voice.

My track record includes 99.996% uptime on 10 PB+ platforms, reliability ownership across 50,000+ node fleets, and SLO/SLI design, incident command, and blameless postmortem practice for distributed systems.

The record in brief
YearsRoleEmployer
2026 to presentFounding Infrastructure EngineerAI Stealth
2024 to 2026Senior Staff Infrastructure SRETikTok USDS, via NERDSTOP
2021 to 2024Site and Platform Reliability EngineerNERDSTOP and stealth startups
Before 2021Reliability, infrastructure and network engineeringPalo Alto Networks, Kaiser Permanente, Cisco Systems, eBay, Yahoo!
Continued in Experience, below

Working on inference reliability, GPU networking, or large-scale distributed systems?

I'd like to hear about it.

Write to the author

Notes

Observability for agentic systems

Tool calls, traces, provenance, eval-as-SLI — what changes when the workload is a model deciding what to do next.

February 8, 2026 · 6 min read
All notes

Experience

  1. 2026 to present

    Founding Infrastructure Engineer, Agentic AI Platform

    AI Stealth
    • Building enterprise-grade infrastructure for agentic AI: eval engines and a context-aware intelligence layer wrapping coding agents.
    • Designing RLHF / RLAIF feedback loops that couple agent output quality to underlying infrastructure and reliability signals.
  2. 2024 to 2026

    Senior Staff Infrastructure SRE, AI Serving and Distributed Systems

    TikTok USDS, via NERDSTOP
    • Owned production reliability for ML training and inference serving pipelines and the data-warehouse pipeline backing them.
    • Operated and optimized TCE compute infrastructure for training and inference datacenter workloads: Ray Serve and TensorRT-LLM serving paths on a large Kubernetes fleet.
    • Led incident response and workflow optimization to scale training and inference deployment across core Kubernetes fleets.
    • Extended serving-path observability with histogram-based latency and congestion telemetry over Clos fabrics and RoCEv2 / PFC / ECN.
  3. 2021 to 2024

    Site Reliability Engineer and Platform Reliability Engineer

    NERDSTOP and stealth startups
    • Built compute, networking, and platform infrastructure from zero to one for multiple startups, scaling them from early stage through later-stage growth.
    • Designed Prometheus / Grafana / Stackdriver observability across Kubernetes services and large data platforms.
    • Administered 10 PB+ HBase infrastructure supporting 99.996% uptime objectives through proactive monitoring, capacity planning, and structured RCA.
  4. Before 2021

    Reliability, infrastructure and network engineering

    Earlier engagements
    • Palo Alto Networks, Kaiser Permanente, Cisco Systems, eBay, Yahoo! and others. Full history available on request.

Areas of expertise

AI serving and GPU infrastructure

LLM inference and training serving paths (Ray Serve, TensorRT-LLM); GPU fleets with NVLink, NCCL and GPUDirect-TCPX / RDMA; inference autoscaling; training-pipeline reliability; observability across the serving path. This is my primary focus.

Reliability and SRE

SLO/SLI and error-budget design, incident command, root-cause analysis, blameless postmortems, and alert tuning at 50k+ node scale.

ML and data-centre networking

RoCEv2 / PFC / ECN lossless fabrics and RDMA, EVPN / VXLAN, Clos topologies, BGP/OSPF/MPLS, and multi-DC operations.

Observability

Prometheus, Grafana, OpenTelemetry and Stackdriver; histogram-based latency and congestion telemetry; queryable, auditable dashboard surfaces.

Cloud and Kubernetes

Multi-region, multi-cloud (GCP/GKE, AWS/EKS, Azure/AKS); Kubernetes 1.18 to 1.34 fleet operations; Terraform, Helm, Istio / Envoy mesh, CI/CD.

Distributed systems

Sync semantics, conflict resolution and idempotency; HBase, Kafka, Hadoop and Spark; capacity planning and production debugging.

Languages

Python (primary), Bash (fluent), SQL (comfortable), Java (production) and Go (working).

Selected work

GPU inference serving reliability and observability

Designed serving-path observability for GPU inference infrastructure, surfacing latency and congestion signals previously invisible to platform teams. Histogram-based telemetry across RoCEv2 fabrics, mapped through the inference path so on-call could attribute long-tail latency to specific fabric segments and queue depth events.

RoCEv2, PFC and ECN · Histogram telemetry · Long-tail latency

Woobi, an AI observability platform

Built a session-intelligence platform inspecting AI-agent activity through timeline events and tool-call traces, turning raw activity into a queryable, auditable observability surface for agent runs.

Timeline events · Tool-call traces · Audit surface

Distributed session-state synchronization

Designed cross-harness session synchronization with explicit conflict-resolution, idempotency, and provenance semantics; operationalized as resilient background services with built-in diagnostics.

Conflict resolution · Idempotency · Provenance

Read enough? Write to the author.

Research collaboration, hiring conversations, infrastructure consulting, or a technical back-and-forth. I read every message that comes through the form.

Write to the author