Case studies
Twelve production systems, written up the way they were built: the problem, the constraint that shaped the design, and how the system behaves when things fail. Clients stay anonymous. The engineering doesn't.
Zero-downtime blue-green deploys with health-gated rollback
A stateful app needed frequent releases without downtime, broken containers reaching traffic, or bad migrations destroying the rollback path. Only a tested image tag can be promoted, traffic moves after health passes, and a failed release rolls back before users see it.
Docker Compose · Traefik · Postgres · shell orchestrationAn agent-operated task engine
AI agents needed to work from the same queue as humans without double-claiming tasks or hiding failures. Agents can claim, heartbeat, complete, fail, and escalate work safely while humans watch the same board update in real time.
Drizzle · Postgres LISTEN/NOTIFY · SSE · contract-first APIIdempotent webhook ingestion at scale
Third-party webhooks retried, duplicated, and arrived out of order; silent loss would only surface later as broken reports. Data-loss risk moved from a quarterly investigation to continuous checks that detect gaps, repair them, and expose operational health.
Go · Postgres (SKIP LOCKED, matviews) · HMAC · advisory locksFrame-accurate video localization
Translated speech rarely matches the original duration, so naive AI dubbing drifts away from the picture within minutes. Long clips stay locked to source timestamps while the original non-vocal audio survives beneath the synthesized dub.
WhisperX · XTTS voice cloning · Demucs · FFmpeg · CTranslate2Evolutionary prompt optimization
Prompt tweaks changed output quality unpredictably, and manual testing could not explore enough variants to find better prompts reliably. Prompt candidates could be compared against held-out examples, making improvement measurable instead of anecdotal.
BLEURT · LLM APIs · evolutionary operators · NumPyA durable job queue that replaced a wall of no-code automations
A growing pile of no-code workflows became hard to observe, hard to retry, and unsafe once external API work started failing mid-run. Failures became queryable rows, retries became safe by construction, and new automations could be tested against live traffic before acting.
Go · Postgres leases · gocron · bearer + HMAC authRound-trip translation of structured documents without breaking them
Packaged document formats scatter translatable text through XML runs where careless extraction corrupts formatting or loses the path back. Translated files open like human-edited originals: same structure, same formatting, and no silent reflow or document corruption.
Python XML/ZIP tooling · multi-format CAT interchangeOrchestrating long-running inference on serverless GPUs
Long-running image-generation jobs could lose work or leak GPU memory when a WebSocket dropped or an ephemeral worker disappeared. Transient disconnects no longer killed useful work, and finished or abandoned jobs released GPU resources instead of leaving expensive workers pinned.
RunPod · ComfyUI · WebSockets · S3 offloadA multi-provider AI gateway with built-in cost accounting
Direct model-provider calls spread SDK quirks, rate limits, failures, and pricing logic across feature code. Model choice became configuration, new providers inherited common controls, and spend became visible while requests were still running.
Go adapters · gRPC / protobuf · persisted in-memory queueMulti-tenant SaaS with isolation enforced on every query
A single missing tenant filter can expose one customer's data to another in a multi-tenant SaaS platform. Tenant isolation became the default path for new features rather than a convention that could be forgotten under delivery pressure.
tRPC · Prisma · Postgres · Next.jsReal-time mirror trading against an on-chain order book
Near-real-time trade mirroring had to detect source trades quickly without replaying the same transaction or accepting bad fills. Each source trade produced at most one follower action, unsafe fills were rejected, and the process could run continuously without memory drift.
Python · Web3 RPC · on-chain CLOBA local-first dictation pipeline for macOS
Cloud dictation was not acceptable for sensitive work, while context-free local transcription produced text disconnected from the current screen. Speech stays local, latency stays low, and the pasted result can reflect on-screen context without collapsing when one OS subsystem changes state.
Swift / SwiftUI · whisper.cpp · Accessibility & CoreGraphics APIs