Notes from building GenAI systems that have to survive contact with real users, and from running them on Kubernetes. Mostly the things I got wrong first. I write these for the engineers I mentor through The AI People UK, and because writing something down is the fastest way to find the hole in your own reasoning.
The demo was great. The pilot was not. Almost every problem turned out to be a retrieval problem wearing a model-shaped mask — and the fixes were unglamorous.
Read →There's no magic in the Model Context Protocol. It's JSON-RPC, a handful of verbs, and a discovery handshake. The interesting work is all in tool design.
Read →How I broke one overloaded chatbot into a coordinator and three specialists — including the part where I over-engineered it and had to delete an agent.
Read →Kubernetes assumes pods are cheap, fast to start, and stateless. GPU inference pods are none of those things. Here's what that breaks.
Read →Every cluster I've inherited had the same handful of problems. None of them are exotic. All of them cause a 3am page eventually.
Read →