Writing

My next build-in-public project: an LLM inference server from scratch. Starting with the one idea the whole thing rests on, why batching is the whole game, derived on my own GPU.

My first chunker passed every test I wrote, then scored 85/100 on a quality metric I built for it. Here is how I got it to 100, and what chunking actually decides in a RAG system.

I benchmarked five embedding models on my own notes instead of a leaderboard. The 1.5B model took nine hours to tie a free one I run in under a minute. Here is why.

The actual vector search is four lines. Figuring out whether it worked took the rest of the post: a broken eval, two popular fixes that flopped, and a CTO's comment that beat everything I built on my own.

The piece I built the whole project to understand, and it turned out to be about twelve lines: one tool, a while loop with a stop condition, and the unglamorous guards that make it something you would actually run.

Eval turned out to be the quiet main character of the whole series: two numbers, a test I had to make honest before I could trust it, and the thing that settled every call by measurement instead of vibes.