LLM context rot isn't a bug. It's a mathematical property of how the attention mechanism works. Three interacting failure modes (softmax's sum-to-one constraint, RoPE's distance decay, and causal masking's primacy bias) combine to produce the U-shaped attention curve where models attend well to the beginning and end of context but lose the middle. Understanding the mechanism changes how you structure prompts and manage context.
Read articleLLM context windows have a marketed size and a useful size, and they're not the same number. Research shows quality degrades gradually based on absolute token count, not fill percentage, and a 1M-token model at 40% full is far more degraded than a 200K model at 40%. Here's the research behind the Smart Zone, the Warn Zone, and the Dumb Zone, plus a Claude Code statusline that tracks which zone you're in and how deep.
Read articleOpus 5 scores 96.0% on SWE-bench Verified vs Opus 4.6's 80.8%, a 15-point gap. But on blind user-preference coding leaderboards, Opus 4.6 ranks 4th (ELO 1547) while Opus 5 ranks 11th (ELO 1530). Synthetic benchmarks and human preference are telling opposite stories. I ran a two-round adversarial research workflow to figure out which signal to trust and landed on a mixed-model strategy.
Read articleThe retrieval pipeline returned chunks that mentioned vaccination schedules but didn't contain the actual schedule data. The model saw evidence that was on-topic but empty of the answer - and correctly refused. The root cause wasn't the model, the reranker, or the query. It was the chunking strategy splitting tables and structured data away from the prose that named them.
Read articleSame question, same branch, same retrieval, same 10 evidence chunks with identical rerank scores - and one deployment refused while another answered with full clinical detail. The root cause wasn't retrieval. It was the LLM making a borderline judgment call on evidence that covered half the question well and the other half barely at all.
Read articleVector search finds the right chunk but ranks it badly and waves through near-garbage that's vaguely on-topic. Reranking fixes that with a second model - a cross-encoder that reads the query and each chunk together instead of comparing two frozen vectors. The whole trick is affording an expensive judge by only running it on the cheap stage's survivors.
Read articleA talk I gave to the Airvet engineering org on working with AI coding agents - the mental model, context engineering, and the Research → Plan → Implement loop that ties them together.
Read articleA talk I gave to the Airvet engineering org as a primer on AI - what GPT and RAG actually are, how neural networks, tokens, and embeddings fit together, and why the difference matters.
Read article