Aug 7, 2026
•
4 min read
Optimization techniques
+1
Most multi-turn RAG chatbots work like this: append each turn to a message list, send the whole thing, let the model sort it out. Splitting them is a small change with a measurable payoff.
Jul 1, 2026
2 min read
+2
To work through your prompt, attention compares every token with every other token. A thousand tokens means a thousand-by-thousand grid of comparisons. Flash attention just refuses to build the grid.
Jun 28, 2026
Best case, several tokens for the price of one. Worst case, slower than doing nothing. And what you're wagering on is how predictable your output is.
Jun 24, 2026
Pop-up wardrobes, tape-to-stream labs, and story prompts powering this week’s experiments.
Jun 22, 2026
Bigger means slower: more weights to drag out of memory for every token. Want more capability? Pay for it in speed. By now it feels like physics. But there is a hack!
