06 / ENGINEERING NOTES
Long-form notes on AI, engineering systems, and organizational evolution.
Tracing Karpathy's MicroGPT from characters and computation graphs through causal attention, cross-entropy, Adam, and sampling—with worked examples and diagrams of the complete learning loop.
How I implemented and trained a decoder-only Transformer on a 16GB MacBook Air, then learned why tokenization, data quality, evaluation, and inference controls matter more than a low validation loss.
