A character-level transformer built from first principles

A decoder-only transformer implemented from the ground up in PyTorch — token and positional embeddings, scaled dot-product attention, multi-head attention, feed-forward blocks, residual connections and layer norm, all written by hand rather than imported.

I built this because using attention and understanding attention are different skills. Working through the tensor shapes, the causal mask, and why residual connections matter for gradient flow taught me more about how these models actually behave than any amount of high-level API work would have.