Vitor Sousa

Search

Try a topic

↑ ↓ Navigate Enter OpenEsc Close

Writing

A reading guide

Start here

If you’re new, pick the path closest to what you’re working on. The ordered paths build on each other; the others are loose collections.

Transformer internals

An ordered path from attention to a working decoder, each part derived and then implemented with tests.

  1. Attention Is All You Need to Implement
  2. Positional Encoding: Teaching Transformers to Count
  3. Building a Transformer: The Complete Forward Pass
  4. A Transformer from Raw Tensors: What 112 Tests Taught Me
Foundations

Reinforcement learning for LLMs

From RL foundations through PPO, GRPO, and GDPO to a map of the GRPO family.

  1. Part 1: RL Foundations
  2. Part 2: PPO for Language Models
  3. Part 3: GRPO
  4. Part 4: GDPO
  5. Part 5: A Map of the GRPO Family
Series overview

Contextual bandits

From the decision of whether to use one, through the theory and four algorithms, to offline evaluation and deployment.

  1. Part 1: The Decision Framework
  2. Part 2: Theory and Regret Bounds
  3. Part 3: Implementing the Algorithms
  4. Part 4: Neural Bandits
  5. Part 5: Deployment and Offline Evaluation
Series overview

Evaluation

What makes a measurement of a model trustworthy.

Recommendation and retrieval

Personalization with bandits, size recommendation, and retrieval for generation.