Study
Courses, papers, and deep-dives I'm working through.
CS336 — Language Modeling from Scratch
Course: Stanford CS336 — Spring 2026
Instructors: Percy Liang & Tatsunori Hashimoto
Unit value: 5 units — implementation-heavy
“We will lead students through every aspect of language model creation, including data collection and cleaning for pre-training, transformer model construction, model training, and evaluation before deployment.”
Prerequisites
- Proficiency in Python (minimal scaffolding, order-of-magnitude more code than typical AI classes)
- PyTorch + systems concepts (memory hierarchy, GPU efficiency)
- Calculus, Linear Algebra, Probability
- Machine Learning (CS221/CS229/CS224N or equivalent)
Assignments
| # | Topic | Key Skills |
|---|---|---|
| 1 | Basics | Tokenisation, embeddings, transformer forward pass |
| 2 | Systems | Kernel optimisation, memory-efficient attention, distributed training |
| 3 | Scaling | Multi-GPU training, pipeline & tensor parallelism |
| 4 | Data | Data collection, cleaning, deduplication, mixing strategies |
| 5 | Alignment & RL | RLHF, DPO, reasoning-oriented RL |
Logistics
- Lectures: Mon/Wed 3:00–4:20pm PT (Skilling Auditorium)
- Recordings: YouTube playlist
- GPU: Cloud compute for self-study (follow-along at home)
Why this course matters
Most LLM courses treat the model as a black box API. CS336 is the opposite — you implement everything from scratch, mirroring the approach of an OS course where you build a full operating system. This gives you:
Read more →Editing normanli.net — Publishing Workflow
Quick reference for maintaining the blog at normanli.net.
Tech stack
- Static site: Hugo (v0.68.3 locally, newer on Cloudflare)
- Hosting: Cloudflare Pages (auto-deploys from GitHub)
- Repo:
normanli33/blog.normanli.net(master branch) - Theme: Custom (ChatGPT Codex design, self-hosted fonts, warm terracotta palette)
Content sections
| Section | URL | What goes there |
|---|---|---|
analysis/ |
/analysis/... |
Long-form deep dives |
projects/ |
/projects/... |
Project case studies |
knowledge/ |
/knowledge/... |
Short reference concepts |
study/ |
/study/... |
Course / personal study notes |
about/ |
/about/ |
Single about page |
Quick edit (fix a typo)
cd ~/blog.normanli.net
vim content/analysis/forecast-bias.md # edit the file
hugo --minify # verify it builds
git add -A && git commit -m "Fix typo" && git push
Cloudflare auto-deploys in ~30 seconds.
Read more →