Complete AI Training
Sign inGet my AI kit

Your job's AI kit

Get your AI kit

Tell us who you are and what you do. We show you your kit right away and email you the link: skills, prompts, AI agents, MCP servers and courses for your job.

500+ jobs ready, and we make a kit for any other job. No payment needed to look.

Share

Video course · 15 chapters · 252 min · certificate

AI Research Playbook: Transformers, LLMs, Optimizers, Open Source

A practical playbook for doing AI research in the open: career realities, cheap LLM training, simple research questions, the math roadmap, DeepSeek innovations, the Muon optimizer, weight-matrix and spectral-norm research, GPT-1 speedruns, datasets, SVD, benchmarks, residual connections and reasoning models.

Become a memberAlready a member? Sign in

What you'll learn

  • Understand the realities of an AI research career
  • Formulate simple, effective research questions
  • Follow a math and coding roadmap for AI research
  • Explain modern optimizers like Muon and spectral norms
  • Run fast LLM training experiments and prepare datasets
  • Read architecture ideas like residual connections and reasoning models

Chapters

15 chapters · 252:20
  1. 21:53 01Career Members AI research realities Lessons from a young researcher: long-term careers, citation metrics, a $5 LLM, architectures and improving training.
  2. 14:02 02Method Members Simplicity and research questions Preferring simple approaches and writing effective research questions.
  3. 16:07 03Roadmap Members Roadmap, math and an open lab An AI research roadmap: core intuitions, math, and an open research lab.
  4. 13:52 04Models Members DeepSeek innovations and optimizers DeepSeek V4 innovations and better neural optimizers.
  5. 9:30 05Optimizers Members The Muon optimizer How Muon works, research questions around it, and automating workflows.
  6. 14:41 06Research Members Weight matrix research Studying weight matrices on MNIST and LLMs.
  7. 25:04 07Theory Members Spectral norm theory Spectral norms and related research from labs like Moonshot.
  8. 10:04 08Speedruns Members Replicating GPT-1 fast GPT-1 replication and speedruns.
  9. 18:57 09Data Members Datasets and loss surfaces Preparing datasets for pre-training and SFT, and loss surfaces.
  10. 17:50 10Math Members Singular value decomposition SVD explained from simple regression to weight matrices.
  11. 18:00 11Goals Members Benchmark and leaderboard goals Building full LLMs and aiming for leaderboards with quality data.
  12. 7:02 12Thesis Members Thesis defense: Muon vs Adam A thesis analysing Muon and Adam in LLM training.
  13. 17:35 13Open science Members The open super-intelligence lab Building a fully open lab publishing code and daily videos.
  14. 26:45 14Architecture Members Residual connection theory Why residual connections matter, plus hardware context.
  15. 20:58 15Future Members Reasoning models and training records Future reasoning models, Householder transformations and breaking training records.

Jobs this course suits

Our AI checked this course against 500 jobs; these get the most out of it. Each job links to its learning path.

Research Scientists