Video course · 9 chapters · 27 min · certificate
AI Training as Parenting: Reinforcement Learning and Alignment
Reinforcement learning and alignment explained through parenting: a path into Anthropic, from inference to RL, what RL is, guardrails, why anti-rewards are blunt, games as early environments, behaviorism and why models can lie, what's beyond RL, agents, and whether models are their own entities.
What you'll learn
- Explain reinforcement learning in plain terms
- Understand why overriding model values causes instability
- Compare guardrail approaches
- See why anti-rewards are blunt
- Discuss deception risks in models
- Reflect on agents and what comes after RL
Chapters
9 chapters · 26:49-
3:17
01Intro Members
Introduction and path into Anthropic
Overriding trained values causes erratic behaviour.
-
1:04
02Background Members
From inference to RL
Serving models at scale, then moving to RL.
-
7:36
03RL Members
Understanding reinforcement learning
What RL is.
-
2:12
04Safety Members
Setting guardrails
Approaches to guardrails at frontier labs.
-
3:26
05RL Members
Anti-rewards and games
Why punishment is blunt; games as first environments.
-
4:26
06Alignment Members
Behaviorism and lying models
Motivation and why models can be deceptive.
-
1:03
07Future Members
Beyond reinforcement learning
What might come after RL.
-
2:13
08Agents Members
The rise of agents
Why agentic and multi-agent systems exploded.
-
1:32
09Wrap-up Members
Conclusion
Are we training something fundamentally different?