Introduction

0:00 machine learning systems are rapidly increasing in size are acquiring new capabilities and are increasingly deployed in high stake settings safety for machine learning should be a leading priority in this course we'll discuss how researchers can shape the process that will lead to strong AI systems and steer the process in a safer Direction this course was developed by Dan Hendricks PhD and the center for AI safety Dan is a machine learning researcher and the director for the center of AI safety welcome to the introduction to machine learning safety this course has multiple goals one of our goals is to get you to the research Forefront in machine learning safety we'll talk about well-established areas of machine learning safety as well as some nascent research topics that are still developing this way at the end of the course hopefully you can help contribute research in the area since machine learning safety is Broad another aim is to provide you with exposures to multiple sub-disciplines in machine learning such as computer vision natural language processing and reinforcement learning a versatile skill set is needed to make machine Learning

1:09 System safer because they're so General we'll additionally talk about conceptual problems such as existential risks these are risks that could permanently curtail Humanity's future since the stakes involved with existential risks are so great they end up having a high priority in this course we'll additionally take conceptual foundations from other areas such as complex systems to think about how to make machine Learning Systems safer so consequently this course will have a bit of an interdisciplinary element as well this will give you hopefully a more solid background in thinking about how to make systems safer including future systems that maybe don't look exactly like the systems that we have today machine learning safety has multiple research areas we'll talk about four in this course there's robustness which at a high level is about building models that are less vulnerable and help them withstand hazards monitoring is about identifying hazards alignment is about reducing inherent model hazards and systemic safety is about reducing systemic hazards we'll describe what these mean

2:18 in more detail now let's zoom into robustness robustness can be described as research that aims to build systems that can endure adversarial events or extreme events so it breaks down into two different topics there's adversarial robustness and the goal of adversarial robustness is how can we build models that handle attacks that are unforeseen another topic is Black Swan robustness and the aim of that is how can we build models that are able to endure once into Century events another research area in ml safety is monitoring monitoring is about identifying hazards inspecting models and helping ml system operators some example topics in the area include anomaly detection the goal of anomaly detection is to help flag novel malicious uses and another area in monitoring is Trojans a goal of Trojans is how can we detect whether models will suddenly behave maliciously the third research area we'll talk about in this course is alignment alignment research aims to build models that can represent and also safely optimize human values these human values are often quite difficult to specify and are complex and sometimes nebulous so one thing one Topic in alignment is value learning

3:51 a goal of value learning is how can we create representations that model intrinsic Goods or things humans care about like well-being or Justice another Topic in alignment is proxy gaming how can we make sure that agents don't over optimize or gain proxy goals that they're given we're treating in this course alignment as a subset of safety this is because people could maliciously align a model and that wouldn't make us safe so there's more to safety than just alignment last let's look at systemic safety this line of research aims to reduce broader contextual and systemic risks that involve how machine learning systems are handled so the scope compared to the previous topics is generally broader we're not just considering one single system we might be considering multiple systems including how the machine Learning System interacts with say human systems or cyber systems it can vary so it's a much broader scope than the previous topics here's an example area in systemic safety which would be machine learning for cyber defense this is about making

5:04 hacking and cyber attacks generally more costly another topic is machine learning for improved epistemics a goal of this is to improve the decision making of political leaders and executives since they are influencing the machine Learning Systems the quality of their decisions and how they want to direct these models will definitely impact the ultimate safety of these models these four research areas try to bring technical research to help address some potential unfortunate future scenarios so now let's describe some of these potential long-term speculative concerns one concern that's common for high impact Technologies is weaponization malicious actors could repurpose AI to be highly destructive and this could be an on-ramp to other catastrophic or even existential risks even deep reinforcement learning methods and ml-based Drug Discovery have been successful in pushing the boundaries of aerial combat and chemical weapons respectively another concern is in feeblement and feeblement can occur if know-how erodes by delegating increasingly many important functions to machines in this situation Humanity loses the ability to self-govern it becomes completely dependent on machines not unlike scenarios in the film Wally similarly eroded epistemics would mean that Humanity would have reduction rationality due to a deluge of misinformation or there could be highly persuasive

6:40 manipulative AI systems another concern is proxy gaming here proxygaming is hazardous because strong AI systems could over optimize and game faulty objectives which could mean systems aggressively pursue goals and create a world that is distinct from what humans value lock-in could occur when our technology perpetuates the value of a particular powerful group or it could occur when groups get stuck in poor equilibria that are robust to a stems attempts to get unstuck emergent functionality could be hazardous because models demonstrate unexpected qualitatively different behaviors they become more competent so elastic control becomes more likely when new capabilities or goals spontaneously emerge deception is commonly incentivized in many systems or for many agents and smarter agents are more capable at succeeding at Deception so we can be less sure of our models if we don't find a way to make systems assert only what they hold to be true or make them completely honest power-seeking behavior in AI is a concern because power helps agents pursue their goals more effectively and there are strong incentives to create agents that can accomplish a broad set of goals therefore agents tasked with accomplishing many goals have instrumental incentives to acquire power but this could make them ultimately harder for Humanity to control so we just considered some future risks

8:18 let's look at some past developments in 2011 the book engineering a safer world was published this book proposed many new ways of thinking about accidents and its perspectives have made complex high-risk real-world systems from multiple Industries safer we'll be using this as a basis for much of our discussion of risk analysis in this course another event happened in 2012. here Alex net won the imagenet competition Alex had an imagenet error rate more than 10 percentage points lower than the competitors and this caused a renewed interest in deep learning in 2014 Super intelligence was published this book spurred interest in preparing to mitigate existential risks from Advanced artificial intelligence we'll be drawing on this book for our discussion of existential risks although we'll try to reflect more current understanding of existential risks and artificial intelligence in 2016 there were multiple new efforts toward reducing existential risks from Ai and the approach was to use Empirical research

9:29 consequently researchers proposed many speculative Empirical research directions such as negative side effects or robust human imitation mild optimization Etc several years later it's 2022 many of the research directions proposed in 2016 didn't end up turning out but now that we have much more information about what's promising and which areas are dead ends we can present you with this course where we'll discuss a refined and Consolidated collection of Empirical research directions aimed toward making machine Learning Systems safer and especially safer in the long term while taking the course there are some details to keep in mind one is that this is a research topics course so we'll expect a solid background in both machine learning and deep learning if you haven't taken either of those courses we'd strongly recommend preliminarily studying that before taking this course there are programming reading and conceptual assignments throughout this course so we're not just going to be having reading assignments as would typically happen in a research topics course

10:41 we have several modules first is Hazard analysis where we'll discuss how can we make systems in general safer and how can we understand risk then we'll jump into more specific machine learning topics such as robustness and we'll talk about monitoring alignment and systemic safety at the end we'll close by having some additional conceptual discussion about