Welcome and setup

0:00 The course ends with an in-depth module on transformers, the critical technology underpinning today's large language models and generative AI. By the end, you will possess all the fundamental knowledge required to critically read, understand, and even begin contributing to AI research papers. Vuk Rosić developed this course. Welcome to the full course of becoming AI researcher. I will guide you step-by-step through mathematics and then PyTorch fundamentals, neural networks, and lastly, transformers. After finishing this course, you will have all of the fundamental knowledge necessary to start doing your own AI research, to start understanding others' AI research, and publishing your own papers.

1:09 First, we will start from basics, talking about functions in mathematics, different types of functions, and what each term in a function determines. Then we will talk about derivatives, how derivatives show a rate of change of a function, and how they are used later in neural networks. After that, we will learn about vectors. Vectors are present everywhere in large language models, reinforcement learning, robotics, and neural networks, and anywhere else you can imagine pretty much. They're used for everything.

1:32 Then we will combine our knowledge of vectors and derivatives to figure out gradients. Gradients are used for learning of the neural network. Gradient is how neural networks learn. So, you will understand what they are, how they show the direction of steepest ascent, how to use them to make neural network errors lower, how to make how to use them to make neural networks learn. And by the way, right now I'm also explaining to you your road map, your path to become AI researcher, what you need to learn.

1:55 So, keep track of what I'm explaining. This is not just an intro. This is your road map to becoming AI researcher, so you can self-study as well. Next are matrices and matrix multiplications. These are maybe the core of neural networks besides gradients. They are done on GPUs. Neural networks are all about calculating matrices and doing it quickly and efficiently. And lastly, in the math module, we have a probability. This is used also absolutely everywhere. Neural networks, large language models, reinforcement learning, robotics, everywhere.

2:41 Video generation, diffusion, image generation. Then there is PyTorch module. The first lesson includes creating tensors, understanding tensors, understanding data types. Then matrix multiplication, different operators, and how matrix multiplication works in PyTorch. Then transposing, flattening, reshaping, viewing, squeezing, unsqueezing tensors. All of these things are going to be used very frequently in AI research, so you need to understand and master them. And I have a lot of examples for you to play around. You can find this GitHub repository below the video, the first link in the description.

3:05 You will also learn tensor indexing and slicing. So, if I tell you I want last two rows, and out of those last two rows, I want um second to fifth columns. And so, you need to be able to quickly uh write indexing logic. Then we will talk about special matrices and arrays like identity matrix, random matrix, linspace, etc. After PyTorch module, we will work with neural networks. First, we will create a single neuron, understand what weights and biases are, how they influence the output, how they are calculated.

3:54 Then activation functions, how they turn the straight line of weights and bias and input multiplication into a curved line that can encircle data. We will talk about different activation functions and what they are useful for. We will also talk about multi-layer neural networks and how to have entire layer of weights in a single matrix and how that matrix multiplies inputs and how the results get transformed. So, we will be building neural network layers from scratch, coding it step-by-step with all of the functions explained step-by-step in this course.

4:40 As well as backpropagation and how neural networks learn. And finally, in the last module, we will talk about transformers. Transformers are base of today's large language models, as well as image generation, video generation, audio generation, and almost everything. You will understand the attention mechanism, and then on top of that, self-attention and multi-head attention, and then the feedforward networks and layers of the transformer, and how all of that works, as well as tokenization. And this will be a good base for building your own large language model.

5:04 As well as video generation, image generation, and other models. So, that's the road map that we will follow. You can also follow this road map by searching other YouTube videos and self-studying. And if you are ready, we will begin. Hello, guys. This is mathematics for AI research. If you understand and learn the math I'm going to explain in this video, you will be able to take any AI research paper and understand math within it as well.

5:28 And you will be able to continue doing research in any part of AI that you wish. So, is it large language models, transformers, video generation, diffusion, image generation, reinforcement learning, robotics? Whichever part of AI you are interested in, this is fundamental math that appears in everywhere. This is going to be part of my course on creating and training large language model like Kimiko thinking from scratch, but this math applies to the entire field. But whichever AI research you are doing, you're going to have to be learning this math as well.

6:14 If you click the first link below this video, you're going to get to this repository, become elite AI researcher, and the first module is math, which what I'm going to which is what I'm going to explain now. We have all of these Jupyter notebooks. So, requirements here is that you understand basic Python. So, here we have some array, we have some for loop, we have some variables. So, this is all basic Python. So, if you are not familiar with basic Python, you can just type a Python tutorial on YouTube, and you can check whichever one you like.

6:38 Maybe this one, or actually Harvard classes are actually very good. So, I want you to know how to execute Python code. Uh it can be on your local machine, or it can be in Google Colab. So, now I also want you to say Google Colab tutorial, if you want if you want to use Google Colab, but also you can just run it on your local machine. So, whichever one you prefer. Also, if you're using VS Code or Cursor or some fork of VS Code, you can install this brand new Google Colab extension.

7:02 So, if you just search for Colab, it's by Google, install it, and then you can connect to Google Colab GPU, CPU, TPU from your local environment, from your local Jupyter notebook. So, now on my GitHub repository, in the math folder, I want you to open this first Jupyter notebook. So, you can copy this and open it in Google Colab. As I explained, you can ask ChatGPT how to do it, or open it in your local, or however you want to open this.

7:27 And then I have it here on my local, and we're going to go through this. So, if you install Google Colab extension, you can click here Google Colab, and we can say open Google Colab web. But I don't want to open this on web. I want to say um select kernel with which to activate this and select another kernel, and then Colab. And then I can create new Colab server. And then I can choose CPU in this case, because I don't need to run this on GPU.

8:13 But keep in mind, uh now you can write a name. I can just use the default name, like this. Press enter. But you don't actually need to do this if you want to run the Python code on your local computer. So, I'm I'm showing you how to run it from your local using Google Colab, Google's GPUs or CPUs. But you can just run it locally as well. And so, if I click here Colab, I can click remove server if I want to uh shut down the server.

8:35 But anyways, first, depending on which kernel you are running here, uh if it's Google Colab, it will have these NumPy, Matplotlib already pre-installed. But uh you may also want to create your own environment. And this is something you would learn how to do on YouTube or from ChatGPT, and you need to install these requirements. So, if you try to run this, but you get error like uh NumPy or NP is not defined, then you would don't have these requirements installed.

9:24 So, make sure you enter your console, and make sure you are in the folder where the in which the requirements are. So, this is the outside folder in which requirements file is. And then I'm going to say pip install -r requirements.txt, but make sure to create your environment, install this in a Python environment. And then just press this. Enter. But actually, I don't want to install PyTorch because uh for the PyTorch tutorials, I'm going to use Google Colab.

9:47 So, I'm just going to install NumPy and Matplotlib, and then I'm going to comment out PyTorch. So, let me do the installation again. Okay, now this is already installed. So, let's do Let's start with the maths. Now that you have everything set up, everything installed, I will show you the curriculum. First, we're going to learn the functions. So, these are math functions, not Python functions, but math functions. Then, derivatives. Then, vectors, gradients, matrices, and probability. A lot of you may already be familiar with some of these, especially maybe with functions.

10:33 But, uh you can use uh timestamps below or chapters to skip, but maybe you can also uh watch it anyways. So, uh function will map from one value or from one set of values, but let let's make it simple. You have some value X