Tabular data classification

0:00 learn pytorch and pytorch syntax this course walks through five Hands-On exercises designed to help you understand and apply pytorch syntax in real world tasks it starts with tabular data classification then moves on to image classification using both custom and pre-trained models you'll also explore audio classification and text classification using Bert giving you a well-rounded introduction to pytorch capabilities Omar developed this course hello and welcome back to this series uh in this series we'll be practicing multiple pych exercises the goal of this series is to get used to pyto syntax since it's a bit uh complicated uh not complicated but it it requires more work to do than uh than a tensor flow so what we'll be doing here is uh this is the first exercise so what we'll be doing here is the rice type classification it will be a table classification we'll use a CSV file to do a classification with py torch uh

1:13 later in this playlist we'll be using uh we'll be dealing with uh um images text and even uh more complex exercises such as image captioning or large language models but for now we'll start by the rice type classification or the table classification uh just to know if it's uh just to to get used to the pyto syntax and we'll be going into more difficult exercis in the later videos gradually so the first uh first thing we'll be using the rice type classification data set the data set is from kagle it's an open source data set I'll attach the link in the description you can find a explanation about the data set here so we have the uh it's a binary classification data set which means that we have either uh the rise based on some features it's either jasine or gonen uh which is one or zero so um I believe maybe this is something related to good or bad quality or something like that but anyway uh we will we have uh here are the attributes or the features which are the ID area measor access length minor AIS length uh

2:25 eccentricity convex area equave diameter and multiple other things together with the class at the end um before we start coding we know that ID is something that we need to remove it's just a number that is assigned to each entry and usually IDs are being removed from the process of of uh machine learning because they are not representing anything so uh it's just a number assigned to to to one entry and uh here is the data set so let's go and download the data set with the same way we have done inside the video that we introduced how to get data set using collab so we'll be installing uh pip install open data sets uh D- quiet which means uh doesn't print the installation process it it does the installation in a silent process let's say and then we'll go into uh import open data sets as OD and OD download theise type classification data set which is just the link from KAG so let's start downloading it let's make sure first that we are connected into T4 GPU uh usually of course you know usually uh

3:36 machine learning or deep learning uh things uh using tensor flow py TCH uh better to use uh py TCH uh sorry better to use gpus because the process will be much faster the training process is much faster with the gpus um I know this is a tabular problem uh CSV file and not a very huge huge file not to be used for deep learning so GPU may not be uh a huge deal here not very important thing here so um but it's always recommended that you go into the GPU okay so now I provided the uh my username and the kaggle key so we had a detailed video about this process uh I'll attach it in the description as well if you want to know more about how we uh how we install or how we download each data set from kagle into Google collab directly okay great great now in the next cell we will do our Imports okay so let's start by doing our Imports uh we'll start by importing pytorch main library which is torch and we will start we will also uh be importing the uh NN class NN class from pyto library as NN

4:54 which means that we are getting torch.nn and giving it such like a nickname or something that we can use instead of writing torch.nn so we'll use only NN uh stands for neural network and this one we'll have all the layers that we are going to use the linear layer or the equivalent to the dense layer uh the convolution if you want it to use any the lstm or the long shorter memory the recurrent neural networks any layers will be in torch.nn okay then we will be getting the Adam Optimizer from torch.

5:19 optim import uh Adam we're going to use sorry uh Adam Optimizer and then we will be uh using also the data set and data loaders object we will know more about this shortly from torch. dat import data set and data loader then we will don't uh uh Don't Panic from these Imports maybe some of them you are not aware about but it's okay we will learn uh each one the usage of each one later from torch uh summary import summary okay this one is to print the model summary or how the model uh architecture will be at the end uh also we will import uh from escalar model selection we will import from this one a train test split okay and also from skar do metrix import accuracy score now this is a binary classification problem so accuracy would be efficient here would be enough but if it is a multiclass problem um it's strongly recommended that you use

6:56 precision and recall as it will be easier to monitor other uh to monitor the exact problem of the accuracy uh import mat plot Leb dot P plot as PLT okay so why we are going to need this we are going to need this because we want to plot the training progress that we will be doing and then we will import pandas as SPD and import nump as MP I don't remember that we will be using num but just let's keep it just in case now next thing is that you have to know that uh pytorch does not automatically detect the GPU okay so you have to specify it automatically unlike tensor flow tensor flow can detect the uh GPU automatically but uh pytorch unfortunately cannot so what you have to do is to uh to tell pyour that you have a GPU you have Auda GPU Okay so we are using Google collab here so it's by default of course it's Cuda when you choose from runtime if you choose that one it's Cuda this is a Tesla T4 GPU from Nidia this is the CPU this is

8:11 tensor point and these ones are available for Google collab Pro so you don't need this we're going to use the three ones H the fre one device equal Cuda so you are telling it the device is Cuda if torch. ca. isore available okay else CPU so what I'm doing here I'm doing is that I'm telling I'm assigning a variable called device device is Coda only if torch. Coda do is available is true so this function is functions that is built in pytorch you can go and search for for it in the documentation uh the function gives true if it's if a GPU is found and gives false if GPU is not found so what I'm doing here is that I'm assigning Cuda if this function is true else I'm assigning CPU and now let's print uh device is uh let's wait for the Imports CPU so we don't have a Cuda device available let's go and make sure that every uh it's not detected yet let's try to run again sometimes this

9:24 happens um okay connecting yes now we should run again [Music] and let's give it a moment yes now it's cool so now it's it was able to detect the GPU since this one I already changed before the start of the video but I think I didn't save so um it's it's a normal thing that could happen no worries just make sure that the run time is T4 okay of course other problems could be uh that P torch was not installed correctly but we will not go into this okay if you followed the exact description in pyto uh uh documentation on how to install it everything will be fine also if you are using Google collab no need to install the pyto it's already installed for you okay great now we will need to work on uh our data so let's start by reading our data set data DF equal pd.

10:10 read CSV and let's get the exact path of the data so so we're going to refresh here okay so we need to run this again okay that's my fault uh because we changed the Kel we changed the run time so we need to run this again okay no problem now when we run this uh okay let me check and yes we are good so here rise type classification CSV file we are going to take path of that file and we are going to use it here and then we will check the data just by printing the uh okay data DF just by printing the first five rows okay and here we go so the data is here looks good okay so um now what we will be doing next is that we will do some basic pre-processing we are not going to do some uh like a huge pre-processing we because this is not the of of this uh series we will have another uh another videos I'll upload another videos on how to pre-process the data to achieve better accuracy but for now the goal is to get used to the pytorch syntax and learn how to use pytor syntax properly okay so we will be just dropping the

11:48 missing values although I don't think there is any but let's see in place equal true so this line uh uh drops any missing values and also we need to drop the ID column since we mentioned before that uh ID column is not necessary okay it's not important in the machine learning okay so dot drop um like that and we will write the ID here and we will mention X's equal 1 which means that it's a a column not a row X is equal 1 means that it's a column in place equal true okay so this is the first step let's print the data shape as well uh data do shape after doing these preprocessing things so let's uh data is not defined data DF okay data data frame so you can see that ID is not found in axis before because we removed it be before and this line gave an error so okay no problem we can run this again we read it and we can run

13:00 this again now we are good okay so we have 18,000 more than 18,000 row and 11 column okay so we can also print data. DF head again to see the data and perfect we don't have the ID okay just uh to make sure that we are good and now let's print uh the output possibilities what are the outputs that we are having so we are expecting zero and one from the data set but let's uh also print them if you want dataor DF of class dot unique which will print these values for you one and zero also you can print if you want you can print as well the uh um uh the frequency of these values so it's value count I think value counts okay so let's see yes perfect so we have the class one is around 10,000 almost 10,000 uh row and the other class zero is 8,200 this data set is not perfectly balanced but the Gap is not that big so

14:16 we can proceed okay uh there are some techniques of course that you can handle the unbalanced data set but it's not uh it's not the scope of this lecture now let's do the preprocessing another part another important part inside the pre-processing which means which is the normalization why we want to do normalization so you can see here that the values are extremely huge okay this is not a good thing okay usually we want the values to be small because the weights of the neural network are relatively small compared to these values also uh when the weights are being multiplied by the inputs throughout the different layers of the neural network the these values are becoming very very huge which means that it will be computationally expensive slower and as well sometimes in uh when you are using the loss function to calculate the loss or the error it may uh it may round up the values okay because the values are very big so it may round up the values so it's better that we uh uh it's better that we normalize these values and by normalize we mean each column the maximum value of

15:29 each column will be one so we how we going to do this we are going to take each column all the values in each column and divide it by the largest value in that column by this we have the largest uh value will B1 okay and we won't uh we we will not lose anything in the data because uh we will be doing this on all the data okay okay great now first let's let's have original underscore data frame equal D dataor DF do copy why we are doing this I'll tell you we are first we are taking a a copy from the data frame that we have and we are having it in in a variable called Original data frame why because we will need the original values or the old values for the inference okay when we do the final inference after we finish all the exercise now let's not uh let's not not get into uh into that now uh and we can uh we can proceed and at the end you will you will know why we did that exactly okay you will see why this is

16:40 important now let's get for column in data _ DF do columns okay so we are uh going through each column inside the data set and then we are taking data Frame dataor data frame of that column so this column is equal to dataor DF of that column divide by divide by what we said the largest value in in that column then dataor DF of that column dot maximum okay it's always it's also better that you add absolute value do abs but it's not important really because these values doesn't seem to have any uh negative values but we can keep it just in case because of course we cannot see all the 18,000 uh row but um I don't think there's any negative values of course because this is something obvious according to the purpose of this data set but let's keep it like that uh if you want okay and then we are going to view the new data frame okay now let's see the result and

17:54 here we go so you see the difference between the values this is 92 now this is is a changed this is uh 64 this is a changed all of them are divided by the largest column by the largest row uh sorry by the largest value in each column okay okay perfect now let's get into uh splitting our data set okay up till now we didn't touch byor yet all of this is just dealing with the data okay so we'll have X which is the inputs can call it inputs but let's keep X for now equal uh nonp array np.

18:19 array of so we're going to convert it to npy array of the data frame uh do ilock okay colum comma colon till minus one okay what does this mean this is actually you are taking uh data frame our data frame you are taking all the I look means take specific specific part of the data frame so we are taking all the rows and we are taking all the columns except the last one this is means that okay this uh uh this colum minus one means except the last one now because we know of course that the class value is the last one so we are not taking it as an input now we will do almost the same with Y okay let me I copied it let me show you how we are going to take all columns as uh uh all rows as well but we are only going to take the last one so we'll remove

19:34 this column okay this means so basically this means all until you can think about it like that and this means only that one okay so colon means all except the last one okay okay perfect now you can run this one okay we are good now what we will do is to do uh uh splitting now we need to split the data set into training validation and testing so you know training data is for the training or for the model to learn validation is to uh validate the model during the training and monitor its progress It's the data that is the model is is uh tested on during training and testing data is the final testing after the model has done all the training okay now let's uh xtrain let's Define xtrain comma uh X test comma y train comma y test is equal to what equal to train test split train test split and we are going to take X as the

20:49 inputs uh and Y as the outputs and we will Define the test size of our data is uh sorry there's a typo here test size is uh 0.3 which means that 70% split we are going to split the data into training 70% and testing 70% by default this function returns four values great and then we can do the same okay uh for validation so we let me show you let's copy this one and first we will need X test here this is going to be X test and Y test and this one will be X validation and Y validation okay and then we will be doing uh we will be taking the X uh we will split the X test so the goal of our the goal of this split is to have 70% training 15 testing 15 validation 15 testing great so since we split already the first one for training we will split the next one for uh from testing so we split the original data into training and testing we will

22:03 take the testing which is 30% we'll split it by 50% so we will take the X uh test and Y test okay cool now let's run okay great now we can check the shapes of the data extr uh dot shape and we can print it let's print X train. shape and let's as well print X test and X validation okay and great so now we have this is 70% of our our whole data This Is 50 50 uh sorry this is 15 15 which adds up both to 30% okay great now um I want to add one thing that we have AO that is containing all the details about uh all this slab inside the GitHub REO that is already uh uploaded on GitHub and having a very detailed explanation of of all what I'm doing now okay so uh you can go through it if if you feel that anything is confusing okay now comes the interesting

23:18 part okay we will be using a data set object we will create a data set object okay why we want to do that first of all pyto think of it as as a huge ecosystem that has a lot of functions a lot of uh classes that can work together in a very seamless way okay okay great now how how py torch understands the data how py torch see the data it doesn't see it as a pandas now the data is pandas or npy array this is different objects but we want a data set a py torch data set object Okay so so we need to to to convert our data into pytorch data set object but we need to specify the details of our data set or how to access the data set okay okay now this is not complicated okay uh let's follow uh let's let's start and you will see what what I want to do okay now we will create a class the class is called data set and the class inherits the data set object that we imported above this one okay this is a class okay it's not something on on it's not a file or

24:35 something this is a whole class by itself okay now the class is having multiple functions okay we will create we will take the functions inside this class and we will overwrite or modify some of these functions okay so to match the properties of our data okay so what we are going to do exactly here is Def uncore uncore in it we will Define our Constructor that takes self what else it will take it will take the inputs and the outputs of course this is totally customizable if you want to change or play with that feel free to do so okay and X comma y so this is taking two things okay this is taking the inputs and the outputs great now uh we will have self do x equal torch.

25:24 tensor of what of X comma D type equal T I'll explain all of this don't panic okay because I know that this syntax may be a bit complicated do to device okay what did I do here okay this whole thing that seems complex it's not complex at all we defined the uh self X which means the x or the input value of that data set object will be the x that we got from the Constructor and this is fine now the X we got from the Constructor is maybe nonp array maybe a normal list maybe a pandas so we need to convert it to torch tensor so torch understands the tensors if you want to deal with by torch you need to have your data as a tensors so inside this data set object you have an inputs of x x is uh converted to torch to tensor torch to tensor is something like a list what

26:35 will be inside this list a list or a vector or a matrix but for now it's let's let's say it's a vector okay because it's a one dimension now how do we know it's one dimension you can see that this is all the inputs are one dimension it's not an image or or or a video or something and D type is torch. float 32 which means that it's a float 32 type and the last part is moving this value to device which means that we move this uh we move this list to device what is device device is the GPU the Cuda device that we defined before great now we're going to do the same exactly the same but with Y so we are going to Define here Y and here y okay okay perfect now that's it that's it for our Constructor let's go into the next function we have two functions that we need to modify from the data set object not going to very deep details but these functions will be used inside the training and how the model will be training so uh pay attention to them okay the first function is called Deathcore Lenore s

27:52 okay now I am not naming this function as uh uh whatever I want okay because as we mentioned before the function is already defined and already built inside PCH but we need to modify it to match our data set and to be suitable for our data set so we will do death Len we are going to return the length of self. X so what does it do this function basically returns how uh like the shape of uh the input okay okay great now uh def again I'm saying these functions can be customizable on your own okay but you have to understand what you're doing okay now the next one is Def get item in the same way it is written like that I again I'm not doing uh it on my own this is something that you have this is something from pyto get item means that we want to get a specific item according to the index so it takes as well index okay so maybe we need item like in this table we need item number three so we

29:04 have to Output all these values and the output okay we need the full item okay so we need all of them so what will we do we are going to return self.x of index comma self.y of index okay and that's it now we have our class ready we need to create objects for training validation and testing so let's create them let's create them in the next set so that you don't get confused okay perfect now no uh no problems now we create the training data will be data set of xcore uh sorry xcore train comma Yore train and and we are going to do the same with the validation let's change the names validation and testing and we are going to give it here xcore uh Val and Y underscore Val and then we are going to give here xcore test and Yore test okay let's see if there's any problems and uh looks good great okay now uh now we are done preparing our data set object we didn't get yet into

30:40 the data loader okay so let's start uh on the data loader now okay so let's start with the concept of data loaders now the data loader concept is a interesting concept the thing is that when you uh do the training Loop okay the the problem here is that a tensor flow is hiding or doing a lot of things for us in the back end and we don't see that and that's why tensor flow is a good thing for for for beginners but let's think about how AI is being trained how any machine learning model can be trained so basically you go through all the data pass it through the model and then uh calculate the weights do the back propagation and do all of these things okay but when you go through all the data how do you go through all the data you don't go to the data one by one you go by the batch size right okay then if we go by the batch size we need to uh go into like a groups or batches of the data which means a each batch is eight or six or four or 16

31:53 or whatever you specify the batch size so to go to the batch size of the data you create a data loader a data loader basically is an object that you use to do the for Loop during each epok uh to to to Loop through a specific amount of batches okay okay so let me uh declare three data loaders so a data loader for each uh data set that we have training data validation data and testing data so let's name train data loader equal data Maybe if if if you don't understand it completely now maybe you will understand it better when we start the training Loop training underscore data comma batch size equal now now let's specify the batch size as eight you see here by default it's one okay so let's specify it as eight okay you can specify it as whatever you want okay and Shuffle equal true we have a lot of of course other uh uh function parameters but we will only use these Shuffle equal true means it it um it shuffles the data or change the order of the data in ter in terms to

33:08 avoid biasing now let's do the same line for each data set so we're going to do validation and here we are going to do testing data loader let's call this validation better and we are going to put this here and that one here okay perfect now we create our data loaders now what is like how how still if you don't understand completely the data loader no issue so let me show you one thing so for uh let's say x comma Y in train underscore data loader print X and let's print like an equal signs and print y then break okay now what am I doing here I am looping through the the data loader so you remember we specifi the batch size is eight right then we which means that the data loader Loops through eight8 eight like uh values of like it Loops through eight uh eight data rows at once Okay so so each time eight data rows next time the other eight rows each next time other eight rows and so on so when we Loop here we Loop through X which are the inputs and Y which is the output and inside the data loaders so here I printed X and break I'm I'm just stopping at the first

34:47 Loop so that we don't get the the screen full of numbers okay so here you can see print X Y uh sorry print X it printed X which means the first row second row third row fourth row fifth row up until eight and the device is Coda zero which because we transferred already them to device and here we printed first label second label third label until eight which means that if we changed this to four okay it means that we are looping through multiples of four for the data okay like like here you can see four rows four labels you see okay that's the purpose of the data loader so we can keep it back eight and we can continue uh our uh training group okay okay perfect now what's next after this process we are going to build our own model okay so we will build the model same way as we built the custom data set okay we have a model and we will be doing the inheritance of the model okay okay let's see class my model NN do modu

36:05 don't worry we are going to explain these things now we are creating a class called my model it inherits from my nn. module which means that it takes all the functions in nn. modules and all the properties in nn. modules but we will modify the instructor of course sorry the Constructor of course and and we will we will modify as well the forward function this is really interesting okay it may seem not to be very interesting now but I will tell you why I'll tell you when we reach to the forward function let's start by the init the uh initialization or the Constructor now we will write super my model K comma self dot underscore underscore init underscore uncore brackets this means what this means that we will use the same Constructor of the NN module so here we created our own Constructor from scratch but here we will use the same Constructor of NN module but by inherit by using this super and then we will

37:18 create the layers let's create self do input underscore layer is equal what now we need to specify the input layer input layer is a a normal linear layer now linear layer is like a dense layer or a um normal multi-layer perceptron okay multi-layer perceptron uh one layer of multi-layer perceptron okay x dot shape of one what does this mean this means the number of columns of X which means that the number of inputs because the columns are the number of features which here showing in features this is the input features what are the output features the output features is the hidden neurons okay so how many hidden neurons do you want in the next layer so you think about it like that it's it's it's something sequential right it's going one by one so the input is how much the input is 10 rows uh sorry 10 columns and the output is going to how many neurons let's say 20 neurons so this is the hidden neurons or maybe better for you that we write hidden neurons and we Define hidden neurons

38:36 up as 10 let's say okay and this is the input layer now the next one is self do linear a normal multi-layer perceptron layer or we can consider this as the hidden layer linear uh of hidden neurons let wait for the okay hidden uh neurons and the output of it is what is one okay don't forget we have only one output which will be either one output layer which will bring a probability either less than 0.5 which means zero or greater than 0.5 which means one since it's a binary classification so we need a activation function between 0 and one for binary classification so self.

39:24 sigmoid and we are going to use NN do sigmoid function you see here NN NN I mentioned before that NN is we use it whenever we want to use any layers or activation functions okay okay great now we defined our layers we know our layers okay you can put much more linear layers you can try drop out you can dry whatever okay but the thing is that now we need to specify the forward propagation what does it mean in this function we need to specify how the data flows inside the model okay how the data goes inside the model okay so the data starts by the data is X represented x and x is equal to self.

39:49 input underscore layer of X which means that first it goes to the input layer and then X the new X we got is self. linear X and then X is equal to self do sigmoid of X and then we are going to return X okay so what did we do here we created the flow of the data so first the data will go to input x to the input layer the X will go to the input layer and then the output of that input layer will go to the linear layer and then the output will go to the sigmoid layer and then we're going to have the final output now this this is a straightforward flow it's a normal flow of the data inside the model forward propagation but this is interesting when you want to build a completely custom model you can play completely and modify completely the how the data flows inside this model okay okay great now let's create our model so let's write model equal my sorry my model okay and we should move it to device now don't

41:30 forget everything you're creating in pyto you need to move it to Cuda device okay or whatever device you have by the way you can put here MPS if you have a Mac and you set up using uh mm Mac silicon processors okay okay now we moved the model to device let's run this and see okay everything seems good now let's create the summary okay you remember we used TCH summary so here we can use this function to see the data uh to see how the model will look like in a very user friendly way so let's try uh summary of model comma X do shape of one comma okay now let's see that great it's working now do you see that okay we have 121 parameters okay let's get to know our function better this function is called uh summary so the function takes the model and takes an input the input shape the function in the back end does something like uh uh it imagine how the input is and then imagines a a back propagation sorry a forward propagation to see the output shape of each layer and to see the type of each layer it's going through the number of parameters gives you more details in tensor flow we

43:02 have something like this it's called Model do uh model do summary I believe I'm not 100% sure but I think it's model do summary but you don't need to specify the the shape of the input but here unfortunately you have to specify the shape of the input as we said before pytorch needs you to do a lot of work it's not difficult if you understand the the the problem not the problem if you understand the the how machine learning works it's not a big deal but you just need to get to know the syntax okay great now we have the layers so layer one layer two sigmoid and the output shape is 10 by minus one why minus one here minus one here represents the batch size when we use more data we will use by batches so we will use 10 columns eight rows because we'll use a batch of eight so here it's represented by minus one because it's just a one in instance okay it's a one row only and 10 then linear is getting one which means that it's getting uh one output and goes this to the sigmoid and gets the final output total parameters is 121 trainable parameters is21 non-trainable parameters

44:15 is zero we'll understand this later uh in the later videos not for now uh and this is the size of the model OKAY looks good now what's next next is that that let's create the loss and the optimizer so the loss function is usually uh criter Criterion okay it's referred to Criterion in most of the references so uh NN do BC e loss okay I think that's the correct way yeah okay now the next one is Optimizer is Adam okay now first the uh loss criteria is nnpc loss BCE loss is binary cross entropy loss okay and the optimizer we are going to use Adam now we need to tell the optimizers that you will be dealing with that model so we need to give it model do parameters which means that we are giving it the weights of the model that Optimizer needs to work on and also will be giving it the learning rate which let's say 1 E minus um uh four or minus three okay and that's it now we created the loss and the uh Optimizer and both works fine okay cool now let's start the the uh training function now we are going

45:53 now into the interesting part so first of all in the training we need to have four lists a list for the total loss of the training throughout uh the training epics the total loss through each training EPO the total loss through uh each EPO but for validation and the total accuracy for training and the total accuracy for validation why do we need this we need to record this after each EPO and then we can plot them and and see if the model is overfitted or not okay total underscore loss underscore train underscore plot this is a list for training and we're going to have total underscore loss underscore validation underscore plot and then we will have total underscore accuracy underscore train underscore plot and then we going to have total underscore accuracy underscore validation underscore plot these are empty lists that will be filled inside uh inside the training group now for EPO in range of epo now for each Epoch in range of epo now we didn't Define EPO so let's write 10 instead of defining or or let me let me to make it easier for you we can

47:29 write here aook equal 10 so that you know what is uh uh what is 10 exactly and then we can total underscore accuracy underscore train equal Z I'm going to tell you each what is the purpose of these things let me write them first Total underscore accuracy underscore validation and total underscore loss underscore validation perfect now what are these total accuracy training is a number that will be incremented throughout the epics it starts with zero we don't have any accuracy and after each EPO we will increase the accuracy and to know uh uh after each batch we will increase the accuracy to know the and in the end we can know the final accuracy of all the batches inside this ook which means all the data okay okay let's start for index comma data or no let's not chuse the index index is not necessary here for data in train underscore data loaders data loader okay um you remember data is having input and output okay so we're going to use inputs comma labels equal data okay we are going to extract data

49:08 data is having two values which is inputs and labels which are values of eight don't remember H don't forget inputs is eight rows labels is eight labels okay because the batch size we specified above is eight and this comes from the train data loader great now we're going to need to move inputs uh no we don't need to move inputs to device because we already removed them okay and then we can do the prediction equal model of inputs okay now let's see if everything works still here it's usually recommended that you stop at some point and try to run the things because if it's not working you can have a a you can easily modify before you continue so let's see okay uh this is this is wrong okay break great it's working fine and then we are able to do the prediction okay so we have our data we gave the data to the model the model outputs a prediction

50:20 which is definitely wrong because the model just initialized it's not yet trained then we are going to calculate the batch plus is equal to uh Criterion uh of prediction comma labels uh sorry I don't know what's happening okay and then we can do uh we let's try to rle here okay you see the error using Target size of torch size eight that is different to the input size of 8 by one so we need to make our input having this one okay the input size is eight only we need to have it uh with that shape 8 by one okay uh pyto cannot do this automatically so what we can do is that when we output the prediction we does this we does the following we does a squeeze function of one now let's try and it's okay it seems working but we need to stop it because we need to do a break to stop the loop now yes working

51:35 okay what is the purpose of this squeeze now let me print before and after squeeze and you can understand better instead of explaining let's print prediction and print or or let's print prediction shape better because it's it it will not fill the screen with very confusing things prediction shape and print prediction do squeeze of one do shape uh no no no run time we need to put a break statement break okay let's run um okay let me comment this because this is having a problem okay do you see the difference the difference here is that if we take one instance from the beginning or any anywhere if we take one in instance this is the first one okay we need to to break after this one okay because it will be better for you to understand okay great now do you see the difference the difference is the default value of prediction is 8 by one okay the value is eight predictions eight labels by

52:50 one but our after squeeze it became eight only okay why because when we compare prediction with labels like in in in the same cases guys in the two cases the values are same it didn't change but we just changed how the data looks like or or the shape of the data we just changed that but the data is the same it's not changing but we changed how the data is represented because the labels are just eight rows it's not uh sorry yeah it's just eight rows it's not having any problem but the predictions is 8 by one there is an extra Dimension so we need to squeeze it and remove this one to become eight only okay great now we can continue uh we can remove that one and uncomment this one back okay now let's uh remove this brake Also let's go next next is so we calculated the loss now we are having the loss now we need to Total underscore uh loss underscore train is equal to sorry it's plus equal

54:04 uh batch underscore loss dot item okay okay what did we do here do you see total loss train that we had here this loss is the loss after each EPO so after each EPO we make the loss and accuracy zero and then we calculate by adding the L of the previous uh of the previous batch uh uh of the loss of the previous batch and the accuracy of the previous batch we keep adding the accuracies of each batches of each batch until we finish all the accuracies and the losses and we have the final loss or the accuracy of of the final uh of the final thing okay okay so batch batore L do item means that we get only the number of it okay okay so you can think about it that we have a uh eight eight rows are being forwarded to the model we are having eight rows we do on these eight rows forward propagation and after we do the forward propagation we can we compare these losses using the loss function

55:15 sorry we compare the output with the original output of these eight rows and when we compare them we have the loss or the error we add this loss to to our loss because this is the loss of only eight rows not for for the whole data we added to the loss of the total loss train and then in the next batch the next eight rows we do the same and we add it to the previous uh uh loss and we keep adding until we finish all the batches we have a final loss value for this EPO and then we make these values zero again and then we do the same process for the next epok remember you are looping through two things I don't want you to confuse here you are looping through two things not through one thing and okay now we have the total loss train then accuracy is equal to how we can calculate the accuracy we need to calculate the accuracy between the predictions and the labels so we use the predictions but predictions here are uh are decimals or float so so we round

56:26 them to Z or one we use the round function and we make sure if it's equal equal label now this line will output to you uh sorry it's prediction only and it's labels okay uh this line will output to you eight an array of eight Elements which is having either true or false let me show you accuracy uh let's print accuracy and break and break so break uh uh allows us to stop the loop after first iteration so that we don't wait for so long okay just just we are still testing our stuff okay we didn't do anything okay you remember we remov The Squeeze so we put the squeeze thing here perfect do you see this a list of false through false true false false true true which means that the rows are the following they are eight row and the predictions are the first one is false true false and so on okay great then we want to know how many are true because this is the accuracy so we use

57:38 sum so let's see some will give us what some will give us three which means that we have three through uh out of out of eight okay and then we use do item which will give us at the end will give us four okay the number is changing don't worry because every time we start training from the beginning we start training with the data being changed the order so it's fine it's okay I mean if it's uh uh if you want we can do it in an easier way here so that I can show you better so uh let me print accuracy so that the number will be saved and not change do you see the accuracy here true false so how many we have 1 2 3 4 if we add dot sum will give us four dot item will give us the exact item four which means this number okay it will convert it from a tensor in this strange format it will give us the final number in just an integer okay so that's

58:51 it we will use this way okay so we're going to copy this one cut this one and paste it here now great we have the accuracy we have the loss now don't forget we need to add the accuracy to the total accuracy of the validation of the training because remember this is the accuracy of the batch accuracy of eight items only not for the whole so we add it to plus equal to this zero we keep increasing the accuracy okay okay perfect now are we done we are not done yet we just did the for propagation and we got the the accuracy and the loss now the next part is batch dot uh batore loss dot backward what does it mean now we need to do the backwards propagation okay don't worry there is no nothing difficult here we are going to the other side or we are going from the other side of the model to do the backward propagation and at this point the optimizer will come so we need to do optimizer do step which means Optimizer please take a step and change the weights because we already have the

1:00:05 loss Optimizer dot zore grad okay what does this line means means that after the optimizer changes the weight in this batch the optimizer restarts again this avoids having uh a iting this is better for not having um like the model will will converge by that way if you didn't do that the model will not progress okay okay perfect now we are done with the training the training is working perfectly fine okay let's try to do one batch of training and it's working fine great now the next part guys is the validation okay we have a validation that I set we want to do a validation uh okay now validation means that we make the model after uh getting trained we make the model uh does a prediction and after this prediction the model will not do a backwards propagation it's just a prediction you are testing the model during the training that's it okay you are doing a test of the model but the model is not yet trained okay okay now with torch Dot noore grad what does this mean this means that with tor.

1:01:14 no grad means that uh use the pytorch model but not for training just for testing so it basically It prepares the model for doing a prediction but without changing the weights we will just do the prediction it's like we're using the model for inference okay for uh inputs comma labels uh or no let's put it the same way as we did before so that you don't get confused in validation data loader uh we are going to have inputs comma uh labels from data and then we are going to do prediction equal model inputs dot squeeze the same way okay we are doing everything is same okay in fact you can copy uh the same code up into here the same that one into down but without these three lines okay but of course total accuracy will be validation loss will be validation okay then we calculate the loss the loss is used here just for

1:02:53 seeing the progress of the model and it will not be used for um for backward pration okay just want you to keep in mind that labels and then we are going to have total underscore loss uncore validation is plus equal batore loss. item then accuracy of um we are going to do accur is equal to two brackets like we did above so we can take this copy no issue uh okay and everything is same then we can write the total underscore accuracy underscore validation is plus equal accuracy and that's all okay now we are having the uh validation part okay okay great now let's try to run this and keeping the break function okay break let's see uh okay and perfect looks good okay so everything is working of course we'll remove these brakes at the end but for now uh we we just want to make sure that the code is working without any errors now we can remove them okay no problem and we can now we can go back and write

1:04:32 inside a EPO level not a batch level because remember here all of this is batch level and this one as well is batch level okay so means that we you have two Loops you have nested Loops don't forget so we are going to the main Loop now the main Loop is the loop for the EPO and we will start appending the values for plotting we're going to append total loss train plot do append you remember the lists we created here okay do append round total underscore loss underscore train divide by 1,000 comma 4 uh the comma should not be here yeah yeah comma should be inside okay I'll explain this to you don't worry now let me uh do the same for all of them now let's change the lists total loss validation will be here and we will use total loss Val and then we are going to use total accuracy train total accuracy train um we are going to use okay now how we are going to use the accuracy accuracy is will be a number okay so the number should be uh okay let me let me show you total

1:06:09 underscore accuracy underscore train multiplied by training underscore data dot underscore sorry underscore uncore get score uh item SC score like that okay multiplied by 100 comma 4 okay what did I do here okay it's nothing it's just the percentage formula so actually I'm using in the first one I'm using the round function okay to do the following I'm taking the loss the loss will be a very large number okay it will not be that large number but it's still you see here total L train 67650 okay it's better that we divide the loss by 1,000 so that it will be normalized we are taking the same concept of normalization here and round comma 4 we are using this one to do the uh rounding part okay so we take only four numbers after the decimal not not more than that and then validation we are doing the same but with the total L Val accuracy we are doing the percentage formula so we are having ground total accuracy uh train with the total accuracy we got from the training divide by trainingcore data do uh sorry it should not be get item it should be

1:07:42 get uh get Len I believe uh Len only without get Len Len what what is this it means that we get the length of the data that we have have okay so the total amount of the data we have so if let's say if we have 10 rows we got six correct uh so six divide by 10 okay and then multiplied by 100 which will give us at the end 60% if we take the example okay so we are calculating the percentage and we are doing round four as well to take only the first four values now we can do the same with the validation train here and you can see here that the total uh accuracy will be for validation and this one is going to be validation uncore data.

1:08:33 Len okay do you see now the the good thing about using the data loaders about using the the data set objects about using the things that P provide it's you feel that the things are totally customizable and at the the same time everything seems useful and working good together it's as I told you it's like an ecosystem okay okay great now we can print the stuff okay so let's print uh let's print the things uh print EPO number okay and this will be uh let's put it uh like that epok + one because it starts from zero and we want to print it in a very nice and neat way train uh loss is we are going to take that part for the train loss okay uh no not the total uh total loss train yes we are going to take that part here and then we are having the train accuracy which we will take this one also training accuracy do you see that one you see what I'm doing okay I'm creating the loop uh I'm creating the print statement for the training okay and maybe we can put a line okay so let's put three codes and three codes and we can put a line for easier uh writing and then we can

1:10:17 write validation loss is equal to uh we are going to take this one one and uh train uh not sorry not train validation accuracy is going to be this thing uh between curly brackets okay you can print it in any way you want okay this is not something uh uh this is not a problem you cannot you can even skip the printing part okay just this is just for giving some good uh look and you can see the progress so let me print also after each ook can print uh maybe 25 equal signs okay and let's try to print the training Loop okay yeah perfect now you can see here that the training is started okay the training is very good actually okay and it's not changing that much you know because it's the data set uh the model needs modifications the learning rate maybe needs to change so uh don't expect the model that it will be very good because we should have like a whole session for optimizing the model and making it better okay now maybe I can change the batch size maybe you can make the batch size 32 and we can try again so 32 32 32 let's run everything again and let's

1:11:55 see the model progress so it's starts from the beginning and it increased up till 98. 71 okay so okay it's good the model is good as I told you uh uh you can play with the model you can optimize the model the goal is for the pytorch syntax okay and that's all now we can proceed with the testing now we can see that the difficult part has passed start with the testing okay so in the testing we will use the testing data set to test our model okay so let's see now we will start by with torch.

1:12:21 noore grad as well this is the same part as what we did for validation inside the training Loop and then we will write total underscore loss underscore test is equal to Zer and total accuracy test is equal to zero okay and then we will uh do the for Loop for inputs or for data in uh testing dcore data loaders we will be having inputs comma labels is equal to data and and then we will have uh predictions prediction is equal to model of inputs same way we are doing everything is same inputs dot squeeze of one then we will write batch underscore loss underscore uh test is equal to CR Ian the same loss function we were using of uh prediction comma labels dot uh item we are going to take the item of that one and then we are going to have it because don't forget that this one is uh also uh batches okay we are doing this

1:14:24 as batches now we can have it as batch one doesn't matter but we are having it the same concept as we are doing for for for validation so we will add it to total loss test so total loss test is plus equal batch dcore loss uh underscore test great and then we are having accuracy the same so maybe we can get the accuracy also here and that one here and we can can change uh accuracy prediction same but this one will be here okay and that's it so I believe nothing is missing so we can print accuracy the testing accuracy is uh round also we can do the same round of okay we can get it from up here now we have to be uh uh efficient you can get it from here so don't waste more time okay uh and write it here but this one instead it will be accuracy test and this one will be testing

1:15:38 underscore um no not testing underscore data len by 100 so you can see here that uh training testing data loaders is not defined okay okay testing data loader I it's supposed to be I think and it yeah perfect now the accuracy is 98.79% which is amazing of course and uh this is you can think about it this is a small data set small model not a very complex problem so it's not difficult to get a very good number of of accuracy uh but still you can play more with the hyper parameters with the network with the things and you can uh you can get better results okay now comes the visualization part okay so first let's create figure comma ax axis is going to be PLT do subplots we'll explain this no problem number of rows equal one sorry equal one uh number of columns equal two and and figure size uh fig size is equal to 15 by 5 now what I'm doing here I am do I'm creating a plot okay two plots I'm creating a figure the figure is having one row so one row and two columns each column belongs to one plot

1:17:20 and this is the figure size okay it gives me the figure and the axis now I will write the first axis of axis Z I will plot here what I will plot the axis Z so it it it brings the screen like a grid okay like one uh row and two columns and the first row uh uh the First Column I'm dealing with it with this one XIs one uh sorry AIS Z do plot what I'm going to plot here I'm going to plot total uncore loss underscore train underscore plot this was the list we got from above you remember label equal training loss and then I am going to have AIS of zero I'm going to put some labels and put uh these things later so but for now let's put this this is the training loss and I'm I'm going to need the validation loss total loss Val then we can put the labels so dot set title now these things you don't need to memorize okay because you can get easily these things from from uh from Google not a problem I think it's

1:18:34 called validation uh you can get it copy paste from Google but you have to understand these Concepts okay and then you can search for for the things in Google it's fine training and validation loss uh maybe let's say over ook maybe okay and then we can write AIS of zero do set X label what do we want on X label the x label is going to be the EPO okay and we are having as well the Y label same almost same just change this to Y and we are having loss as the Y label and then uh I think it's it's pretty clear when you when you write it okay I I advise you that that you rewrite the exercise again even if it's on GitHub but you rewrite it again it's better so that you practice the the the you you practice the structure this structure okay and you can set a limit as well AIS of 0 do Set uh do set

1:19:47 uncore y limit okay why do we need y limit because we want to specify where y should stop so let's put y between 0 and 2 maybe okay and axis of 0 dot Legend okay okay now that's it for the accuracy uh sorry for the loss we're going to take the same and we are going to put here one XIs of one in all of this instead of rewriting everything again now here we will plot the accuracy so total accuracy and total accuracy we're going to rename it here to accuracy I'm going to take this into here and aook accuracy and the limit will be 1 to 100 because accuracy is in percentage and we're going to write here accuracy and that's it we can do PLT do show now let's see the results and uh expected end calls uh okay I believe it's without an underscore I think uh let's see yeah perfect now we have the the training loss and training validation so you can see here that the validation the training loss training

1:21:20 validation so you can see here that there is no overfitting and the model is going good quite good in in uh training loss but the validation the red line is not shown I'm not sure why but let's see let me see the accuracy um accuracy is almost same so I think it's showing but maybe it's not uh it's not very clear so probably we need to SP to to set the spacing between the diagram or something I'm not sure but let me try to put 50 I'm not sure I'm trying this together with you but um okay anyway it's it's not the problem uh it's not that problem now because it's not it's not showing here total accuracy validation plot um okay let me let me see am I total accuracy validation plot yeah I didn't append it do you see here the problem it's an empty list I didn't append it that's my fault sorry guys that's my fault so let me see here it's not total accuracy validation plot

1:22:40 should be that one and we can run the training again and then we will have some results here now the training already now it's running for the second time so there is no increase maybe you can run again from here and you can see difference but uh okay let me run again to see the different plot so we can run here uh L and Optimizer and these things and the training Loop yes it's starting from the beginning now and we are checking the accuracy again and run that one okay let's make it from 0 to 100 okay let's see yeah now it it makes sense now there is no overfitting and the other line appeared so do you realize the problem the problem is that I didn't fill this list I didn't append any value to this list as I appended accuracy training and accuracy validation inside the same list which came as as wrong obviously so that's that's my fault that's a typo but luckily it was not a big deal not a big problem and that's it now we are having already the training now what if we want

1:23:51 to do an inference so we want to use our model in production okay or we want to use it on a mobile app or we want to use it in whatever I want to do one single prediction I don't know the result I want the model to to know the prediction for me so let's say we have area of uh whatever um six maybe this number okay I'm writing just random numbers and we have measure access length Okay now comes the thing okay before we write these things you remember the original DF you remember that this we will be using to do the normalization because we did the normalization using the same values from here so we will use it to do the normalization again during the inference so how we going to do that let me show you now we will write area equal a specific number any number maybe I don't know that one maybe and we write original data frame of area divide by one divide by um sorry original data frame area dot Max or do

1:25:07 absolute we did the same Max and we are going to do the same here with measure axis length let's write any number maybe 81 and we put this one here and then we are going to do the same here measure minor XIs length uh 42 let's say anything doesn't matter okay and do you see what I'm doing now these numbers as a user you should insert these numbers and at the end you will find the prediction so you can put let's put uh more things let me uh take it from the data frame so it would be better just copy pasting okay minor access length and this thing can write here uh 32 and convex area we can write here uh let's say 12 um sorry need to copy this paste it here and so on you can continue with this process uh let me let me write it quickly uh yeah this is missing this thing okay uh let me put okay and then we will have more columns so let's write this okay let's take uh after confix area we have this

1:26:46 diameter put maybe 33 then we have extent we put let's say uh 98 and then we have parameter we put 927 roundness and we put this as maybe 677 now any numbers that doesn't necessary I I was just want to show you the final result on how to do a prediction okay without using the data set okay and maybe we can keep uh 24 okay and that's it now we going to use model Dot predict or we can only use uh model no need to use predict predict is used intens flow can use model only and we can take these values inside the list here area mejor axis length minor AIS length this thing convex diameter I know this makes takes a little bit of a while you can skip this section by the way no problem but this is just for you on how to do the final inference okay roundness because it's important to know how to integrate this Inside a Mobile app or a normal software okay now that's uh that's the list don't forget what did we do inside the data pre-processing

1:28:20 part we did the following okay now we can do the same actually we can take the data set object and create it or we can create without the data set object since you already know how to create the data set object we will do it without the data set object okay uh now we converted to tensor and then we moved it to device because we the model is on the device don't forget the model is on the GPU is not here it's on the GPU so to use these values you need to move them to the GPU so let's do the same we are going to have a list going to do torch dot tensor we are going to have dtype equal torch.

1:28:44 float 32 and then all of this tensor we are going to move it to device same way we did in the data set okay and that's it prediction equal or let's let's call it my prediction equal and then run and here we go working fine mycore prediction what is my prediction my prediction is zero which means from here it's not justment it's going in or this thing okay and we can you do do item to get the T that is only zero and that's it so sometimes this number can come can come as a decimal okay which maybe 0.70 something okay you can use the round function always so you can do like that round and here you go you have zero okay so this is important that we you use it if your output is something like the 0.73 is the same way we used the above it rounds it to one okay okay that's it for this lecture so you

1:30:27 are going to find the REO for this complete exercise explained in very deep details on uh in in the description you're going to find as well the other videos the video related to how to use the uh data sets inside collab directly and also uh the video on how to install Python and IPython or the Jupiter notebooks in case you need to work locally I always advise that you work on Google collab but in you if you want to work locally make sure you install Cuda properly okay that's it and thank you for watching subscribe and like for more video uh for more videos like this thank you if you have any ideas specific ideas in mind that you want me to implement please write in the comments thank you and welcome back to this series so