Algo trading and ML fundamentals

0:00 in this comprehensive course on algorithmic trading you will learn about three cuttingedge trading strategies to enhance your financial toolkit latchezar teaches this course he is an experienced quantitative researcher and data scientist in the first module you'll explore the unsupervised learning trading strategy utilizing SP 500 stocks data to master features indicators and portfolio optimization next you'll leverage the power of social media with the Twitter sentiment investing strategy ranking NASDAQ stocks based on engagement and evaluating performance against the QQQ return lastly the intraday strategy will introduce you to the gar model combining it with technical indicators to capture both daily and intraday signals for potential lucrative positions hello and welcome to this free code Camp course on algorithmic trading machine learning and Quant strategy with python my name is Lazar and I'll be your instructor toout the course and in this course we are going to develop three big quantitative projects from start to end and the

1:11 course overview would be the following first we're going to talk about algorithmic trading Basics then we're going to talk about machine learning in trading some obstacles and challenges we may face while using machine learning in trading then we are going to develop the first project which would be an unsup rised learning trading strategy using stocks from S&P 500 the next project would be Twitter sentiment and it would be using data from NASDAQ 100 stocks and the third one would be focusing on one asset it would be an intraday strategy using a gar model to predict the volatility it would use simulated data and after that we are going to have a quick wrap up and with that we'll finish the tutorial but before we continue I would like to mention that this tutorial should not be ConEd as a financial advice it is for educational and entertainment purposes only we are going to develop some Concepts and come up with strategies in the end but it's not a financial advice and you shouldn't make any decisions based on it as well for this course I would assume that you have at least some python knowledge and

2:22 understanding because down the road will deal with some complex problems and if you're new to python you may get B down nevertheless you learn some very interesting Concepts so stay along and let's get into it okay algorithmic trading Basics so what is algorithmic trading it is trading on predefined set of rules which are combined into a strategy or a system it is developed by a programming language and it is run by the computer it can be used for both manual and automated trading by manual what I mean is that you may have a screener which comes up with a set of stock stocks you want to trade on a given day or you may have a algorithmic strategy which is developed into an alert system and whenever the conditions are triggered you get an alert but you execute it manually on the other hand you may have a completely automated complex system which does a lot of calculation comes up with uh positions and sizing and then executes the trade automatically okay so what is the role of python in algorithmic trading python is the most popular language used in Alor trading quantitative finance and data science and this is mainly due to

3:35 the vast amount of libraries that are developed in python as well the ease of use of uh python it is mainly used for data pipelines research back testing strategies as well it can be used to automate strategies but python is a slow language and it can be used to automate low complexity systems if you have a really high-end system which is really complicated and it needs to execute trades really quickly you would use Java or C++ for those strategies our gmic trading is a great career opportunity it's a huge industry there are a lot of jobs with hedge funds Banks Prop Shops and I've just checked the average yearly base salary for Quant researcher is around $173,000 and this is not including the yearly bonus it is a great career opportunity and if you're interested into it the main things you need to know are python you need to know how to back test strategies you need to know how to replicate papers and you need to know machine learning in trading if you're interested into it I'll definitely

4:44 advise you to go for it okay so let's move on and talk a little bit about machine learning in trading and some use cases of machine learning when we talk about supervised learning we can use it for Signal generation through prediction for example we can come up with buy or sell signals on a given stock or a given asset based on predicting the return or the sign of the return of that asset we can as well use it in risk management for example we may use a prediction to determine the position sizing or the weight of a given stock in our portfolio or to predict where exactly should our stop loss be and with unsup learning we can use it to extract insights from the data for example we can discover patterns relationships or structures within the data for example clusters and use it in this way to uh help our decisions what are some of the challenges that we may face while trying to apply machine learning in trading and the first theoretical challenge we may face is the so called reflexivity feedback loop and it is referring to the

5:57 phenomenon that if we for example have a machine learning model that is uh predicting that a stock is going to go up each Friday and we can form a strategy around to profit from that phenomenon for example we are going to buy each Thursday and then sell on Friday to capture that price increase move if we find this strategy throughout predictions and start trading it with time other Market participants as well are going to find this Market phenomenon and start to exploiting it as well which would cause the price to start going up on Thursday because everybody's buying now on Thursday instead on Friday and then this strategy is going to be Arbitrage away so it's this reflexivity feedback loop which is making predictions quite hard what is most hard while applying machine learning the most hard thing is to predict returns and predict prices the next quite hard thing to do is to predict return signs or the direction of a given asset is it going to to go up or down the next thing is to predict an economic indicator for example it is quite hard to predict nonfarm payrolls or weekly jobless

7:09 claims and a thing which is not that hard or quite straightforward is to predict the volatility of a given asset furthermore there are some technical challenges like overfitting a model or generalization overfitting is that the model is learned the train data too well and it fails on the test data and generalization is that the model is not performing the same as on the real data as well we may have nonstationarity in our training data and regime shifts which may ruin the the performance of the model and the last thing is that if we have a really complicated model or um neuron Network it is like a black box and we are not able to interpret it correctly what is the usual workflow process in algorithmic trading and machine learning that would be to collect and prepare the the data then develop a hypothesis for a strategy then you have to code the model and train the model and then finally back test the strategy okay guys and some key takeaways from this course so you learn high level Concepts in quantitative Finance as well practical machine

8:18 learning in trading you develop a project from idea to back test final results however we will not automate or execute any trades we'll just develop a strategy this course is all about developing a strategy from start to end so you see the workflow and this is just a purely research project for educational purposes I repeat it should not be construed as any Financial advice whatsoever and with that we can move to the first project and let's get into it okay so first project is about unsupervised machine learning trading strategy we're going to use data from S&P 5 500 stocks let's talk a little bit about unsupervised learning in trading so it involves machine learning techniques to analyze financial data and discover patterns relationships and structures within this data without predefined labels or Target variable unlike supervised learning where the model is trained to make predictions unsupervised learning focusing on extracting insides from the data and some use cases would be clustering which we are going to use in the first project

9:29 dimensionality reduction anomaly detection Market regime detection and portfolio optimization in this project what we are going to do is first we're going to download all the prices data for all S&P 500 stocks then we're going to calculate different technical indicators and features for each stock next we're going to aggregate on a monthly level and filter only the top50 most liquid stocks from the S&P 500 for each month next we are going to calculate PL monthly returns for different time Horizons to add up to the features and the next step would be to download the F French factors and calculate rolling Factor betas for each stock as well to add to the feature set and at this point we'll have enough features to fit the model and either make predictions or in our case we're going to fit a c's clustering algorithm an unsupervised learning model and we will use it to group group the stocks into similar assets into clusters and from those clusters then we will be able to for each month select stocks from a

10:38 given cluster and we are going to analyze the Clusters and select a particular cluster and then for each month we are going to select those stocks within this cluster and form portfolios however these portfolios will be optimized so the weights of the stocks within the portfolio we're going to find them by using the efficient Frontier Max Sharpie ratio isue portfolio weights and then we are going to form the portfolio hold for one month and rebalance at the end of the month and you know form another Max sharp ratio portfolio in the end we'll have the strategy returns for each day and we will be able to compare our portfolio strategy returns to the S&P 500 returns themselves actually not small limitation is that we're going to use the most recent S&P 500 stocks list which means that there may be a survivor buyers in this list this is a huge issue actually in reality you should always uh back test strategies using survivorship free bias data what is survivorship bias it is the condition when a stock which have actually went out of the S&P 500 because it was failing is currently not in the

11:49 list right so last year for example there was a stock which was failing and going down down down so at some point in December last year they removed it from S&P 500 and they included a new stock so if we have made the optimization last November we could end up with having this stock into our portfolio and actually it affecting our portfolio results but if we use the most recent S&P 500 symbols list this stock would not be there so that's survivorship bias and for the given project we are not going to deal with this survivorship bu so the list we are going to work with has survivorship bus most probably so yeah that is limitation you need to know in the second project we are going to develop a Twitter sentiment based investing strategy we are going to use the NASDAQ 100 stocks and Twitter sentiment data what is sentiment investing this approach focuses on analizing how people feel about certain stocks Industries or the overall Market it assumes that the public sentiment can impact stock prices and for example if many people are positive about a particular company on Twitter it might indicate potential for that company

13:01 stock to perform well what we are going to do first we're going to load the NASDAQ stocks Twitter sentiment data then we're going to calculate a quantitative feature of the engagement ratio in Twitter for each stock after that we're going to rank all the stocks cross-sectionally for each month and create an equal weight portfolio in the end we're going to compare the return of this portfolio to the NASDAQ itself the second strategy is much smaller than the first strategy there is no machine learning modeling in this strategy but the idea here is to show you how alternative or different data in this case sentiment data can help us to create a Quant feature and then create potential strategy out of it that's the idea of the second project and the third project is about an intraday strategy using a gar model in this one we are going to focus on a single asset and we'll be using simulated data actually we'll have daily data and intraday 5 minute data but what does an intraday strategy means this approach involves buying and selling Financial assets within the same trading day to profit from the shortterm price movements intraday Traders usually use technical

14:13 analysis real time data and different risk management techniques to make decisions and profit from the strategies what exactly we're going to do in this project first we are going to load the simulated daily data and the simulated 5 minute data then we are going to define a function which would fit in a rolling window a gar model to predict one day ahead volatility of the asset after we have that we'll calculate prediction premium so we'll predict the volatility and we'll calculate the prediction premium and form a daily signal from it after we have uh calculate that then we'll merge the daily data with the intraday data and calculate intraday technical indicators to form intraday signal so we will have a daily signal and then on top an intraday signal so two signals and after we have that we'll generate position entry and hold until the end of the day in the end we're going to calculate the final strategy returns and this is it for the third project the idea is to show you how predicting volatility Works in uh intraday strategies and yeah with that guys we are ready to jump into the first project so let's get into it okay let's