Transcription
In today's video, I'm going to show you why most back tests are lying to you and how you can actually use AI to properly back test your trading strategies, helping you become a more profitable trader.
We're not just going to use simple Trading View back tests since, as you'll see later in the video, they're basically useless. Instead, we'll be training our own AI model with complex math logic completely with Claude code. This way, we'll have a proper back testing engine that can actually adapt to unseen market data to help you become a better trader and ultimately use the same systems as quant and hedge funds.
So, make sure to stay until the end of the video so you can see how you can build this out and test it out for yourself. And make sure to subscribe if you're interested in more AI trading content.
Now, like always, I'm not a financial adviser and this is not financial advice. I'm not guaranteeing any profits in this video. I'm just simply showing you how you can use the power of AI to improve your back testing and ultimately your trading strategies as a whole.
So, why do normal back tests fail so badly? Well, it's from a trap called curve fitting. When you test a strategy on years of historical data all at once, your bot is basically memorizing the past. It's like giving a student the answer key the night before an exam. They're probably going to get 100%, but they don't actually know how to do the math.
When you see people flexing these massive Trading View back test returns online, it's generally because they worked backwards to find the perfect cherry pick settings on a very specific time window. So to fix this, we're going to be building a walkport analysis engine with Claude code. This is going to be a huge step up from the fake 300% back test returns you would typically see on Trading View.
So with this new engine, rather than letting the bot cheat, the walkboard analysis forces the model to train on a very small window of data. Then it will lock its settings, then train on the next few months of completely unseen data. The back testing algo will basically tell us if our strategy will survive the real world since it's not working with all of the historical data all at once.
Now let me go ahead and show you how this back testing engine works. So here on the lefthand side, we can set all the settings. So first is the ticker and then we can also choose the start and end dates for when we want the back testing window to start and end. Then below this we can set the actual walk forward windows which is the training window as well as the blind test window.
So the training window here which is for 12 months or a year is where the bot is allowed to basically look up the answers. So during this one-year period, it optimizes thousands of different parameters to find the exact combination that produces the highest profit or sharp ratio. So this is basically just what a normal trading view back test does. But because we're building our own engine, we want to go a step deeper and that's where we use this blind test window.
So once we find the perfect settings from the training window, the settings are permanently locked and then the bot is pushed into this blind test window. So you can think of it like for 12 months we're finding the perfect settings based on those 12 months of data. Then once we do we're going to run that same exact setting on 3 months or the next 3 months of blind data.
Now because this 3-month chunk is data that's never been seen before. If the strategy was just faking it right from the past 12 months then it'll lose money here. If the strategy actually does have an edge then it would also apply in the next 3 months. And then once this three months of testing is over, the entire engine walks forward and then repeats everything again. Hence the walks forward analysis.
Now if that's a little confusing, this right hand side basically explains everything once again. So here with the problem of traditional back testing, it's basically that it uses all the historical data at once. So if we were to run this back test from 2018 to 2024, it would look at all this data for the past seven years and then find the best settings, which again is something we don't want to do because the point is just because you get the highest return in the past period does not mean that it's going to be the best strategy moving forward.
So what we're doing here instead is the same process of finding the best parameters. So like traditional back testing but only on a one-year window and then using those same best parameters on 3 months of unseen data. So down here you can visually see what's happening. Basically from 2018 to 2024 we're going to group all those chunks of data into folds. So the first year is going to be from January to December 2018. This is going to be the entire training window. And then for the next three months, so January, February, March of the next year, 2019, is where we're going to apply the best settings from the initial training window. And then all we're going to do is repeat this once again. So for the next 3 months, we're going to use the past 12 months of training data, then look at the next 3 months completely blind, apply the same parameters to see the true performance, and then rinse and repeat until we reach all the way to the end.
Now, the strategy here we're using is pretty simple for the back test. I'm sure you've seen it a million times before. All this basically does is say that when RSI crosses above the entry and the price is above the 20 period SMA, then long. In traditional back testing, we would look at the entire data set from 2018 to 2024. And then based on this entire historical period, we're going to find the best RSI numbers and entry lines. And then you obviously think in your head like, wow, this was, you know, a great strategy. It was able to dodge the 2020 crash and then also capitalize on the 2021 bull run. And you would think that it predicted the market, but again, we just gave it an answer sheet and it found the best possible combination that doesn't actually apply to present day.
But instead, with a walk forward analysis, we're going to find the best RSI numbers for this one year, then apply it blindly to the next 3 months, see the true performance, and then finally, what I've added here is just simple transaction costs. We'll have like slippage and exchange fees. So now if we go ahead and click this run walk forward analysis, we can see that it's actually downloading the data and then testing each fold individually. So we have 19 total folds from 2018 to 2024.
So just to summarize on the timeline, we have 19 folds. So this was the start from January 1st, 2018. And as you can see, we first do the insample training and then the blind test. In sample training, blind test. So 3 months all the way down to present day or 2024 in this case. Below this here we can also see the best RSI parameters being chosen in each fold. So fold one had an RSI period of 21, 7, 10, 21, etc. But below this is where the real magic happens and what truly differentiates this back testing engine from just very standard trading view one.
So if you look at these results, it's pretty insane, right? The difference on the retail back test. It's showing basically almost a 3x return and it's basically telling you that this RSI strategy is a gold mine. You would have made a ton of money. Uh, you have a sharp ratio over one and barely a five, 6% draw down. But then if we look at the walk forward analysis, this strategy just completely falls apart. So when it trains on data that it's never seen before, it's only returning 5%, which is obviously very, very different from 199%. Furthermore, if we, if we scroll down, we can see the return degradation, it's basically saying that 75% of this total return, it was just completely fake and created by curve fitting.
So to sum it up, in the traditional back test here, it was basically looking at the entire 5 years of data, crunching millions of combinations to find the optimal RSI parameters and applied it, you know, historically to get this massive return. But if we were to run it live, it would completely fall apart. In this walk forward here, what we did is apply a little fold. So each fold we tested on ideal parameters and then blind tested and repeated this 19 times to get the true performance and how it would have, you know, behaved assuming that it didn't have all the data beforehand. And then finally, I know I've hammered this through like a billion times, but just to show you visually one last time, this blue chart is what the retail back test looked like, and this orange chart was the real performance. So again, massive, massive discrepancy.
So basically, all this tells you is that this RSI strategy is fundamentally weak. Even if you were to optimize the past 12 months with the perfect RSI parameters, you still wouldn't be making any real money because RSI is a lagging indicator. And this is obviously huge because we just saved ourselves from an extremely bad strategy.
Now, let me go ahead and show you how you can build this yourself so you can begin testing out your own strategies and potentially even find new ones to trade on the market. Now that you see why traditional back testing is basically useless, let me go ahead and show you how you can build this walk forward engine from scratch. And before we continue, if you want a more detailed guide and templates on how to back test with AI, as well as improve trading performance with AI tools like Cloud Code, make sure to click the link in the description for my community, as we have complete guides on how you can actually use AI to automate your trades and improve your performance.
So, starting off, you don't need to know how to write Python code at all. We're basically going to have Cloud Code do all the heavy lifting for us. So, all you want to do is open up Visual Studio Code or VS Code. Then, once you're in Visual Studio Code, go ahead and click open. And all you want to do is open up a blank folder. And this is where all the code for your back testing engine is going to live at. So, if you don't have a dedicated folder, just click create new folder.
Now that we're inside our blank folder where we're going to build out the back testing engine, the first thing we need to do is add Cloud Code to VS Code. So, all you need to do is go over to plugins right here and then search for Claude Code. If it's not already installed, you can just go ahead and click install here and it should pop up sooner or later on this top bar right here. So once you have Cloud Code installed, just go ahead and click open.
So after opening up Cloud Code, if this is your first time, it's just going to ask you some setup questions, settings, and then for you to log into your account. And if you don't want to log in, you can also just use Cloud Code via API as well. But now assuming all goes well, what we have is a completely blank claude code screen where we can now begin to prompt uh what we want to build and in our case a very robust back testing engine.
So next what I have here is the prompt that we're actually going to use to build out our back testing engine. So what we want is to build the entire streamlit dashboard. We also want the sliding window walk forward engine. So basically all the things I showed you in the beginning of the video as well as just a very basic RSA strategy that I'll show you how to tweak later on.
So going through this one by one I'm just saying build a professional walk forward trading app in Python using streamlit plot lead and Yahoo Finance. For the asset you can change this to any ticker you want but we obviously have spy from January 2018 to today. Then again like I mentioned the strategy will be that same simple RSI crossover strategy which we will upgrade later on. Uh for the back tester once again we want to prevent curve fitting. That's why we have this to be trained on a 12-month rolling window and then tested on the 3-month blind window. And then to evaluate the performance we only want to stitch the out of sample returns together to get a true measure of how the strategy did. For risk management we want to make sure that we're actually factoring in the transaction cost. So we have 0.1% exchange fee uh and slippage. Then finally for the UI we want to show the progress for each of the folds. This way we know all of the data that's being chunked together and what is actually happening on the back end. And then just like the beginning of the video I've added some uh charts and this is to more so visualize how this back tester works versus a more traditional method.
Once that all looks good to go just go ahead and click run and let cloud code basically do its magic. It should first install any dependencies that you'll need for the application. So in some cases you'll need to install Python. And after waiting a few minutes, we can see it's gone ahead and built the app. So we can just open up this URL to see the progress. Uh, but in terms of the strategy, it's the same simple RSI crossover strategy with costs. We have the walk forward engine that rolls a 12-month training period with a 3-month testing window. And then again, only the out of sample returns are stitched together. Then finally, we have the dashboard that basically shows all of the progress and what we're testing in terms of the strategy and the returns.
Now, again, this is just the previous app, right? We have the same failed returns, 5.8% versus almost a 3x if we did a traditional back testing route. But let's go ahead and update the prompt so we get something a bit more tangible and useful, uh, rather than just relying on this lagging indicator for RSI crossings.
Now, if we were on Trading View, we would basically keep tweaking the RSI numbers until we saw a green. But since we want to behave just like institutions and quants, if the underlying math is weak, then what we should do is just take it out and upgrade it. So, that's exactly what we're doing here inside this new prompt. All I wrote was that the walk for engine proved the retail RSI strategy fails on unseen data. So, what we want to do is just completely rip out the RSI and SMAs, and we're going to replace it with a volatility scaled momentum indicator. And this is going to be based on average true range instead. And then we'll smooth out the price action using a two-pole Butterworth filter.
So here I'm just saying build a volatility scaled momentum indicator using ATRs. We want to filter the price noise with this butterworth filter. Now one small nuance if you do want to use a similar strategy is that typically when you work with AI coding agents, they love using this filt filt rather than L filter which is what we want. Now, this filt filt function here does make the chart look perfectly smooth, but one issue is that it actually uses future price data. So, it's basically just using future prices to calculate today's indicator, which is a massive look ahead bias. So, by using L filter here instead, we're basically guaranteeing strictly casual forward only math. And this is just to help stop the AI from hallucinating fake returns.
So, once this looks good, let's go ahead and run it. and then see our updated back testing engine on a completely new strategy. So we can see in the to-dos it's basically replacing the RSI SMA logic with volatility scaled momentum. Then it's going to update our dashboard with labels, sidebars, tables for the new strategy and then finally actually run the walk forward back test on our completely new approach. So here you can literally see all of the changes in the code that the AI coding agent is making for us to reflect this new strategy that we want to implement.
Now we can see what was built. I'm not going to go ahead and open up the dashboard again because it's going to show the same exact type of charts. Instead, I just have a text summary here. So again, what we did was rip out all the RSI SMA logic and replaced it with more institutional grade logic instead. So here we have a two-pole Butterworth low pass filter average to range and volatility scaled momentum. And now looking at the results, this is honestly probably the most important lesson. Once again, just really showcasing what I talked about in the beginning of the video.
So, if we looked at the in sample illusion back test, this had over a 1,500% return, which again would be massive. So, if someone basically showed the strategy, it would seem like a guaranteed money maker. And that's because we are using a bit more of a legitimate strategy that is actually adopted by a lot of institutions. But once we go back to our reality which is the walk forward analysis, we can see once again that the true return is not 1,500% rather 7% if it was testing on blind data. And as you can see the sharp ratio did basically collapse. And this basically proves that you can curve fit all you want advanced institutional math as easily as you can curve fit like a basic lagging RSI indicator. Adding complex math does not magically create a profitable edge. And if you do truly want to find an edge in the market, it's going to take a lot of back testing and true back testing, especially with something like a walk forward analysis versus just believing these numbers that people, you know, typically show with their returns.
So the point here is that the markets evolve constantly. So by using tools like claude code, you can actually create a meaningful back test that isn't used to show, you know, a beautiful chart that just goes straight up. The goal is to ultimately find out if the strategy that you have will actually make you money in the markets.
Now, if you found this video helpful, make sure to like, comment, and subscribe since I'll be covering everything AI trading and investing related. Uh, so I'll be going over all the latest tools like Cloud Code and how you can actually apply them to either your trading performance or your investments.