Transcription
I used Claude to run over 9,000 back tests of trading strategies to find ones that actually work. This included every popular one that you can think of like trend following, mean reversion, momentum, breakouts, and this was across 30 different assets and 15 years of data.
Now, if it's your first time on the channel, my name is Brendan. I studied math and econ at UCLA, spent three years in investment banking at Raymond James, and have been building out trading systems for clients for the past two years. And in this video, I'll go over how I built this entire testing system with Claude. I'll go over what the results were, how you can replicate the test yourself, and how you can use it to improve your own trading strategies as well.
Now, as always, this is not financial advice, and I'm not guaranteeing any profits. I'm simply showing you all the data and methodologies behind these tests and how you can leverage them as well.
So, diving right in, this is a system that I built with Claude. I basically took every single strategy and ran 9,000 back tests across 30 assets. So, for each strategy, I put it through a whole stack of testing to see what actually holds up.
Now, this is the complete overview of the testing system. It probably looks like gibberish, so I'll explain in more detail every single one of these tiles. What I tested was daily bars across liquid assets. So, this includes things like major ETFs like S&P 500 and NASDAQ. It also has all the sector ETFs, gold, oil, bonds, Bitcoin, and Ethereum, and large cap stocks like Apple and Nvidia. So again, these tests just covered large-scale trading, right? So not intraday or futures or options. So keep that in mind because it shapes what these results mean.
And now secondly, I wanted this to be purposely tough and pretty comprehensive. So that's why I ran this over the longest reasonable period, which was 15 years. The goal here wasn't to find the one strategy that works right now. It was to find what actually holds up in every single kind of market. So we took strategies in its most base stripped-down form. So if something survives basically this rigorous testing across what we have here this window 15 years it means we have a real edge in the market.
So at a quick glance what we have here in the top left-hand corner is the validation funnel. This is basically just going to show you how the 9,000 back tests narrow down. Then we have this scatter plot here which shows overfitting. And then to the right here we have survival rate by category. So spoiler alert, mean reversion is the only trading strategy family that really survives this whole rigorous testing. Now, I'll explain this in a lot more detail and how you can interpret the rest of the strategy families as well. And then to the left of this, we have the validation gauntlet. This actually shows you how we ran all the tests because they're pretty thorough. So, they include things like walk-forwards and robustness checks.
So, now let's first go through the validation funnel, which covers all the back tests of the strategies that we ran. So, here we can see that we started with over 9,000 back tests at the top and then by the time we get to the bottom of our filters, we're just left with 524. This means that thousands of strategies and back tests got wiped out simply from our six filters here.
Now, for the actual back testing here, we did walk-forward testing. Walk-forward basically means that you build and tune the strategy on an older chunk of data and then you test it on a newer chunk of data that it's never seen before. The reason that this matters is that it stops the strategy from overfitting the past. Anyone can make a back test look perfect on previous data that it's seen before, but the walk-forward makes sure to test on new unseen data.
Now the first and biggest filter that we ran is the Sharpe filter. Sharpe is just a score for how good the returns are versus how much risk you took. So ideally the higher the better. Here we just required a Sharpe over .5 on data that the strategy had never seen before. And this one cut removed almost 8,000 strategies leaving us with only 1,218.
Then after this we added a drawdown filter and we threw out basically anything that had a drawdown of higher than 35%. And if you're unaware, drawdown just literally means the biggest drop from a high point to a low point. So here, drawdown just means that you're down 35% from your peak. Now, obviously, if you are a riskier trader, you'd have a higher drawdown tolerance. And some people are completely fine with sitting through a high drawdown so long as the long-term payoff is worth it. But and just to reiterate, this is a pretty general test. I wanted to set reasonable drawdown limits to have this applied to almost everyone. And generally a pretty deep drawdown typically means that the strategy is pretty fragile or overly risky. I wanted this to be general, but later on in the video I'll show you how you can set this to have your own risk tolerance instead.
Then below this, I just added a filter for stuff that did way better in testing than out of testing. We're basically just testing for overfitting here. And then finally, we at least wanted a minimum number of trades to make sure that the strategy was actually viable. And all we're left with is 524. So basically in plain English, each of these filters are just a common-sense way a strategy can be, you know, fake, useless, or just weak.
Next we have an in-sample versus out-of-sample scatter plot. And this is one of the single biggest reasons strategies die, which is due to overfitting. Now stick with me here because this is a core concept. Now each dot here inside this chart is a back test. So the bottom axis is how well it did on the data it was built and tuned on, while the side axis shows how well it did on data that it had never seen before. So overfitting is when you tweak a strategy, the settings, the numbers until it looks amazing on past data, but all you've really done there is fit to random noise that already happened. So in those instances, it looks perfect in the past, but then completely falls apart as it gets introduced to new data or live data. So this is exactly what we're testing for here.
And on the right, I've actually put some numbers on this. So of all the strategies that look strong in testing, which again is looking at past data, only 44% stayed strong out-of-sample, which is tested on new data. This basically means that more than half of the good-looking strategies were basically fake. So coming back to this chart here, the green dots above the line are the few where the performance actually carried over to fresh data. So that's what you're looking for, consistency between the test and the real thing, not just a good back test.
And this whole reason is why we didn't use something like TradingView, just because the back testing is basic. It'll pretty much show you a strategy that looks amazing and gives you none of this validation here. It doesn't correct for luck. It doesn't stress test anything and it's all just overfit. So people typically just run one back test on there, see a great number, go trade it, and then have it completely fall apart in the live market.
Now on this next section, we have the main answer to our question of what strategy actually survives all this rigorous testing. So this section here groups every strategy into different types or families. And the two big families you can think of are trend and mean reversion. These are basically opposite bets. So trend bets that something moving in a direction keeps moving. So the idea here is that if a stock is climbing, you buy expecting it to continue climbing. Momentum is a version of that same idea, betting that what's been strong stays strong. But mean reversion here on the other side is the opposite. It bets that when a price stretches too far from its average, it goes back. So the idea is that it reverts to the mean. So you buy when something has dropped too far below its average expecting a bounce and vice versa.
Then there's also other strategy families like volume, volatility, pattern strategies that look for chart shapes. And then inside each of these families, there's a handful of actual specific strategies. So here on the trend category, we have things like moving average crossovers and breakout systems. Mean reversion has stuff like RSI-based snapbacks, Bollinger Band reversions, and so on across all the families. And then we test all those different strategies across a range of settings because obviously the same strategy at a different setting will behave differently from one another. So that's how we arrive to all 9,000 back tests.
So now the result here from our system shows that mean reversion is actually the only category of strategies that came out positive on average. Everything else. So trend, volume, composite, volatility, pattern strategies are all negative on average. What this means is that in its bare stripped-down form, so trading completely naked with no other rules over 15 years of every single market condition, mean reversion is the only one that holds up on its own. But again, this doesn't mean that the other strategies don't work per se. They're just a lot more situational. So it depends on the right market conditions. It might depend on the right asset that you're trading or combined with other strategies as a whole or applied at the right time. So, they can work. You just can't run them on autopilot the same way you would run mean reversion. And later on in the video, I'll show you how to actually apply some of these findings to determine when to actually use some of these other strategies. But as a whole, mean reversion really is the one that survives.
Now, the final test here we did was a bootstrap. What this means is that we took every surviving strategy's trades and then reshuffled them 500 times. The reason for this is to see all the different ways that the trades could have played out rather than the one way that it did happen. This makes sure that a successful trade wasn't just a lucky sequence and that if we were to reshuffle this 500 times, it still would have produced a winning trade. So here in these results, we can see once again that the mean reversion, the RSI revert, still has a solid verdict, meaning that if we were to reshuffle the results, it still would have performed well. But if we were to look at dual momentum strategies like ones on Nvidia, we can see that the max drawdown is actually massive. Down 61%, down 51%. What this means is that if we reshuffle these possible trades in an alternate universe, this thing would have basically cut your account in half. So some of these momentum strategies, like this one in particular, only looked clean in the very exact path that the market actually took. Meanwhile, the mean reversion didn't rely as much on luck, and no matter how much we shuffled the trades, it still had a solid outcome.
So now let's go into what survived and why. So mean reversion was the only category that really came out positive out of the major strategy families. And let me show you where it gets really interesting. So we originally had 524 strategies that passed all six filters. But this is further filtered down to 478. It's only on assets with over 10 years of history. So things like the XLC ETF doesn't really make this cut because it wasn't launched until mid-2018. So only has around 6.5 years of data from that 2010 to 2025 window. And then out of these 478 strategies, 64% of them are mean reversion strategies. So that's this bright green chunk. So it didn't just survive, it basically made up almost 2/3 of everything that did survive.
And then like I mentioned, mean reversion again is just betting that a price that stretched too far either above or below will always snap back to its average. Now this pretty much lines up with research, right? Mean reversion is one of the most backed ideas in all of trading. There's decades of academic work behind why prices overreact and then revert, tied to human psychology as well as just how the way the market works. So if you were to compare that to like say a moving average crossover, which is just one line crossing another line, there's no real reason that should predict anything and the data shows it mostly doesn't.
Now moving on here, we can see the actual top survivors by score. Number one survivor is actually a trend strategy, Turtle on Apple, at 1.18. And you'll see a few other trend strategies up high, too. So, trend strategies can absolutely work. It just worked in specific spots on specific strong trending names like Apple and Nvidia, not broadly across everything. And a lot of these trend scores were the fragile ones that we just looked at with the huge drawdowns as well. So, the lesson here isn't that trend strategies are dead, but more so that it's situational. So on these specific assets or specific market conditions, they absolutely do outperform. It works great on specific assets, but you can't count on it everywhere like you can with mean reversion. You can think of mean reversion strategies as a broad, reliable base, and then trend as more of a use-it-when-it-fits kind of thing.
And now this is the slide that really shows why mean reversion is so powerful. Shows the different types of mean strategies that it actually worked on. So we can see that RSI mean reversion didn't just survive on one ticker. It survived on 20 different tickers. We have the Keltner reversion on 18 different tickers. Basically, if something only works in one place, it might be luck or just very situational. But what we're showing here is that across 20 completely unrelated assets, we have this RSI mean reversion essentially working everywhere. And then here we can even see this laid out by category. So as a group, mean reversion really is the only category with a real positive edge. Creates a trend, volume, composite, volatility, and pattern strategies.
Now again, the survivorship is pretty boring. You have modest returns, controlled risk, but this is what's expected out of our test. These numbers are pretty modest just because this is honest testing across frankly, like 15 years and every single kind of market. If I only tested a recent stretch or specific assets, the returns would have been significantly higher. But we're just testing for the general durability of strategies across basically every single type of asset. And then one thing I really want to stress is that we're looking at the floor, right? Not the ceiling. So, this is the most conservative possible version, base strategies, completely naked, no improvements, ran over 15 years, including every single crash. But the moment you start dialing in on these specific strategies, like focusing on a shorter time frame that fits current conditions, combining a few of them together, adding layers that I'm about to show you in the next section, these numbers can get a lot better. So, you can think of this as just a raw foundation before you build anything on top. But just for like a plain takeaway, across every popular strategy traded on its own in its bare stripped-down form over 15 years, mean reversion is the only one that broadly holds up on its own. Basically survived across almost all assets, made up about 2/3 of everything that passed.
So now that we understand the most conservative, modest, and boring strategies, we need to know what to do with this. Right? This isn't a finished strategy. It's a foundation. So let me go ahead and show you how you would build on top of this. So the first thing is that there's a lot of nuance beyond just looking at stripped-down strategies. So obviously trend and momentum strategies did show strength in certain specific situations. So I expanded the test a bit further. So if you remember, momentum scored basically zero as a basic strategy on a single asset. But there's a smarter version called cross-sectional momentum where instead of asking, "Is this one stock trending?" you rank a whole basket of assets against each other and then only go long on the strongest ones and then short the weakest. I tested that version separately and as you can see here in these results, it scored way better, going from basically zero. So what this means is that obviously momentum isn't broken. The basic version that we tested. So all 9,000 basic versions obviously wouldn't work at its core form.
So this then leads to my next point, which is the framework on how you actually build a real trading strategy with Claude, because it's layers, right? So at the bottom you have your base signal. So something with a real edge, which we proved is something like mean reversion. Now on top of mean reversion, you would add risk management and position sizing. And this is the boring part that, you know, nobody wants to hear, but it is pretty much what makes up the crux of your strategy. Everyone has a different tolerance to risk and how much they want to size their positions, but this is huge for, you know, minimizing drawdowns on specific trades that you have. And then on top of this, you would combine it with more uncorrelated signals. So this allows you to diversify across signals that don't particularly move together because again, we only tested all the strategies in their bare form. So basically, one weak signal alone is noise, but several that don't move together can actually be something real when you combine them. And then at the very top of this foundation, we have regimes, which is something I cover pretty extensively on my channel. So this is where we categorize the market and then only use a strategy when the market actually suits it. So here you would use Claude to basically build a hidden Markov model, which is basically a model that detects what state of the market you're in right now. So, it'll tell you if it's a bear market, trending, or a choppy market, or a full-on bull run. So, you would then run the momentum strategies on the trending or bull run markets, and then switch perhaps to mean reversion when it's choppy. So, the idea here is that you would take a momentum strategy that fails when you just run it blindly on autopilot, and then combine it with something like targeting the market, and then you would see that it's actually situationally useful in these different contexts. So, here you can see in a trending market, you would lean momentum. In a choppy or ranging market, you would lean mean reversion. So although mean reversion worked almost all the time on autopilot, momentum strategies with this new kind of tag, you'll be able to situationally apply it better.
Now, one last point I want to end this video on before I show you how to build a system yourself is that this is very personal for building your own strategy. There's no single right answer that works for everyone because everyone has different capital. So the portfolio sizes are different, they have different risk tolerances, time horizons, and how much of a drawdown they can actually stomach. So the idea is that you would take a successful strategy or a real edge like mean reversion. You would then add risk management and sizing based on your own personal preferences and portfolio. You would then combine it with uncorrelated signals and then apply it based on market regimes. There's no copy-paste answer because again, everyone has completely different capital, risk tolerance, time horizons, and drawdown tolerance.
Now, if you do want to learn how to best use Claude to build out your own trading strategies, trading systems, and just more robust testing, I have a school community where I run the largest AI-focused trading community all about using the latest AI tools like Claude specifically for this. In there, we have full in-depth guides and active community members all building different systems across crypto, stocks, options, futures, basically every single asset class. And the important thing to stress here is that with AI, you can truly build out some very complex systems, test your own ideas in record time, and be able to just generally improve your trading over time. So, make sure to click the link in the description if you are interested.
Now, let me go ahead and show you how you can build the same exact testing system yourself with Claude. So, I built this whole thing with Claude in a few layers, and each layer is just a separate prompt. The first prompt just builds the data and strategy library. So, it pulls all the assets and sets up all the strategies. The second layer builds the backtest engine and the walk-forward testing. The third adds the filters and validations. So we have things like the multiple test corrections and realistic transaction costs. And then the fourth just adds robustness checks and the regime changes.
So here we can see that it includes every single popular strategy in its base form across 30 assets and 15 years. And again, you can tweak any of these to be specific assets that you want to trade in particular or adjusting for a shorter time frame. But it does have a lot of validations that most tools skip. So the real advantage of using this over something like TradingView or any off-the-shelf backtester is that when you build it yourself with Claude, you basically control every single part of the validation. You can add things that just simply aren't in TradingView, like walk-forwards to make sure you're not overfitting. You can correct for luck across thousands of tests. You can put in realistic transaction costs. And you can even stress test the survivors. This is how you can tell if a strategy is actually real, not just what a simple backtesting platform would tell you.
So, moving on, I'll go through all of the layers. And like I mentioned in my previous videos, if you want to build these, it's super simple. You can literally just screenshot the prompts and then feed it to an AI model. And then you can build it either in the web app, so like Claude.ai, or you can use Claude Code. It's a little bit more advanced, but you'll have one singular place to just edit all your code, make changes to the codebase, and continually update it as well.
So, here's the data and strategy library. So simply copy and paste this directly to Claude Code. Then we have the backtest in the funnel. So this is where we do the walk-forwards, out-of-sample, and the six-filter funnel. This is the first part of the prompt. Second part. And then again, just make sure to pause whenever you need to. Then we have layer three, which is robustness checks. This tests for parameter sensitivity and a bootstrap stress test. It's the first part. And then finally, we have the cross-sectional momentum check if you are focused specifically on momentum-based strategies.
And now with these, you can test your own ideas properly instead of trusting a backtest you once saw on TradingView. Now, if you enjoy this video, make sure to leave a like, comment, and subscribe, as it greatly helps out the channel. And if you do want to go deeper, my community is in the description, as it's the largest AI-focused trading community. Everyone in there is building trading systems across stocks, crypto, options, and futures with step-by-step guides, even if you have zero coding background.