Transcription
I found six trading data providers that will save you $34,000 per year. Now, why do I say $34,000? Well, direct market data can cost $34,000 a year if you're going professional. And in this video, I'm going to show you versions of that you can get completely for free so that when you're starting out, you don't have to have any of these sort of crazy expenses.
Now, if you don't believe me, uh while pricing on NASDAQ and multiple different exchanges do normally remain hidden, uh they don't publicly say how much they're going to charge, most of the time. If you actually do like a sales call with them or you have contacts, most of the time they'll be charging around $2 to $3,000 per month, sometimes more depending on the firm and what sort of data they need.
My earlier years of trading, I've always wondered, and maybe you've wondered as well, how do you get quality market data without paying thousands for it like this? Now, I used to assume you'd only really get professional data at this premium price. But there are workarounds and there are data providers you can use for certain use cases that I'm going to go over in this video. But I did used to think that if you wanted to build, you know, proper strategies or proper strategies that can do, you know, proper back testing or algorithmic trading, you're going to have to use Bloomberg or direct exchange fees that can cost a lot of money. But after building and testing hundreds of strategies over the past 6 years, both for retail traders and also even for funds, I have made some discoveries like there are some free data providers that can match paid services for 70% of the use cases and how the best data isn't always actually the most expensive. Sometimes it can just be based on the provider.
Now, in this video, I'm going to go over six specific providers you can use to replace $34,000 per year, both for free and also cheaper alternatives. Plus, because I know many people in this video are maybe starting out as beginners, I'm going to go over which ones are perfect for beginners in different sort of markets. But just before that, you might actually be wondering what does this sort of data look like when you're using in systematic trading. I tested all of these providers that I'm going to go through with real strategies both in back testing and also even live automation and then validating them against quality expensive sources like Bloomberg or Faxet or different sort of data providers.
Now often you're going to be pulling this data and sort of creating price charts like this. This is like really uh this is basically I think it was 1 second or one tick uh data for CCXT which is on crypto markets. And additionally if you want code examples for accessing these sort of providers check out the boot camp in the first link in the description. I have kind of code templates for each one of these providers. Anyway, let me show you the six data providers that will save you $34,000 per year and which ones you should use for different specific needs.
Now, the first one I'm going to go over, hopefully everyone's heard of this before, is Y Finance. This is primarily a kind of Yahoo Finance API and it has been one of the top repositories for a very long time because this is normally the way that most people get into systematic or algorithmic trading is just simply downing some Yahoo Finance data. Also, especially with AI nowadays, most AIs are trained on Yahoo Finance data, meaning they know how to interact with it, know how to use it, etc. So, this is a classic choice for newcomers. Um, what it really includes is you get equities, forex data, a bit of crypto data, and also ETFs data. Pretty much anything that's on Yahoo Finance's website, you'll normally be able to get through their API as well. Um, even though it's not directly supported.
Now, what is great for, it's dead simple to use. Even if you have no coding experience, you could probably get AI to, you know, pull some Yahoo Finance data quite well. It covers most asset classes. Uh, and it does also have historical data going back decades depending on the asset. And lastly, most importantly, it's completely free. So, if you're just trying to get started and you don't want to spend, you know, $100, $200 per month on data, you can kind of get this up and running very easily.
Now, obviously, there's going to be cons to this, right? So, the data quality can be spotty, meaning you're going to sometimes get missing borrows, sometimes going to be incorrect uh dividend adjustments, sometimes there's going to be delayed updates. for example, I just was looking on Yahoo Finance data um and like Nvidia didn't have uh one of the recent uh daily bars. That's going to be things that's going to happen with Yahoo Finance data. So, my recommendation really is it's perfect for initial testing and understanding Python as well. If you've never used Python before and you just want to kind of get and use some trading data, Yahoo Finance's perfect for it. But I would always validate your data. If you're going to do any like serious uh back testing or even automation using some of this data, I'd be very careful around that. But for like 80% of retail traders testing on daily or weekly strategies, you're normally going to be fine. I would say in most cases where you're just getting an initial test done. And even to this day, sometimes I, you know, I pull up some Yahoo finest data if I'm just trying to pull some very generic uh data. Um, and you know, this is like the quickest route.
Now, number two is Panda's data reader. Most people haven't actually heard of this one. Uh, it is slightly outdated, but it can be useful for specific use cases. What makes it different to like Yahoo Finance data for example is it connects to economic data sources, not just price data. So you could get things like Federal Reserve economic data, you could get World Bank data. They have a lot of different things you can look up and kind of read through. And for each one of these, if you're not too sure where to like find out information around it, just simply type in the name into Google and it will normally pull you up a page with their PIP install and it will give them documentation, general information around it.
Now, as with Yahoo Finance, this one's completely for free. My recommendation here is if you're trying to implement some kind of economic trend data uh and you're on a budget or you're just kind of testing that or you're trying to build maybe a small project, this is a great one to use just to kind of get started.
Now, number three is kind of where we get a sort of serious pick, I'd say, which is IB Incync. Now, IB Incync I actually use for automation when automating on interactive brokers. If you hadn't guessed, uh IB stands for interactive brokers, and in sync is basically the way it connects. Now there are a couple of uh alternatives to using this but in my opinion from using pretty much most of the Python libraries um IB in sync is normally the best one and normally doesn't have that many issues.
Now what you can do with this uh Python library is it basically interacts with everything on Interactive Brokers TWWS. So you can get real-time market data. You can also pull historical price data and you can also do order execution if you want to submit an order. It covers really everything that Interactive Brokers covers. So you can do stocks, futures, options. Uh they do have sometimes a bit of annoying ways to connect to these different asset classes. They're not always the same. So just keep that in mind.
Now, the requirements, uh, you will actually need Interactive Brokers accounts. Now, this could be a demo account. You could, uh, like download a demo TWWS just to get a general idea. Um, but I'd probably recommend just, um, actually creating Interactive Brokers account, funding of a small amount. But I do believe last time I attracted, you can have it unfunded and still connect and work. But obviously, if you're going to have like actual live market data, you're going to need to be paying for that market data through TWWS. Um, and you'll be able to just, you know, connect it through this Python library. You also will need it running when you're trying to pull data from um TB2S which is you know the platform IB is using. So it's going to need constant uh connection.
Now this is pretty powerful for a lot of beginners because you can back test and also execute live with the exact same data source and most of the time there shouldn't really be any discrepancies between back test data and live data. Now the catch here here is you're having to keep TWWs open and connected. it can be finicky with connection stability and you don't really get the most data further um going back meaning you know compared to like let's say Yahoo Finance at least from the beginning of this list that can have decades of data IB is not going to have decades of data on most particular tickers so that's just something to keep in mind you're not going to be able to back test as much data with it my recommendation is if you already have an interactive brokers account this is definitely a no-brainer to get started just to test it out see how it works um test it out with very small live orders or like demo orders for example and also see what data you can sort of pull yourself uh from For the most part, Interactive Brokers does have great documentation. Uh, same with IB Insync has pretty decent documentation. So, you should be able to work out a lot of it or ask AI and kind of uh, you know, do some back and forth and you'll be able to work out a decent part of it. Now, keep in mind though, if you're going to be deploying, you know, serious capital or any actual capital that is meaningful to your account. This one you definitely have to get up to scratch with. You have to really understand it quite well to be able to do a full automation through it.
Now, next is Polygon. This is my main pick. I probably talk about this on most videos. Polygon is my favorite for clean and reliable data. They offer most markets, so stocks, options, forex, crypto, etc. Now, I will be transparent. The main one I use is for stocks. I don't really use the other offerings. I've tried their options data. It's okay. Uh option data. Um, if you're trying to get into options, for example, using sort of API data is very difficult and can be very expensive to get. Uh my recommendation just for options would maybe be feta data. Uh they do charge, I think, a decent amount, but that's really the best quality data. All other options providers really are very low quality in their data, at least in my experience.
Now, why does Polygon stand out and why do I use it to this day? Because I've been using it for like five years, four years, give or take. Um, and the data quality for the price is amazing. Now, this is a misconception in trading. People think there is a uh standardized kind of pricing data. They think there is a central hub that everyone is getting this data from and that price data is correct. Not true. Uh, trading data overall is unstandardized. meaning that each brokerage, each uh data provider has different ways they interpret price data. Uh meaning you know how they interpret volume, how they interpret whether that price has moved or not. Now obviously dayto-day most of the time they're going to sort of line up and depending on the liquidity of that market, meaning how much volume is coming through, most of the time they line up closer and closer and closer. But if it's something that's kind of illquid, you could definitely have one data provider saying one price and it's it's completely different on the other uh data provider. So please keep that in mind. You're not going to ever really get perfect data in trading. Uh, and if you are trying to get to that, you know, pinnacle of 99.99% accuracy, that's where you're going to have to be paying $34,000, you know, per year to get that sort of quality. But for most retail traders, you're not going to need uh, you know, tick by tick or those sort of crazy uh, levels of data because you're not doing market making or things like that. If you're executing a daily strategy or a 5 hour, 4 hour strategy, you're not going to need that sort of tick data.
So, that's why most of the time for me, Polygon's perfect because they've got proper split uh, and dividend adjustments. They got reliable uh aggregations. They got great fast API response times and they got great endpoints. Um, this is a thing I think a lot of people overlook maybe if you're new to development is the endpoints meaning what you can call to kind of get data back really uh can affect your workflow. For me, Polygon has a couple great ones like aggregated daily bars and also some great snapshots which mean you can pull a lot of data very quickly in a very efficient manner. Whereas with other providers, you'd have to do an individual call and loop that like 100 times to get all that data, which isn't a very reliable process. Also, just takes a crazy amount of time. So, Polygon's great for those sort of things. Also, they're quite reasonable with pricing. You can start for free, but they got a really uh small amount of API calls per minute, which is five. Or you can pay $29 per month and get unlimited calls um per minute basically. And you can get a lot more data. You get 5 years of historical data. So, this is normally what I'd recommend cuz $29 at least, you know, in trading terms, you probably lose more on an individual trade than $29. So, I'd say it's always worth it.
Now, next is Alpaca PY. So, this one I've heard friends use quite a lot. Um, I've tested it a couple of times, but I don't personally use it full-time. Like, I use Polygon and IB Insync quite full time, but I don't use this one as much full-time. At least from what I understand, it's Alpaca is primarily like a mixture between a brokerage and an API, meaning that you can have an API that sends off orders and they send off that order for you. And it's also like commission free, I believe, and they've got a couple of uh different benefits.
Now, the one thing I did enjoy from their offering is they try and be all-encompassing. They've got quite a lot of things that you can get started with very quickly. Uh, and they try to provide you with all the basics that you would need to get into, let's say, algo trading. However, I would I would say the limitation for me is that with anything, at least for me, for coding, I like to be in control of the development side of things. I like to know all the assumptions being made. I like to know what assumptions I'm making myself. Um, and when you're going through a provider like this, sometimes that can get a bit unclear and that can cause sometimes issues. So, I would recommend to try it out. They've got a really decent free tier that you can try out and pull some historical data and also send off some orders. But, if you're trying to do um depending on the strategy, I wouldn't always recommend it. um if it's going to be lower time frame strategies, if it's going to be something that's more complex, meaning you've got a lot of logic or you're combining multiple different data sources, wouldn't really recommend it. Uh sometimes if it's a very simple strategy, then yeah, I maybe I'd recommend going through uh Alpaca. Just depends on the case by case, but definitely something worth to check out.
Next, we've got Binance data. So, Binance is my go-to for crypto data and it's completely free. And that is one of the big benefits around trading crypto is for development side of things, you are spoiled. You have amazing documentation. You have amazing APIs. That's primarily all free. And you have amazing amount of data you can go through. And Binance is probably one of the most liquid and also um they have the most historical data for the most part, at least for free. It's what I use to connect real time to market data via the websockets. You also have candlestick data for really any time frame. You've got aggregated trades and it supports everything that's on Binance or all crypto pairs. You can also do pers, you could do spot, meaning um the underlying, you can do the derivative. Both of those work perfectly.
Now, I use it primarily because they have the deepest liquidity. In crypto, the biggest issue with pulling data is because uh there's so many different exchanges, right? And each exchange has different volume. You normally are going to get differences between those exchanges or very small differences sometimes. So, what you'd normally want is to pick the one with the deepest liquidity because that's going to be the most, let's say, accurate data on that standpoint. So, that's why I normally go with Binance. Uh, and they also have very clean data quality and they're also up most of the time. In case I didn't mention it, you don't even need an API key to connect to Binance. you can just go through their open uh API and you can just download historical data very easily through it. This is great for back testing. This is great for developing strategies overall on crypto and this is also quite easy then to execute on Binance uh for example using their API. The only limitation here is obviously it's only really for crypto. So no stocks, no futures, no forex, anything like that. Um, but for crypto I would say this is pretty much the gold standard. There are a couple of paid versions above this that can be very good in specific cases or depending on your need of quality of data.
My recommendation is if you're actually just building anything in general, I would start in crypto. Whether that be you have a bit of developmental experience or you have none. Main reason being is you've got the lowest overhead for testing and just finding out. A lot of learning to code especially for algo trading is doing projects doing um actual live things. So you know executing some orders on very very small size uh back testing strategies yourself. Um, and for that most of the time when people are starting out they don't want to have a ton of overhead like data cost etc. Um, and for the most part, crypto is one of the better places for documentation, also learning resources. Uh, there's a lot of people using it. The only downside, right, is that obviously crypto is a bit more volatile there. It's going to depend on what strategies you're executing, but for at least just learning, I would say it's a really good place to start out.
Now, one thing just I want to remind everyone with this video is that these are all meant for beginners. Um, if you're starting out and you don't want to spend a ton on data, these are great resources for that. But if you are needing to get the highest quality uh accuracy or if you're needing uh you know completely survivorship bias free data all those sort of things that's when I'd recommend moving over to paid options normally those paid options what I would recommend is Polygon very cheap and very affordable and also very good quality data also survivorship free uh data for crypto I'd still recommend Binance or Hyperlquid both those are great options for options that's probably going to be Feder data and then for futures data that's normally where I would go for a platform like I do for Ninja Trader just because futures data to get really high quality futures data, you normally have to pay professional fees, which is going to be around at least 1 to 2 grand per month. Um, and normally you can get away uh through that by just using a platform like Ninja Trader and then you're licensing data basically from them for, you know, maybe 50 bucks, 20 bucks per month. So that would normally be my main recommendations. But of course, if you want code examples for all of these providers, I actually have them with setup instructions in the boot camp which is linked in the description below. Feel free to check it out if you want help with using any of these data providers.