Transcription
Hello everyone, and welcome to the Numeraai Fireside chat for Q4 2025. My name is Noah Harz. I'm the Chief Tournament Officer here at Numera. I'll be hosting the fireside chat alone this time around, and that's because Richard, as well as the entire Numera team, has been very busy this year. Richard has been fundraising our Series C, which you might have seen on our blog yesterday. He's also been talking to LPs and leading research projects to improve the fund.
In fact, the entire Numerai team has made 2025 a fantastic year so far, and we still have a month left to land a few more changes. Uh, you can see here, here I've teased this on our Discord announcements channel. Uh, we have a few more updates coming through the pipeline. Uh, so stay tuned for more on those.
But today, we're going to be focused on signals payouts, and I'll be proposing that we increase the clip on our signals tournament. Uh, this is something that many of you on Discord have mentioned and requested. So let's take a closer look at that.
Today, we introduced the new signals alpha and MPC scores at the end of July this year, which was a huge advancement in the payout feedback loop of our hardest tournament, signals. So, how is signals holding up since that update?
In terms of actively staked counts, you can see here that we've been stable recently, around 40 to 50 users. Other than the perturbation around July due to the alpha and MPC announcement, we've mostly returned to normal, and signals seems healthy according to this metric. Stake submissions is also relatively healthy. Uh, in fact, we've seen a modest increase since one year ago. Total stake on signals is down from this summer, probably in part due to the alpha and MPC launch, uh, and a few whales leaving the ecosystem, but it seems to be recovering. Uh, we would like to get back over the 100k NMR mark, but we don't really see any glaring problems with signals as it stands now. So, to answer the question, yes, signals seems okay. Um, but, you know, I think it can be better.
So, let's talk a little bit about payouts. The three primary goals of payouts are to ensure stable value for a hedge fund. And to do this, we have to reward our good users, the best users. Uh, but, you know, we also kind of have to bound this risk of ruin. Uh, that third primary goal is why we have clipping in our payout function. It ensures that we bound this risk, and so, uh, it reduces the likelihood that any one difficult era of burns is going to completely bankrupt a user. That's not what we want to do here.
So, with the introduction of alpha and MPC, while it does provide value to the hedge fund, it also reduced compounding and it lowered the clipping, uh, to also reduce risk, thus lowering the profitability of the signals tournament to some of the best users. Uh, I calculated the payout percentages for each user for each round over the last year and plotted a histogram that you can see at the bottom of the slide here. Uh, on the left side is the correlation plus MMC payouts that we were at before introducing alpha and MPC, and on the right side is the alpha and MPC payouts with the 1.7% clip. And you can just see that the 1.7% clip has drastically increased the proportion of users that get clipped. It's compressed this histogram significantly. And the bars on the left and right sides indicate this clipping is impacting a dramatic number of users.
It also hurts max return. So, if I simulated the compounding return of these payouts and we looked at the one-year return, basically the total return possible assuming a unit stake of one NMR for staked users. It reduces the max return that our best users can make. Uh, some of our best users were sometimes able to make 10x, 20x, 30x on the past payouts. Uh, but over the same period under an alpha and MPC with a 1.7% clip, they're making much less. That's unfortunate. And it also drastically, uh, it also drastically reduces, uh, the likelihood that you're going to get more upside than the risk of getting downside here. And you can see that from this histogram, the negative portion, the users in the negative portion of the histogram have increased as well.
One silver lining that kind of comes out of this is that Sharpe ratios of returns over this one-year period, uh, actually got better for our highest Sharpe users. So, users that were earning the most, that, that had the highest Sharpe, rather, they got even more Sharpe, which, you know, if you're in finance, you know that this is risk-adjusted returns. It's, it's a valuable number to keep track of. So, there is something here to alpha and MPC, uh, that is improving Sharpe of your payouts, but we're still clipping those payouts a lot, and it leads to really low one-year returns.
So, how do we fix this? Well, I propose we start using a 3.5% clip. So, we go from the 0.17 to 0.35. That's nearly doubling the clip. And we can see what the effects are of this. So, just plotting this histogram again, going from a 1.7% clip on the left side to a 3.5% clip on the right side, we can see that significantly fewer payouts are clipped. Uh, this maintains more information in this payout feedback loop. And we'll see the effect of that in the next slide as well. It allows good users to earn more. So, some of our top users under a 1.7% clip over the last year could make maybe 6x or 7x, uh, but under a 3.5% clip could maybe be making 10x or 12x, uh, in that one-year period. One unfortunate thing here is that the worst models under this framework also increase risk. So, that's something to keep track of.
Interestingly, it does contract the range of Sharpe values. So, your Sharpe ratio for the best users under, when we increase the clip, your Sharpe ratio will be reduced. But the downside for the worst models is also contracted. So, the worst models actually end up getting slightly better Sharpe, slightly less negative Sharpe. This might be a reasonable trade-off to make. Um, and 1.5 Sharpe is, you know, not necessarily terrible.
So, to summarize, under a 3.5% clip, which is double what we have now, we will actually only be clipping 6% of the time instead of 22% of the time over a one-year period. Top 10% of users, the best 10% will make 85% one-year return over the last one year instead of 62% over the last one year. And the top 1% would have standed, would have stood to make 312% instead of 221% over the last one year. Uh, and down here at the bottom, you can see different, the, uh, return streams of different percentiles of users. Uh, the green line being top, the 99th percentile user. Orange is 95th percentile, and blue is 90th percentile. And the ranges of all of these return streams, uh, just get better with a higher clip.
This is probably seeming quite nice so far. Uh, and there's one more kind of important thing to stress here. All of these simulations were ran on backfield scores for models that were not optimized for alpha and MPC. And that just means that the risk might be overstated here. The max return might be understated here. And increasing the clip value in signals is probably going to have a better chance at improving your profitability, uh, under an alpha and scoring framework. We might even be able to raise the risk even more if everyone ends up feeling safe in their modeling under a 3.5% clip.
My plan is to be, uh, is to post a poll in Discord, uh, with different clip values and, uh, let everyone vote on it and figure out what is the right risk-return trade-off for the community. Uh, my goal here is to include all of you. Obviously, we want signals to be valuable for the hedge fund, but we also want it to be valuable for you, and we care about your feedback, and we're trying really hard to make this, uh, kind of a deeper analysis than we've ever done.
Okay, so that's it for signals payouts. I do have a couple of bonus slides here, uh, and we'll hop into those now. First bonus slide is about Numerai payouts. Uh, we're going to be moving to a Tiger-like target. Uh, internally, we've seen that correlation against Tiger is lower on average. So, we're also considering increasing either the multipliers or the payout factor for a time to balance this reduction in correlation. Stay tuned for more updates on this. Uh, we'll plan on having some backfills done by mid-December, and we're going to announce the final multiplier and payout factor values for 2026, uh, before the season starts.
Additionally, we want to add core payouts back to crypto. A lot of you have been asking for this with the introduction of the V2 universe making the competition a bit harder. Uh, we're going to reintroduce core payouts at a small weight. Uh, this will require reducing weight on MMC to bring the profitability of the crypto tournament in line with the other tournaments. But this payout formula will increase Sharpe of your returns on Numerai crypto using the same simulation framework that I used for signals.
So, my proposal for all the payouts, this is my final proposal slide. The breakdown for 2026 is we keep Numerai basically how it is. We're going to update the target to the new Tiger target. Uh, obviously, this is subject to change based on simulated returns that we see in backfills, and we'll, we'll keep you updated with that. I suggest that signals should move to a 3.5% clip, and crypto should switch to 0.5x core plus 0.5x MMC.
Okay, we'll be moving on to the Q&A portion. Pulling up the Slido now.
Okay, for Q&A, Wim asks, "Could you reflect on 2025, please? Discuss the good, bad, and ugly aspects, and also share roadmap and plans for 2026." Uh, yeah, this is a good question. Obviously, 2025 has been great so far. I showed you that slide at the beginning. If we just go back. Yeah, 2025 is great. We raised a $30 million Series C, 550 million in AUM, and 500 million JP Morgan capacity. All these were announcements on our blog. So, if you follow our blog, none of this is news to you, but, uh, I think if you've been, if you've been keeping up with Numerai, uh, you see the rosy picture for 2025. And 2026, we're going to be planning soon. Uh, obviously, we have Numercon coming through, and with that is going to come a lot of planning and a lot of announcements. So, make sure you're staying tuned to Numercon. If you don't have a ticket or bookings yet, um, feel free to reach out to contact@numer.ai and, uh, and stay tuned. Yeah.
Woke also asked, "Can you give an update on the new master plan, please? Where are we on the roadmap?" Yeah, we, I mean, we care about the new master plan. To us, uh, this is a very long-term vision. If you haven't read our new master plan blog post, go back and read it. It's on our blog. We do care about the master plan. We are keeping track of it, and more to come at Numerai.
Anonymous asks, "Will you be planning to deprecate V5.0 and V5.1 pipelines? There are a couple of models I've tweaked with a lot of research for V5 data, and it would take months to redo this research for new data sets. Like we had it for V4.x, can we continue running V5.x? We hit a big blocker." Yeah. Uh, I think we're perfectly comfortable keeping the old data sets around. Uh, we don't want to break anyone's pipelines. We just want to give you new features at this point. Um, as many of you know, OG users will know, going from V4 to V5 was a breaking transition. That's why we did the major version change, but these minor versions, we tend to keep around for a while, uh, because we, we still appreciate all of the hard work you put in, and we know that these models can still be valuable. So, old versions of V5, they're not going anywhere anytime soon.
Anonymous asks, "What's the purpose of a data release V5.1 if a V5.2 was planned for a month later?" So, yeah, this might seem a little confusing from the outside perspective. Users are like, "Well, you know, why not just wait for V5.2?" Um, we're always pushing, and we're always finding new data, and we care about giving you the latest data. So, for us, it's not so much about, you know, what is the right time to release these things, we just want to release them as soon as they're ready. Uh, if we know it's an improvement and it's ready, we just push it out because we know it's going to make you and the tournament better. Um, and as soon as we release V5.1, we get back to work and, and we keep pushing, and we found out that, oh, we do have a lot more value to add here, and that's why we're cutting V, V5.2 so, so much closer. And, um, you know, we haven't touched data in a long time. V5 was, was a big update. Uh, but we've been doing a lot of work, and we had a lot of value to add. So, we just wanted to add that value as quickly as possible, and cutting new versions is the easiest way to do that.
WE asks, "When stake management take profit returns NMR back to the wallet? So you could allow auto-restaking the balance into chosen models' tournaments with configurable percentages and a scheduled cadence." Um, yeah, stake management is kind of a meme in the community. I've been putting a lot of thought into it. Um, and I want to, I want to create a very concrete plan for implementing a really good stake management system for all of you, and maybe I will be announcing that at Numercon.
Anonymous asks, "Have you seen Numerile? What do you think about it? Can it be useful for Numerai and meta models?" Um, yeah. I don't think we have looked too deeply into Numerile. I know I looked briefly at the website. It's trying to predict performance of models. Um, can it be useful for us? I'm not, I'm not sure. You know, maybe it's useful for people that are looking to buy models on NumerBay potentially. But for us, um, kind of my thinking on this is, is if you have predictive power, if you know how a model's going to perform, why not just encode that predictive power into the model itself and make it perform better? Like, if you know, if you know a model is going to perform really well or really terribly, why wouldn't you just adjust the model to then perform even better? Um, I, I think it's, I think it's a fun project, but yeah, I'm, I, I haven't looked too, too deeply into it.
Anonymous asks, "Don't you think it would be, it would make sense if you fill the sparse features in the V5.1 data set yourself since you have a better understanding about the true nature of these features and can make meaningful backfills?" Um, yeah, this is a really good question. Nan-filling is a, uh, it's a big point of conversation internally, and it always has been, and I think it, I don't know, it might always, it probably always will be. I think our, the, basically the final decision on this was, it's going to be difficult for us to determine the best imputation strategy, and we know that all of you, the data scientists, have different strategies for imputation and nan-filling [clears throat] and nans. Uh, there's an argument here that nans are a piece of information themselves. So, not having that data is actually information in and of itself, and you might be able to utilize that piece of information better. So, if we did something very simple like imputing with median or something, um, you might be losing that information, and you might have a better imputation strategy, uh, generally for data sets that you find works better and produces better models. So, we just didn't, we didn't want to make that decision for you, and we wanted to leave it up to you to, to decide what the best imputation strategy was for your model.
What is the current state and future of Numerai crypto? Anonymous asks. How is it received in the crypto community? Yeah, Numerai crypto is interesting. I think there are, uh, there are some individuals in the crypto community that really like the idea of crypto. They're even using the, the crypto meta model at times. Um, the current state is that it's steady. Numerai crypto is stable. Um, I talked a little bit about it in this presentation. I do want to bring core payouts back to, back to Numerai crypto, um, and what the future of it is. I think it's going to continue being a stable, um, value add for NMR. You know, um, people that have crypto data can feed the Numerai crypto meta model and can make really good decisions about a market-neutral crypto market-neutral portfolio. Um, we don't currently have any big plans for it. Uh, but as soon as we, as soon as we nail something down, we, we will let everyone know.
WE asks, "Is the signals swim stake-weighted meta model being used now as input to your optimizer to construct your target portfolio? If so, is it useful and additive to Numerai tournament swim?" Uh, I think the answer we can give on this is that we think signals is useful, and everything we've done to implement improvements to signals in the past couple of years, V1 data, V2 data, the churn threshold, alpha and MPC, the turnover threshold, all of these things have added value to the signals meta model. It's, it's made it much better, and we want that, uh, we want to continue that value add. We want signals to continue adding value. So, uh, we do think that it is quite valuable. We want to, we want to make it even better, and we want to make it better for you as well.
Anonymous asks, "Everyone puts a lot of effort into improving the models, but quote unquote the other way. Predicting model performance is underexplored, and so is cross-ensembling. Do you agree?" Um, there's kind of, there's kind of two questions in this. So, number one is predicting model performance. Uh, we just talked about Numerile. It's kind of trying to do that. And I think if you can predict the performance of a model, you could probably improve that model. So, I'm, I think I think it's probably better to put that work into just improving the model and making it better in backtests and diagnostics. The second question here is, uh, cross-ensembling, uh, and is there a lot of effort put into this? Uh, the answer is yes. That's what we do, uh, on the, on the fund side to generate meta models. There's been so much research put into how do we ensemble users the best way. And it's kind of funny because we always kind of keep coming back to this really simple stake-weighted meta model solution, um, that seems to work so well. Uh, we did put a lot of research into signals for how to best ensemble users there, and that's where we got alpha and MPC from, is that we created these neutralization matrices and ensemble weight, uh, vectors that when you apply those, uh, to ensembling a bunch of users, you find that the meta model that you produce on the other end does also get better. So, if we also use these things to score users, it'll, it'll make that ensemble even better. Um, and that's kind of where alpha and MPC came from.
Any plans to decentralize staking? Uh, eg bring your own wallet and/or scores, trust but verify. Uh, what are the benefits, drawbacks, limitations, risks? Decentralized zero-knowledge proof of score, perhaps? Uh, yeah, good question. We have been thinking about this. Um, I will say that again, this kind of ties into stake management and bring your own wallet and decentralization, all kind of tie into this. Uh, I want to create a proper, fully formed proposal and I want to present that at Numercon, and we're going to talk a little bit more about that later in 2026. Uh, some interesting thoughts here are what are the benefits, drawbacks, limitations, risks. Um, I'll say that something very general about decentralization is you have to be sure that the system is concrete and unmoving because it's incredibly difficult to update once you've decentralized. And once something gets to a point where you're confident that it's not going to be updated drastically, uh, and that you know that the system works and it will continue working, at that point, it makes sense to decentralize. And so, we could be nearing that point. And I think there's, there's plenty here for us to explore and plan, and we're going to, we're going to keep thinking about this for sure.
Joe asks, "How many of you are planning to join us for Decentralized AI Day San Francisco on January 27th?" Um, as far as I'm aware, I believe Mike P and Michael Oliver, uh, our Chief of Data and Chief of Research, respectively, uh, will be doing presentations at the Decentralized AI Day in San Francisco. Uh, this is a few days before Numercon. So, if you're part of the, of the Numerai community, if you're one of the Numerai, uh, come by the Decentralized AI Day in San Francisco on January 27th. You can see Mike P present on data. You can see Michael Oliver present on some research. Um, yeah, it'll be really great.
Anonymous asks, "Daily crypto rounds in 2026." Um, maybe, kind of maybe. Uh, one thing I will say here is that during the V2 data update, I overhauled the data pipeline for how we generate our, our crypto data set, and that does run every day now, including weekends. So, it's been in the back of my mind. I've been thinking about it, and we are making engineering choices around this. Uh, we are moving towards that world. Uh, I can't tell, I can't give you a date. Uh, I can't say for sure it'll happen in 2026, but, uh, stay tuned.
Anonymous asks, "What do you think about cooperation of two or more competitors, ensembling and splitting the payouts, reaching more than 2x the individual payouts?" Um, I think cooperation is fantastic. We love the idea of cooperation here. We think that if more users collaborated together, if more data scientists got in a room and planned their models together, uh, you could end up making better models. We could end up with a better tournament and a better meta model, and it could improve the overall system, right? And it could also push other users to be better. So, yeah, cooperate, get together. Um, you know, we, we don't have any structure on our side to enable that. Um, but we think it's fantastic to cooperate. So, definitely check that out. Let's see. Just going through a few of the pending options here.
Okay, Linkster asks, "AI/LM should help to lower the barrier to entry for new competitors and data scientists. Is there a plan to promote Numerai and try to increase the number of participants in the coming year?" Um, short answer, yes. I always want to find ways to, uh, reach new and better, uh, data scientists and users and players. And, uh, I think 2026 is going to be a great year for onboarding integrations with AI and LLMs. And, yeah, stay tuned because we got a lot of ideas and we're going to execute on them.
Okay, just reading through a few other options here. Anonymous asks, "In the V5.1 data forum post, it said that the new features are great by themselves and are ensembled to improve performance. Do we now want feature correlation with these new features instead of aiming for feature-neutral predictions as advised before? Feature neutrality is more of a proxy for potentially good performing models. Uh, with the thought being that if you're feature neutral and you still have correlation, uh, that you should be relatively immune to feature inversions. Uh, and what that means is if a feature is working for a long time and then it just stops working, if you're exposed to that feature, that means your model will stop working. Now, new features like the stuff in V5.1, um, I think it's up to you. Maybe for you, it does make sense to be exposed to some of these features early on, uh, while they're working. Maybe you create a few models that are specifically very good at these features, and maybe you put a small weight in your account on those things. Um, but that's, that's up to you. I mean, it's, and, and there's also a world in which you train with these features and you still neutralize and you ensure that you still have alpha and information coming from these features without being fully exposed to them. Um, so feature neutrality is more of like a risk control than it is a, you know, silver bullet for making the best possible, uh, models."
Anonymous asks, "Will you move the open of the weekend round to Monday so that in signals we can have live sample weights and neutralizers?" Uh, this is something I want to do. Um, I planned it for this quarter. Uh, with the target updates and payout updates coming through the pipeline. Uh, I'm going, I'm still going to try to get to this, and we'll see if we can get it before 2026. Uh, but worst case, uh, I want it to happen in Q1 2026 by the latest. So, yes, it is coming, and, uh, and we're gonna, we're gonna do it.
Okay. WE asks, "With the increased clip in signals, you could easily lose more than what you've staked. Will you require topping up negative balances before allowing staking more in signals? I kind of think you should." Yeah, this is a, this is a good question, and it's kind of something I went back and forth with Richard about. We, we know that there's a bit of an asymmetry here, right? If we allow negative stakes, there's a bit of an asymmetry. There's a bit of an attack. Uh, so the trick is to still have high payouts while trying to limit this asymmetry. And I think something like a 3.5% clip, uh, kind of strikes a balance between these two things, right? We're not just allowing infinity leverage, but, you know, uh, but we're also kind of like trying to give you a little bit more of a leg up, a little bit more returns if you, if you're good. Um, also, we're kind of the opinion that nobody's going to try to lose all their money on purpose. So, that's kind of, that's kind of something here that, you know, I think if you're thinking of the rational data scientist, rational user that's looking at signals, that's playing signals, um, nobody's going to try to get a negative stake value. And if they do, then it's just kind of like a permanent mark on their account that like, yeah, I didn't do risk management correctly here. Um, and I maybe staked too much or I yoloed too much on this one model. Um, obviously, I think risk considerations are different for everyone, but I think the 3.5% clip is kind of the right balance to strike here. Um, but again, I'll, I'll be putting a poll on Discord, and we can see how, how the community reacts.
Okay, Wim asks, "When will V5.2 be released again? Any more targets other than Tiger 3? How many more features? How much better are they going to be compared to V5.1? Are they just transforms of V5 features or new features?" Uh, yeah, all this information will be released in mid-December. Um, as soon as we have all the features ready to go and tier 3 ready to launch, um, everything will be pushed out mid-December, including backfills, including, um, payout simulations that I'll run, and a final decision about the payout formula for Numerai.
Sebastian asks, "I'm a beginner. After trying examples, after reading all of numer.ai/home and office hours, where is the best next place to go to learn about how to train a good model?" Yeah, that's a really good question. Um, obviously, we have all of our tutorial notebooks, uh, that give you some starting stuff. Uh, obviously, we have all these office hours, and another place to go is the forum. The forum is always a great place to go, and, uh, the fourth place that I'll say is Discord. Everyone in the community is, is quite helpful, and if you ask, you know, meaningful, thoughtful questions, there's always going to be someone there to answer them in a meaningful, thoughtful way. And you can learn a lot by just asking questions and continuing your path. Uh, it just takes time. So, keep going. Check the forum. Check Discord. Um, and, and start reaching out to people and say, you know, I'm looking to learn. Um, yeah, keep, keep going. It just takes time.
Okay. Anonymous asks, "Can we expect to have the model scores backfilled against the new target and payout formula so we can have an idea about how our models would perform once the changes will be introduced? If yes, by when will this be done?" Uh, yeah, mid-December. Mid-December, we're going to have V5.2 out. Tiger 3 is going to be out. I'll perform the backfills, and I'll do the payout simulations, and we're going to try to have everything out before 2026 starts. Um, so that you at least have some information to base your, your decisions on.
Okay. Yeah, I think, I think that's, uh, all the questions we're going to be answering for today. Okay, last question I'll take from. Will you continue to release new data sets before new future seasons start, or will we continue to expect new data sets throughout the year as well? Uh, I think V5.2 will be the last data set we release for this year. As for next year, I can't speak to what we have in store. Uh, we're always looking for new data. We're always looking for new features, and we're always looking to improve the system. So, as soon as we find something that improves the system, we're going to tell you about it. We're going to release it. And, uh, we're going to try to get everyone on board with, with improving the meta model, improving your models, and making everything better for everyone. We're aligned on this, and, and we want everyone to be, to be in it with us.
All right, thank you everyone for coming to the Q4 2025 Fireside Chat.