# Can You Beat Random Numbers? **URL:** \ **Category:** Signals\ **Created:** [September 1, 2021, 9:55am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022 "2021-09-01T09:55:15Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![robo\_boi](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/robo_boi/32/2311_2.png) [@robo\_boi](https://forum.numer.ai/u/robo_boi)\ **Post date:** [September 1, 2021, 9:55am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/1 "2021-09-01T09:55:16Z")
As an experiment, I’ve been submitting random numbers between 0 and 1 for all the available tickers. The model is called ‘[Jersey Devil](https://signals.numer.ai/jersey_devil)’ and is currently ranked 71 for corr 20 as of round 278 with one more day to go. I’m not staking on this model (just the minimum .01 to see returns). I use it solely as a baseline for my other models. If they can’t beat random numbers, they die. Yes it hurts when you put a lot of time and effort into a model and it doesn’t do better than random numbers 😆. I was pretty surprised this has positive corr at all but we’ll have to see if that continues in the future. [Here](https://colab.research.google.com/drive/1T36WcUhq6v-NGsU_Cme96ieY6NDkGk86?usp=sharing) is a colab notebook with the exact code. _Jersey Devil vs My Current Best Model_ ![jd_ogo](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/9/9d324614851ce0c53c77ee62fc3ab2889d185e63.jpeg) ![jd_ogo_cumm](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/e/e8f5eacb439d0f1f9b4a3bd83a8c1788e98c3f81.jpeg) ---
**Author:** ![of\_s](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/of_s/32/2627_2.png) [@of\_s](https://forum.numer.ai/u/of_s)\ **Post date:** [September 1, 2021, 1:55pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/2 "2021-09-01T13:55:21Z")
This would be a perfect demonstration case of the effects of neutralization. Reporting pre- and post neutralized scores would be highly informative. ---
**Author:** ![objectscience](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/objectscience/32/2913_2.png) [@objectscience](https://forum.numer.ai/u/objectscience)\ **Post date:** [September 1, 2021, 4:57pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/3 "2021-09-01T16:57:09Z")
My ego could never survive this. 🙂 ---
**Author:** ![rapidautumn](https://avatars.discourse-cdn.com/v4/letter/r/54ee81/32.png) [@rapidautumn](https://forum.numer.ai/u/rapidautumn)\ **Post date:** [September 6, 2021, 6:43am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/4 "2021-09-06T06:43:54Z")
An interesting experiment, thanks for sharing! This is not surprising on second thoughts. A randomly selected portfolio of stocks often performs quite well, especially in long bull runs. From memory there are some studies showing that it handily beats most fund managers. Whether a random portfolio could also beat a hedge fund with great data… I suspect no self-respecting hedge fund would submit to that test. ---
**Author:** ![robo\_boi](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/robo_boi/32/2311_2.png) [@robo\_boi](https://forum.numer.ai/u/robo_boi)\ **Post date:** [November 26, 2021, 10:21am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/5 "2021-11-26T10:21:56Z")
Update. Currently around 30% avg. 3 month returns ![update](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/3/3476ab10fbe307375783044d5030317432d2b7b8.jpeg) ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3582_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [November 26, 2021, 5:40pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/6 "2021-11-26T17:40:18Z")
But are random numbers still even random after neutralization? ---
**Author:** ![autratec](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@autratec](https://forum.numer.ai/u/autratec)\ **Post date:** [November 28, 2021, 4:48am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/7 "2021-11-28T04:48:51Z")
If the random number, beats majority of submission, what does it mean ? ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3582_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [November 28, 2021, 4:54pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/8 "2021-11-28T16:54:29Z")
Doesn’t mean anything really unless it consistently beats everything for a long time. It would be quite expected to have both good and bad runs but be close to zero when averaged over many rounds (but many more than you might intuitively think is necessary). ---
**Author:** ![taori](https://avatars.discourse-cdn.com/v4/letter/t/ebca7d/32.png) [@taori](https://forum.numer.ai/u/taori)\ **Post date:** [November 29, 2021, 11:21am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/9 "2021-11-29T11:21:03Z")
I agree @wigglemuse. Nevertheless, it is a very interesting test and I am glad @robo_boi is sharing that with us. ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3582_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [November 29, 2021, 9:00pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/10 "2021-11-29T21:00:56Z")
Yes. Although, as I joked above, neutralization of random numbers would actually add a tiny bit of structure to the randomness (because that which it is neutralized against is not random). Probably not enough to notice scorewise, and if it did turn random numbers into a consistently positively (or negatively) scoring model, then that would probably indicate a bug somewhere. ---
**Author:** ![autratec](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@autratec](https://forum.numer.ai/u/autratec)\ **Post date:** [November 30, 2021, 12:11am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/11 "2021-11-30T00:11:59Z")
if the random number is able to beat the submission, it probably means the market is moving towards certain direction at the end. and it could be driven, not by the individual company performance, but the the overall trend or human race behavior, like inflation, innovation, corporatization, etc ---
**Author:** ![testorganisation](https://avatars.discourse-cdn.com/v4/letter/t/e56c9b/32.png) [@testorganisation](https://forum.numer.ai/u/testorganisation)\ **Post date:** [December 2, 2021, 1:49am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/12 "2021-12-02T01:49:37Z")
Is anyone able to explain why its corr and mmc track each other so closely? (Of course better corr usually equals better mmc but my models do not display this behaviour) Could it be that if it randomly does well one round, it is likely to get high mmc as other models are unlikely to have submittted predictions similar to randomness? ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3582_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [December 2, 2021, 2:35am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/13 "2021-12-02T02:35:54Z")
@testorganisation Because a random submission will have near-zero correlation with the metamodel and so not much if any of the prediction signal is residualized away in the MMC process. (Generally, correlation with metamodel sends MMC towards zero.) So basically the reason they track the same is because they are pretty much the same. You’ll see this behavior for everybody’s models if the metamodel itself is at zero – when the metamodel essentially is giving no (actually predictive) signal, then each component signal will have an mmc near equal to its main predictions. ---
**Author:** ![bor1](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/bor1/32/3286_2.png) [@bor1](https://forum.numer.ai/u/bor1)\ **Post date:** [December 2, 2021, 10:16am UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/14 "2021-12-02T10:16:07Z")
I haven’t thought about random for signals - but my thoughts on random in classic is that as we are predicting very-close-to-random anyway (correlation strengths of 0.03), it is pretty easy for random to perform well. You could have 10 different random submissions, and see how well they cluster around 0, or that they indeed swing collectively above/below zero as well. In different tournaments, random might never perform well - say, if the tournament is “Landing rovers on Mars”, a random rocket construction and route-plotting program is never going to get close to performing well :-). ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3582_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [December 2, 2021, 3:34pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/15 "2021-12-02T15:34:11Z")
Yes, these tournaments are symmetrically designed so that random is at zero in the center. It is exactly as difficult to score -0.1 as it is to score +0.1. But yeah, since even good score magnitudes are so low most of the time it is natural that randomness can be competitively positive for any given round. But it should over time be negative about as much. (But we also know from random walks that once it is on one side of the line, it can manage to stay there for quite a while as far as cumulative performance.) ---
**Author:** ![kenfus](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/kenfus/32/3397_2.png) [@kenfus](https://forum.numer.ai/u/kenfus)\ **Post date:** [December 15, 2021, 1:12pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/16 "2021-12-15T13:12:04Z")
Maybe I am crazy, but how is 0.011 35% return, when we start with 0.01? Should it not be 0.013? I think the return calculation is just wrong? When we only look at NMR, we can see that it is random because it’s still at around 0.01. Or wait, today minus 3 months equals to 15.09, when the model went down to 0.08 NMR. The difference to 0.011 NMR it is around a 35% increase, haha. So I guess we are not losing (yet) against a random model. ---
**Author:** ![of\_s](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/of_s/32/2627_2.png) [@of\_s](https://forum.numer.ai/u/of_s)\ **Post date:** [April 1, 2022, 4:01pm UTC](https://forum.numer.ai/t/can-you-beat-random-numbers/4022/17 "2022-04-01T16:01:54Z")
The real question is: **Can you beat all 0.99’s**? I was running this experiment from the observations in these two prior posts I shared: - [Possible p / (1-p) vulnerability in Signals](http://forum.numer.ai/t/possible-p-1-p-vulnerability-in-signals/4314) - [Is TB200 neutralizing the Moore-Penrose Inversion neutralization step in Signals](http://forum.numer.ai/t/is-tb200-neutralizing-the-moore-penrose-inversion-neutralization-step-in-signals/4308) All 0.99 submissions: [https://signals.numer.ai/of\_s\_3](https://signals.numer.ai/of_s_3) All 0.01 submissions: [https://signals.numer.ai/of\_s\_4](https://signals.numer.ai/of_s_4)