Model shuffling

Tournament
#1

are there any obvious advantages or disadvantages to model shuffling? what i mean is some variant of :

  • suppose i have 10 good models
  • instead of posting weekly predictions from model A to numerai model account a, B : b, etc…, models {A,B,C,D,E,F,G,H,I,J} predictions are shuffled before being posted to numerai, so they may end up A : c, B : e, etc.

i keep wondering if this might accomplish masking some signal, and if masking signal would enable the user to achieve more sustained MMC. these are my intuition but this is super-not-my-field.

#2

I do that all of the time. I try to keep them constant, but sometimes I have to rotate them. One reason to rotate them is if I need a boost to my earnings and the best model does not correspond to the higher staked models. Its much easier to do that then to pull a stake and restake it on another model. Zero tax consequences too. Have not seen anything bad other than ranks getting shifted a bit. It will not have any effect on MMC, it will only cause interested parties to wonder what is going on there. It has no negative effects to the Hedge Fund either since they look at the test set scores and stake to determine the model weight for the MM at the outset of every round.

I think it actually helps the Hedge Fund if you rotate models the way I have outlined.

2 Likes
#3

that makes sense, thanks!

i think my thoughts about MMC stem from assuming that the Hedge Fund can use predictions submitted to them to improve their own models. and that somehow, faithful model submissions would improve that for them, compared to shuffled model submissions.

#4

Shuffling makes no difference to the fund – there is no past at all as far as the fund is concerned. Each week is brand new and they create a whole new metamodel. The idea that if users submitted the same models persistently (and in the same slots) it is better for the fund somehow is a common misconception about the tournament.

From the user side, the only major reason it would matter is to stake differently on a model week-to-week, i.e. moving a model that has a current stake on it of 10 NMR to another slot that also has a current stake of 10 NMR is mathematically pointless staking-wise, but if you have different stakes on different slots than you might move them around for quick staking changes without having to actually add or remove overall stakes.

2 Likes