COR vs MMC staking

Facing the dilemma of “COR vs MMC staking”, I looked closely at round performance of my (and other users’) models. I simply plotted COR vs MMC scores of my models and also included performance of current top500 users.

Each scatter-plot shows the results of all models in one round (rounds 212-209 haven’t resolved yet). Each dot represents one user model with X coordinate is correlation score and Y is MMC score (blue triangles are my own models). At each scatter-plot, I added the indifference line: ABOVE the line model is better of staking at MMC score, bellow that line user better to stake on correlation.






At “bad” rounds (like 195-198), MMC is a good way to improve returns for most of the models.
At “good” rounds like 203-209 COR staking bits MMC staking for most of the users’ models. So in general MMC staking strategy provides some kind of protection against performance volatility.

9 Likes

update for new dayly scores for unresolved rounds (212-209) with COR distribution of unresolved rounds


current round 212 today looks exactly like round 211with average COR close to zero

2 Likes

I’ve done my own simple theoretical research of COR vs MMC staking using validation data. As the rough approximation of the MMC payout I’ve used the equation: 2 * (my model’s correlation - example predictions correlation * 0.5) per era in validation set. 0.5 was chosen because my predictions have 0.5 average correlation with metamodel for the last 4 rounds. The results are 0.0268 average payout (sharpe 0.9) using MMC staking and 0.0278 average payout (sharpe 1.7) using COR staking. Current live results (4 unresolved rounds) are: 0.0243 average calculated payout (sharpe 1.1) using MMC staking and 0.0268 average payout (sharpe 1.4) using COR staking.

Probably all that just mean that my model is not really useful for the metamodel

1 Like

I’ve been checking models returns COR vs MMC, and I think it makes no sense moving from COR to MMC. Checking returns of big stakers that have been moved to MMC…

I haven’t explored other models but I have a couple of models with low MMC correlation but MMC score is heavily dependant on COR performance. When I have COR < 0, even being on top percentiles (less negative than the rest) I get a bad MMC because my model is contributing on a negative original way. I understand numerai wanting fresh approaches, that’s why I see MMC as a leaderboard/originality bonus replacement, “you do well, you get paid. you do well and your model is doing something original, you get paid a bit more”.

5 Likes

I agree with this idea - rewards originality, and avoids punishing that originality when/if the MMC correlation changes. If I understand correctly, the actions of other participants will affect the payout/burn (unlike with COR - which is determined by the market).

What does switching the method look like? If I wanted to try MMC for a week and then go back to CORR the following week, could I? Is the 4 week time frame relevant in any way for switching?

We should be able to switch for each round – wouldn’t make sense otherwise. So if you are set on MMC at the start of a round (meaning Thursday when any other stake changes would take place), then the payment for that round 4 weeks later will be on MMC. (Remember you only get paid for a round at the end of it now.) But presumably, you could change it back to CORR the next week, but it would apply to the next round. So you can’t change CORR->MMC or MMC->CORR on a round that’s already started, but you could alternate round to round if you wanted.

Gotta admit, not crazy about MMC either. To me it seems like a way to reduce risk of volatile or bad time periods at the cost of introducing risk involving what other people do, and what random subset of them is chosen.