I’m using the actual formula for MMC calculation. Basically, in the MMC formula, you do the following steps:
- Transform all predictions to standard uniform
- Neutralize against the meta-model
- Transform neutralized predictions to standard uniform
- Compute MMC as the covariance between neutralized predictions and the target, divided by 0.29^2 (variance of standard uniform) to get to correlation space
This is the formula from MMC2 Announcement
The MMC is not influenced in any way by how the meta model performs in combination with specific user predictions. It is influenced by the overall meta model performance and the user’s exposure (correlation) to the meta model, as seen in the above steps.
Now, using the above formula when the meta model raw corr is -1% with two user models:
Model A: raw corr -1%, meta model corr 0.99 (so a meta model clone)
Model B: raw corr -1%, meta model corr 0 (same performance, but completely uncorrelated)
You would get roughly MMC model A score of 0, MMC model B score of -1%.
So originality gets ‘punished’, as you get much lower MMC with the uncorrelated model.
Again, this only happens when the meta model has negative correlation with the target.
You could think about it like this: model A has 0.99 exposure to the meta model, which this round is a negative feature/predictor. By removing the meta model from model A, the MMC for model A is increased just because we remove a negative corr exposure. But model B is not correlated to the meta model, so neutralization doesn’t change anything. It doesn’t “benefit” from the negative performance of the meta model.
In good periods, model B really benefits from MMC. Using the above MMC formula when the meta model raw corr is 1% with two user models:
Model A: raw corr 1%, meta model corr 0.99 (so a meta model clone)
Model B: raw corr 1%, meta model corr 0 (same performance, but completely uncorrelated)
You would roughly get scores of around 0% MMC for model A, but around 1% MMC for model B.
So total CORR+MMC scores are 1% for model A (basically no originality = 0 MMC) and 2% for model B (originality makes MMC=CORR leading to twice the payout of just CORR).