@wigglemuse Leaderboard bonus was (still is until it goes away in September) free money (perhaps, easy money would be more articulate), regardless of how you spin it. And I say this as someone who has benefited from that scheme, albeit not by submitting tweaked example predictions.
Sure, example predictions is just another model. But, how many of your models (or mine for that matter) have the consistent public track record that integration_test has? Especially in the light of what you said on rocket chat yesterday (that you shuffle models around every week). With that prior, staking on example predictions is a lower risk strategy than staking on a new novel model that one would’ve built. Leaderboard bonus only exacerbated that problem. And to state the obvious MMC is a genuine attempt at ameliorating that problem.
Also, the proof is in the pudding. I’ve seen a few high stakes accounts with novel models switch to submitting example predictions (or tweaked versions of it) after suffering a couple of rounds of burns. I’m not going to name them here, it’s quite easy to spot them from the daily scores. Is this switching behavior not indicative of what people percieve to be the lower risk strategy?