Question about open-sourcing models

You can’t base any contest on being good on a known validation set

You can, if the code is submitted. The team can evaluate if the model is legit or if the creator cheated by training on the validation set.

Selling a model is an option to make money as an individual. But the benefits for the meta model would be pretty small. Having a tournament in which a couple of models get released will not only benefit the meta model but also each Numer.ai user, cause they get new ideas and approaches for their own code.

Right now we have to hope that people or generous enough to publish their secret sauces. Michael did that a couple of days ago in Feature reversing input noise. But a tournament with some NMR involved would kickstart that process and offer new user more baseline models (besides integration_test) to build on.