I may well be missing something elementary, but my recollection is that I used to use the “example_predictions” provided by Numerai as a gauge to assist in measuring my model’s relative performance.
Recently I tried to merge the example_predicitons.csv and example_validation_predictions.csv into my train/valid df files as I used to do, however none of the 'id’s for the example predictions exist within the new data set. Is this working as intended? Or have I made some mistake coming back to this after so many months?
My hope is to have a set of predictions, supplied by the Numerai team, which is a reasonable approximation of the meta-model. This would allow me to compare my models to a known entity and hopefully leverage that information while I tune my performance.
Is there a way to go about creating this information? I can train my own model as a base line, but I would like to use something more official.