Model ranked low....predictions CSV comparison?

Tournament
1

I submitted a model and was ranked very low; I see a lot of other people that have tied at that rank so I’m wondering if there’s a fundamental problem with the submitted predictions CSV.

Can someone in a higher rank share one of their older predictions CSV files? Or is there an example I can look at on the Numerai site?

2

you can download the example predictions and compare. the corresponding model is numer.ai/integration_test

1 Like
3

Can you clarify what you are talking about? Your leaderboard rank is cumulative over 20 rounds, so when someone submits for the first time they will always have a horrible rank because it takes time to build it up. (Even if you are doing awesome on the round you submitted.)

4

I was wondering if my predictions CSV had errors in it that would cause low rankings but what you said makes sense, it’ll build up rank over time.

Though I did discover issues when comparing my predictions CSV to the example predictions; my submission has far less precision (0.25 instead of 0.2499 for example).

5

Yeah, you should avoid ties in your predictions as much as possible, i.e. try to have all unique values. (Even though there are only 5 target values because of the way we are scored it is best not to have ties.) So more precision is good – go for that.

2 Likes
6

Awesome, looks like I had an issue within my machine learning code and was copying over the wrong column.

Looks like this now :smiley:

id,prediction_kazutsugi
n0003aa52cab36c2,0.46591565012931824
n000920ed083903f,0.4782555401325226
n0038e640522c4a6,0.5369644165039062
n004ac94a87dc54b,0.48990345001220703

Just submitted to make sure it validates and it looks good.

Thanks for the help!

1 Like