I noticed that new stakers start off with a -0.0989 correlation score right off the bat. For the last 4 weeks I have submitted my predictions on time and have even increased my stake but my correlation score still hangs in the low -0.08 range despite all my predictions currently correlating positively. Why is this? Will it take a full 20 rounds to resolve before I start seeing a positive correlation score? Will it take 4 rounds to resolve?
1 Like
Well, whether it will end up positive or not of course depends on your actual performance. But yeah, you do have to fill up the 20 rounds (including the open ones) to be at your “true” score/rank (there are major penalties for missing rounds), i.e. submit 20 weeks in a row. The good news is you are pretty much guaranteed to see your rep/rank go up every week for that whole period – in bad times having your true score/rank can be kind of depressing when it plummets or plateaus at a not very high level. But also remember that your rank is basically meaningless in the sense that you don’t get paid for it and the 20 rounds time-frame is arbitrary.
3 Likes
@wigglemuse Is reputation calculated on a per-model or per-user basis? I’m curious because I noticed a single account can support up to 15 models. If I have several running in parallel, do I need to submit each model every week to avoid hurting my reputation?
Rep score is per model.
1 Like
Great thanks for the clarification. A note for future readers: once I submitted one model, Numerai expected me to submit all of them otherwise they would be marked late.
I had a couple follow-up questions
- Is the model name what gets publicly displayed on the leaderboard? Initially I thought I’d name my models according to the algorithm, but if that is the case that doesn’t seem like a good idea
- Is there any penalty for deleting a model once it exists? Particularly before it has matured to the 20 round lifetime
Yes, the model name is what’s on the leaderboard. On model slot names: currently you can’t rename or delete them, so use names you can live with (and probably don’t assume they are going to hold the same underlying model forever). Any model slot you aren’t submitting on-time every week is going to have missing rounds for reputation/rank purposes, and those will suffer leaderboard-wise. But really, as a new user your rank is the last thing you need to be worried about since it is just there to look at unless that is all you are interested in.
If you are staked, and don’t submit, or submit late, then your stake will not be at risk for that round, so don’t worry about losing stake for missing or late rounds.
Hi wigglemuse, if i remember correctly once you are staked, that stake will remain there for 4 weeks right? Wouldnt that mean that if you suspect that the next round(s) are going to be bad rounds for your model you could avoid burns by just not submitting predictions (and yes your rank would go down a bit)
Emmm… I don’t you think you quite understand how staking works in terms of overlapping and perpetual restaking, but still the answer to your question is yes – if you don’t submit or simply submit late (after the Monday morning deadline) then you won’t be at risk for that round. But you can’t “abort” an in-progress round for which you are already staked just because you see it is going badly…