hello, i made my first submission but i am not in the leaderboard, which submissions are eligible for the leadearboard?
1 Like
If you just submitted your first “daily score” will be out on Thursday, but a single round is 4 weeks long. The daily scores from now until your first round resolves don’t really matter, only the final score at the end of the 4 weeks.
The leaderboard is the weighted average over the last 20 rounds. So you are going to need to wait at least 24 weeks before you will have a full idea of your position on the leaderboard. Empty rounds have a -0.1 correlation score for every round you don’t submit.
This is a long game and the leaderboard isn’t what you should be focusing on. With the rounds being 4 weeks long, you are going to have to be patient to see the results of your modelling work.
3 Likes
Thank you very much for your reply!really appreciated …my next concern is how to optimize the numbers appeared in ‘‘performance’’, ‘‘risk’’, ‘‘mmc’’(3 each)?
all i found are these formulas :
def _score(sub_df: pd.DataFrame) → np.float32:
“”“Calculates Spearman correlation”""
return spearmanr(sub_df[“target”], sub_df[“prediction”])[0]
# Calculate metrics
corrs = df.groupby("era").apply(_score)
payout_raw = (corrs / 0.2).clip(-1, 1)
spearman = round(corrs.mean(), 4)
payout = round(payout_raw.mean(), 4)
numerai_sharpe = round(sharpe_ratio(corrs), 4)
mae = mean_absolute_error(df["target"], df["prediction"]).round(4)
Shouldn't be a formula for each of 9 metrics (performance,risk,mmc)?
1 Like
Take a read of this, should answer your questions on the definitions of the metrics
1 Like