CWMM lower than numerai computation

Tournament
1

I’m using this function to compute CWMM:

cwmm <- function(mm, preds, era) {
  pred_dt <- data.table('era' = era, 'pred' = preds, 'mm' = mm)
  pred_dt[, preds_ranked_gauss := qnorm((rank(pred, na.last = 'keep') - 0.5) / .N), by = .(era)]
  pred_dt[, preds_ranked_gauss_pot := sign(preds_ranked_gauss) * abs(preds_ranked_gauss)^1.5]
  pred_dt[, mm_ranked_gauss := qnorm((rank(mm, na.last = 'keep') - 0.5) / .N), by = .(era)]
  pred_dt[, mm_ranked_gauss_pot := sign(mm_ranked_gauss) * abs(mm_ranked_gauss)^1.5]
  corr_dt <- pred_dt[, .(CWMM = cor(mm_ranked_gauss, preds_ranked_gauss, method = 'pearson'),
                         CWMM_pot = cor(mm_ranked_gauss_pot, preds_ranked_gauss_pot, method = 'pearson')), by = .(era)]
  return(corr_dt)
}
> cwmm(mm, preds, era)
     era      CWMM  CWMM_pot
   <int>     <num>     <num>
1:  1100 0.8589433 0.8400368
2:  1101 0.8624765 0.8466103
3:  1102 0.8651279 0.8510147
4:  1103 0.8685777 0.8562474
5:  1104 0.8814365 0.8703542

The CWMM of the model in numerai CWMM column is always greater than 0.93

Do you know where is the problem? Can you reproduce CWMM?
In numerai-tools in github there isn’t the script for computing CWMM.

2

I’ve tried with:
ranked pred vs ranked mm
ranked gauss pred vs ranked gauss mm
ranked gauss pot 1.5 pred vs ranked gauss pot 1.5 mm
and all combinations give me CWMM < 0.90 but numerai says in 0.92-0.93
Did someone get reproduce the calculation?

3

This is the code we use for it:

    predictions = tie_kept_rank__gaussianize__pow_1_5(predictions)
    scores = predictions.apply(lambda sub: pearson_correlation(sub, meta_model))
1 Like