Differentiable Spearman Correlation

[gaugesym]

I want to try out to directly train on the ‘corr’ on which we are scored on. For this I would have to implement the Spearman correlation and get the gradient and hessian to be able to use it as a custom loss function in xgboost (or keras etc.). So far I was able to implement it in tensorflow and also get the gradient but the hessian does not work yet. I created a question with the details on stackoverflow:

Hessians for Spearman Rank Correlation

Maybe someone of you has already tried to do something similar and knows why my code isn’t quite right yet.

[oiboy]

A couple people have been trying this. Check out

Differentiable Spearman in PyTorch (Optimize for CORR directly)

@mdo previously showed how to use a custom loss function which involved taking the gradient of the sharpe ratio of the Pearson correlations over different eras. Although Pearson and Spearman might return similar values, it could be rewarding to optimize for Spearman directly (or Sharpe of Spearman). Since the ranked Spearman correlation needs a sort operation (which is not differentiable), it has not been possible to compute the gradient with respect to predictions, which eliminated the possibil…

and

Objetive Function

Just writing this to share which target functions you use the most when training your models. I was thinking of customizing an Objective Function for boosted models in order to beat the common methods already developed. I know Spearman’s correlation is non-differentiable due to sort and rank steps, but I found some references to try to deal with these problems: SoDeep fast-soft-sort I’ve tried to use SoDeep loss functions when training my MLPs and it was a complete disaster. So it would be ni…

I can’t find where I read it, but I believe the fast-soft-sort package’s Spearman function is only differentiable once, which would cause your None issue. You could try @mdo’s solution, which is to just use a matrix of ones as the Hessian.

[gaugesym]

Thanks for the reply. I got it working by setting the hessians to 1 instead of actually calculating them. I’m able to train with it in xgboost but even after quite some parameter tuning I’m so far not able to get better results than from just using rmse or similar standard loss functions.