# Pearson vs. Spearman scoring confusion

**URL:** <https://forum.numer.ai/t/pearson-vs-spearman-scoring-confusion/2559>\
**Category:** Numeraire\
**Created:** [March 27, 2021, 6:23pm UTC](https://forum.numer.ai/t/pearson-vs-spearman-scoring-confusion/2559 "2021-03-27T18:23:50Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![oiboy](https://avatars.discourse-cdn.com/v4/letter/o/d07c76/32.png) [@oiboy](https://forum.numer.ai/u/oiboy)\
**Post date:** [March 27, 2021, 6:23pm UTC](https://forum.numer.ai/t/pearson-vs-spearman-scoring-confusion/2559/1 "2021-03-27T18:23:50Z")

</div>

Hey all, newbie here.

I keep seeing posts which state that Numerai scores are based on the Spearman correlation coefficient. The [comment in the example](https://github.com/numerai/example-scripts/blob/59d82639b11b67e51c4b8e7eee08ac38455dfc81/example_model.py#L20) seems to agree with this. However, [the same example](https://github.com/numerai/example-scripts/blob/59d82639b11b67e51c4b8e7eee08ac38455dfc81/example_model.py#L22-L23) and [the documentation](https://docs.numer.ai/tournament/learn#scoring) state that the correlation for scoring is calculated using:

```auto
ranked_preds = predictions.rank(pct=True, method="first")
return np.corrcoef(ranked_preds, targets)[0, 1]

```

which returns the Pearson’s correlation coefficient, according to [numpy’s documentation](https://numpy.org/doc/stable/reference/generated/numpy.corrcoef.html).

Which is actually used by Numerai to evaluate correlation for scoring, Spearman’s or Pearson’s? Is ranking the predictions enough for `np.corrcoeff()` to return the Spearman correlation?

---

<div class="post-metadata">

**Author:** ![quantized](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/quantized/32/2255_2.png) [@quantized](https://forum.numer.ai/u/quantized)\
**Post date:** [March 28, 2021, 11:34am UTC](https://forum.numer.ai/t/pearson-vs-spearman-scoring-confusion/2559/2 "2021-03-28T11:34:25Z")

</div>

The second line in the code above does indeed calculate Pearson. However, the line above, which ranks the predictions, means that Spearman is in fact calculated. You can think of this:  
**Spearman = Ranking + Pearson**

---

<div class="post-metadata">

**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3094_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\
**Post date:** [March 28, 2021, 2:41pm UTC](https://forum.numer.ai/t/pearson-vs-spearman-scoring-confusion/2559/3 "2021-03-28T14:41:11Z")

</div>

Not quite – only the predictions are ranked (and ties broken), not the targets (ties remain). So it isn’t fully Spearman.

---

<div class="post-metadata">

**Author:** ![oiboy](https://avatars.discourse-cdn.com/v4/letter/o/d07c76/32.png) [@oiboy](https://forum.numer.ai/u/oiboy)\
**Post date:** [March 28, 2021, 3:42pm UTC](https://forum.numer.ai/t/pearson-vs-spearman-scoring-confusion/2559/4 "2021-03-28T15:42:58Z")

</div>

So is this code accurate for how Numerai calculates corr for scoring? Or should I rank the targets as well?

scipy also offers [scipy.stats.spearmanr](https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.stats.spearmanr.html), I’m wondering if that’s the easier option.

---

<div class="post-metadata">

**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3094_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\
**Post date:** [March 28, 2021, 3:50pm UTC](https://forum.numer.ai/t/pearson-vs-spearman-scoring-confusion/2559/5 "2021-03-28T15:50:38Z")

</div>

It’s accurate, yes. The difference from actual spearman is slight, though.