# Objetive Function

**URL:** <https://forum.numer.ai/t/objetive-function/2235>\
**Category:** Data Science\
**Created:** [March 8, 2021, 12:36pm UTC](https://forum.numer.ai/t/objetive-function/2235 "2021-03-08T12:36:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![javiermoral](https://avatars.discourse-cdn.com/v4/letter/j/bbe5ce/32.png) [@javiermoral](https://forum.numer.ai/u/javiermoral)\
**Post date:** [March 8, 2021, 12:36pm UTC](https://forum.numer.ai/t/objetive-function/2235/1 "2021-03-08T12:36:01Z")

</div>

Just writing this to share which target functions you use the most when training your models. I was thinking of customizing an Objective Function for boosted models in order to beat the common methods already developed. I know Spearman’s correlation is non-differentiable due to sort and rank steps, but I found some references to try to deal with these problems:

- [SoDeep](https://github.com/technicolor-research/sodeep)
- [fast-soft-sort](https://github.com/google-research/fast-soft-sort/tree/master/fast_soft_sort)

I’ve tried to use SoDeep loss functions when training my MLPs and it was a complete disaster. So it would be nice to hear some tips from you all. Do you keep going with RMSE, MSE, MAE. MAPE, LOGLOSS…?

---

<div class="post-metadata">

**Author:** ![silentj](https://avatars.discourse-cdn.com/v4/letter/s/dbc845/32.png) [@silentj](https://forum.numer.ai/u/silentj)\
**Post date:** [March 8, 2021, 6:14pm UTC](https://forum.numer.ai/t/objetive-function/2235/2 "2021-03-08T18:14:42Z")

</div>

I’ve tried using KL Divergence for learning to rank (see here: [https://theiconic.tech/learning-to-rank-is-good-for-your-ml-career-part-2-lets-implement-listnet-11af69d1704](https://theiconic.tech/learning-to-rank-is-good-for-your-ml-career-part-2-lets-implement-listnet-11af69d1704)). Ended up getting slightly worse results than just regular MSE so I didn’t explore it too much. I might come back to it eventually, seemed cool at the time

---

<div class="post-metadata">

**Author:** ![greenprophet](https://avatars.discourse-cdn.com/v4/letter/g/da6949/32.png) [@greenprophet](https://forum.numer.ai/u/greenprophet)\
**Post date:** [March 9, 2021, 2:13am UTC](https://forum.numer.ai/t/objetive-function/2235/3 "2021-03-09T02:13:58Z")

</div>

I just used pearsonr since it seemed close enough without the sort. With NN and pytorch era batches i got better validation with pearsonr + mseloss than just mseloss. Only have a couple rounds started on live though.

Would like to eventually get some ranking and feature neutralization directly in the loss.

---

<div class="post-metadata">

**Author:** ![javiermoral](https://avatars.discourse-cdn.com/v4/letter/j/bbe5ce/32.png) [@javiermoral](https://forum.numer.ai/u/javiermoral)\
**Post date:** [March 10, 2021, 10:41am UTC](https://forum.numer.ai/t/objetive-function/2235/4 "2021-03-10T10:41:58Z")

</div>

Did you code yourself the person correlation loss function? Or is it implemented elsewhere?

---

<div class="post-metadata">

**Author:** ![greenprophet](https://avatars.discourse-cdn.com/v4/letter/g/da6949/32.png) [@greenprophet](https://forum.numer.ai/u/greenprophet)\
**Post date:** [March 10, 2021, 6:31pm UTC](https://forum.numer.ai/t/objetive-function/2235/5 "2021-03-10T18:31:39Z")

</div>

I used this code

> <https://gist.github.com/ncullen93/58e71c4303b89e420bd8e0b0aa54bf48>

define the function variables

```
    criterion = nn.MSELoss()
    corr_loss_fn = pearsonr

```

then in pytorch loop with loss functions I called like this. but depending on your modelling results might be different indexing. Also not sure if constants on losses are relevant. I get confused about this.

```
preds = model(x)                                
                                    loss = criterion(preds[0], y)
                                    corr_loss = 1 - corr_loss_fn(preds[0].squeeze(), y.squeeze())
                                    if USE_CORR_LOSS:
                                        loss += corr_loss * 0.05
                                    if phase=='train':
                                        loss.backward()
                                        optimizer.step()
```

---

<div class="post-metadata">

**Author:** ![lucky\_chicken](https://avatars.discourse-cdn.com/v4/letter/l/c67d28/32.png) [@lucky\_chicken](https://forum.numer.ai/u/lucky_chicken)\
**Post date:** [March 14, 2021, 12:25am UTC](https://forum.numer.ai/t/objetive-function/2235/6 "2021-03-14T00:25:41Z")

</div>

I’m using fast-soft-sort for my neural nets, is better than MSE for me, but still worse than simple xgboost. I must be doing something wrong.

---

<div class="post-metadata">

**Author:** ![javiermoral](https://avatars.discourse-cdn.com/v4/letter/j/bbe5ce/32.png) [@javiermoral](https://forum.numer.ai/u/javiermoral)\
**Post date:** [March 28, 2021, 3:44pm UTC](https://forum.numer.ai/t/objetive-function/2235/7 "2021-03-28T15:44:28Z")

</div>

Same for me, it does not work as good as expected.

---

<div class="post-metadata">

**Author:** ![javiermoral](https://avatars.discourse-cdn.com/v4/letter/j/bbe5ce/32.png) [@javiermoral](https://forum.numer.ai/u/javiermoral)\
**Post date:** [April 13, 2021, 5:41pm UTC](https://forum.numer.ai/t/objetive-function/2235/8 "2021-04-13T17:41:35Z")

</div>

I can’t see how the gradient and the hessian are computed in your code