# New Target Nomi Release **URL:** \ **Category:** Tournament\ **Created:** [September 19, 2020, 1:41am UTC](https://forum.numer.ai/t/new-target-nomi-release/959 "2020-09-19T01:41:12Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![mdo](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/mdo/32/3398_2.png) [@mdo](https://forum.numer.ai/u/mdo)\ **Post date:** [September 19, 2020, 1:41am UTC](https://forum.numer.ai/t/new-target-nomi-release/959/1 "2020-09-19T01:41:12Z")
We are releasing today a new target, target\_nomi, that you can optionally use to train your models. You should still submit predictions in the same exact way as before but just using the outputs of the newly trained model. Fundamentally, this target represents the same thing as target\_kazutsugi, but just a bit more faithfully. Consequently, it is compatible with target\_kazutsugi and we will continue scoring using target\_kazutsugi for the time being. We will give plenty of warning before we eventually switch to scoring with target\_nomi (likely late October). In my experiments, training models using target\_nomi and scoring against target\_kazutsugi, gave about the same scores on average as training on target\_kazutsugi, but with less variance era-to-era, and therefore higher Sharpe. On our side, this target allows us to construct portfolios with higher expected returns. We expect this to be win-win and want to make the transition as smooth as possible. Even after the transition to scoring on target\_nomi, your old models trained on target\_kazutsugi will still work, but will likely score slightly worse than if you had retrained on target\_nomi. So let us know how it goes! ![image](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/1X/f076d72fad8f2d7b35820125eadd28fb47da5c40.png) You can download it here as a parquet file: [https://drive.google.com/file/d/1K7CnXKoEW7sKCwb9D6TboQsVDWgVdiez/view?usp=sharing](https://drive.google.com/file/d/1K7CnXKoEW7sKCwb9D6TboQsVDWgVdiez/view?usp=sharing) Or here as csv: > **[numerai\_training\_validation\_target\_nomi.csv.tar.gz](https://drive.google.com/file/d/190NsspVfMqNega_8nOpqgKkKELrxjtTv/view?usp=sharing)** > > Google Drive file. You can watch the discussion of the new target in OHWA here: https://player.twitch.tv/?video=v743826342&parent=numerai.discourse.group&autoplay=false ---
**Author:** ![perfect\_fit](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/perfect_fit/32/2527_2.png) [@perfect\_fit](https://forum.numer.ai/u/perfect_fit)\ **Post date:** [September 19, 2020, 11:26am UTC](https://forum.numer.ai/t/new-target-nomi-release/959/2 "2020-09-19T11:26:30Z")
Nice, excited to check it out! Are there diagnostics on how the example model (INTEGRATION\_TEST) performance changes with the new target (including feature exposure, max drawdown, etc.)? Or is that something we have to figure out ourselves? šŸ™‚ ---
**Author:** ![jrb](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jrb/32/3472_2.png) [@jrb](https://forum.numer.ai/u/jrb)\ **Post date:** [September 19, 2020, 2:38pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/3 "2020-09-19T14:38:52Z")
Hereā€™s a [modified version](https://github.com/jeethu/numerai-example-scripts/blob/nomi/example_model.py) of the [example model](https://github.com/numerai/example-scripts/blob/master/example_model.py) that trains on the new target. And hereā€™s the [diff view](https://github.com/numerai/example-scripts/compare/master...jeethu:nomi?expand=1), if youā€™d like to see what Iā€™ve changed. ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3094_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [September 19, 2020, 9:49pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/4 "2020-09-19T21:49:58Z")
@mdo So obviously youā€™ve compared models trained with old targets vs models trained with new targets and both scored the same way, and concluded that trained with new targets = better. And for the time being we are still be scored on the old targets. So thatā€™s all good. But (and I missed the office hours and havenā€™t watched that video yet), if we transition to being scored on the new targets, I have a slight concern (based on a very quick test with sample size of one) that being scored on the new targets will lower the magnitude of scores overall and therefore even if they are better we may all end up with a slight haircut going forward (i.e. the whole range of scores will shift slightly caused by the new target distribution). At least people that are positive most of the time will get a haircut, however the effect would be symmetrical and change the magnitude on the negative side in the same way. Have you looked into this question? So for instance if you took 100 models (doesnā€™t matter how they were trained) and scored them on old targets, and score the _same_ (not retrained on different targets) 100 models on the new targets, is the average score going to be pretty much equal, or will the distribution of the new targets tend to cause greater magnitude or lower magnitude? Which would mean everybody possibly getting paid less, but also burning less I guess. In my test, there is a not insignificant difference, but obviously there is going to be SOME difference when Iā€™m only looking at a single model so wondering if youā€™ve done any such tests in bulk. (I should watch that video in case youā€™ve covered this.) Thoughts? ---
**Author:** ![lackofintelligence](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/lackofintelligence/32/774_2.png) [@lackofintelligence](https://forum.numer.ai/u/lackofintelligence)\ **Post date:** [September 19, 2020, 10:08pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/5 "2020-09-19T22:08:50Z")
Same delicious Lucky Charms, (LARGE SIZE), but in a thinner box? ![image](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/1X/9c9a1be33a1df691df9b6e94f2a1105f7a2d33f4.jpeg) ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3094_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [September 19, 2020, 10:23pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/6 "2020-09-19T22:23:08Z")
Well, yes, we can actually figure this out analytically. Since the targets are not continuous and have a bunch of ties in them (that are not broken), but our predictions do have their ties broken, and then are scored based on cor(preds,targets) the actual range of possible scores under the current system is not [-1,+1]. With the current system of 5 roughly equal buckets, the range is about [-0.98,+0.98]. With the new distribution of 5,20,50,20,5 (percents), the range narrows to about [-0.91,+0.91]. So the mathematical capacity to get higher (or lower) scores is reduced because the whole magnitude is reduced. Now of course we arenā€™t getting anywhere near the max/min values of with our predictions, but still this will be a change that may hurt somewhat, i.e. we will make less than what we would have made under current system. HOWEVER, we are also promised that our models will be improving (if we retrain), so then maybe it is a wash? Maybe, maybe not. (And burns would be lessened also.) It would seem to indicate at the very least that we probably will not be making more under new system. Does it matter? (The current system is the current system, but that doesnā€™t make it sacred.) Possibly there should be an adjustment factor to our CORR scores (even under current system) to fix the range at [-1,+1], or maybe just to fix the new range to match the old range. (Or just keep scoring under old targets.) Thoughts? ---
**Author:** ![mdo](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/mdo/32/3398_2.png) [@mdo](https://forum.numer.ai/u/mdo)\ **Post date:** [September 19, 2020, 11:15pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/7 "2020-09-19T23:15:04Z")
The range issue you mention is interesting, thanks for pointing that out. I will investigate and see if we need a scoring/payout adjustment that is due the the distribution of the new target. I do think the new target is slightly harder because there is less regularization built in and it more closely matches the quantity of interest. For example, the increased rareness of the extreme values 0&1 can make it easier to overfit. The flip side is that if you do well with the new target your model should generalize better than before whether scored again Kazutsugi or Nomi. I think if you make your goal to get the same generalization performance when training and scoring against Nomi as you had when training and scoring against Kazutsugi, your model will end up much better overall and you should see no haircut after the transition (barring the scaling issued already discussed). ---
**Author:** ![ia\_ai](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/ia_ai/32/3543_2.png) [@ia\_ai](https://forum.numer.ai/u/ia_ai)\ **Post date:** [September 19, 2020, 11:36pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/8 "2020-09-19T23:36:46Z")
Can we add two more lines (based on `integration_test`) to the comparison, please? - train\_kaz\_predict\_nomi - train\_nomi\_predict\_nomi ---
**Author:** ![permistiro](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@permistiro](https://forum.numer.ai/u/permistiro)\ **Post date:** [September 20, 2020, 1:11am UTC](https://forum.numer.ai/t/new-target-nomi-release/959/9 "2020-09-20T01:11:28Z")
In case anyone needs to fetch the new files from command line/script, I pinned them to Infuraā€™s Ipfs service. Hereā€™s the link to Cloudflareā€™s fast http gateway: [https://cloudflare-ipfs.com/ipfs/QmX1cDy6iqKGy8B4wSx8PkwxDTgPmWuZZBTM7tL1WNizv8/](https://cloudflare-ipfs.com/ipfs/QmX1cDy6iqKGy8B4wSx8PkwxDTgPmWuZZBTM7tL1WNizv8/) ---
**Author:** ![permistiro](https://avatars.discourse-cdn.com/v4/letter/p/b782af/32.png) [@permistiro](https://forum.numer.ai/u/permistiro)\ **Post date:** [September 20, 2020, 1:14am UTC](https://forum.numer.ai/t/new-target-nomi-release/959/10 "2020-09-20T01:14:25Z")
I have a question: will the target be named nomi or kazutsugi after the migration? ---
**Author:** ![wigglemuse](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/wigglemuse/32/3094_2.png) [@wigglemuse](https://forum.numer.ai/u/wigglemuse)\ **Post date:** [September 20, 2020, 3:36am UTC](https://forum.numer.ai/t/new-target-nomi-release/959/11 "2020-09-20T03:36:08Z")
> The range issue you mention is interesting, thanks for pointing that out. I will investigate and see if we need a scoring/payout adjustment that is due the the distribution of the new target. Cool, thanks. Iā€™m sure you will make the right decision! ---
**Author:** ![jeremy\_berros](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jeremy_berros/32/3478_2.png) [@jeremy\_berros](https://forum.numer.ai/u/jeremy_berros)\ **Post date:** [September 20, 2020, 4:44am UTC](https://forum.numer.ai/t/new-target-nomi-release/959/12 "2020-09-20T04:44:14Z")
Hi @jrb. Thanks for your update. However I get the following error message while submitting using strictly your version of example model with nomi: **invalid submission ids. ids must match current tournament data exactly, including ordering. make sure you are using the latest tournament data** ---
**Author:** ![jrb](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jrb/32/3472_2.png) [@jrb](https://forum.numer.ai/u/jrb)\ **Post date:** [September 20, 2020, 7:26am UTC](https://forum.numer.ai/t/new-target-nomi-release/959/13 "2020-09-20T07:26:26Z")
@jeremy_berros Are you sure youā€™ve got the latest tournament data that was released yesterday? I donā€™t intend on ever submitting example predictions (or any xboost model for that matter), but I tried it out of curiosity, after I saw your post and hereā€™s what I got: ![Screenshot](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/1X/49ef32dfae4ee492a80ecf6deb8c52386bdb4ca8.png) ---
**Author:** ![jeremy\_berros](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jeremy_berros/32/3478_2.png) [@jeremy\_berros](https://forum.numer.ai/u/jeremy_berros)\ **Post date:** [September 20, 2020, 7:53pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/14 "2020-09-20T19:53:10Z")
@jrb. Tried again this morning and it works. I guess thatā€™s what happens when you try too hard too late šŸ˜“ ---
**Author:** ![mdo](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/mdo/32/3398_2.png) [@mdo](https://forum.numer.ai/u/mdo)\ **Post date:** [September 20, 2020, 11:42pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/15 "2020-09-20T23:42:09Z")
It will either continue as ā€œtarget\_nomiā€, or possibly just ā€œtargetā€ ---
**Author:** ![loracle](https://avatars.discourse-cdn.com/v4/letter/l/ba8739/32.png) [@loracle](https://forum.numer.ai/u/loracle)\ **Post date:** [December 15, 2020, 3:00pm UTC](https://forum.numer.ai/t/new-target-nomi-release/959/16 "2020-12-15T15:00:04Z")
Hello, do you have old tournament data pinned somewhere on ipfs by any chance ?