# (Way too early) Comparison of legacy & new models **URL:** \ **Category:** Tournament\ **Created:** [December 9, 2021, 12:53am UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595 "2021-12-09T00:53:29Z")\ **Posts on this page:** 8\ **Page:** 2
**Author:** ![yxbot](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/yxbot/32/2906_2.png) [@yxbot](https://forum.numer.ai/u/yxbot)\ **Post date:** [December 9, 2021, 6:36pm UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/21 "2021-12-09T18:36:15Z")
from RocketChat a while ago… ![image](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/b/bbcef860a00476100ed4756016f8774eadc930e3.png) ---
**Author:** ![platemort](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/platemort/32/1143_2.png) [@platemort](https://forum.numer.ai/u/platemort)\ **Post date:** [December 9, 2021, 7:32pm UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/22 "2021-12-09T19:32:29Z")
I remember that post. It doesn’t help though if I understood your suggestion correctly since they won’t tell us which ones are the 304 to be isolated. ---
**Author:** ![themicon](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/themicon/32/2487_2.png) [@themicon](https://forum.numer.ai/u/themicon)\ **Post date:** [December 9, 2021, 7:52pm UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/23 "2021-12-09T19:52:51Z")
The JSON file tells you which 304 features are the old “legacy” features. You should read this again: [October 2021 Updates](http://forum.numer.ai/t/october-2021-updates/4384) ---
**Author:** ![aventurine](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/aventurine/32/2825_2.png) [@aventurine](https://forum.numer.ai/u/aventurine)\ **Post date:** [December 10, 2021, 6:41am UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/24 "2021-12-10T06:41:14Z")
![image](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/8/825620193b2482d600c6ed7373cd2d285b9b9cae.png) ![image](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/1/1a31cb5ced6851319941f24ad9c77b469687fcd5.png) Looks like two of my models on new data seem OK but can be better. The best ones so far for me are low to almost 0 proportion feature neutralized models. MMC not so hot. ---
**Author:** ![autratec](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@autratec](https://forum.numer.ai/u/autratec)\ **Post date:** [December 10, 2021, 1:51pm UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/25 "2021-12-10T13:51:46Z")
I have tried to loaded two versions of examples weekly, legacy vs mass , legacy performance is better. ---
**Author:** ![platemort](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/platemort/32/1143_2.png) [@platemort](https://forum.numer.ai/u/platemort)\ **Post date:** [December 12, 2021, 7:42pm UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/26 "2021-12-12T19:42:32Z")
Thanks, I missed that communication. ---
**Author:** ![kenfus](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/kenfus/32/3397_2.png) [@kenfus](https://forum.numer.ai/u/kenfus)\ **Post date:** [December 13, 2021, 9:21pm UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/27 "2021-12-13T21:21:14Z")
It seems that some new targets are indeed better for live data. I will still wait 3-4 more weeks, then change to either a single model or an ensemble model on the new data. ---
**Author:** ![sneaky](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@sneaky](https://forum.numer.ai/u/sneaky)\ **Post date:** [December 20, 2021, 3:46pm UTC](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595/28 "2021-12-20T15:46:00Z")
How hard is the new data validation set? If I apply my old pipeline from the old dataset I get model with these stats on the new validation dataset: corr 0.01 sharpe 0.4; however, the same model trained on the old dataset manages to do 2.5x better on both stats on the old validation dataset. Is the new validation dataset significantly harder or is my pipeline not easily transferable? Do you have same experience? [Previous page](https://forum.numer.ai/t/way-too-early-comparison-of-legacy-new-models/4595.md?page=1)