Feature request - Easy comparison of model performance

I often struggle to compare performance of my models. Comparing CORR and MMC model by model is very time consuming.

I found the rainbow chart on the models page useful, but many of my models don’t have 20 weeks of history and I would like to compare them on a CORR + 2xMMC basis as well.

I would suggest to improve the rainbow chart with the following features.

  • add a “from round” input field. So that selected models are displayed and compared for the selected rounds only. (Less than 20 round) This allows easy comparison of new models on the cumulative chart, which have a few rounds of history only.

  • allow CORR+MMC and CORR+2xMMC options as well. These are the metrics we are paid for, thus these give the most comprehensive information.

Thank you

Peter

3 Likes

It would also be nice to compare our models with the metamodel’s CORR, so we can use it as a benchmark.

1 Like

@degerhan 's script is very helpful for this purpose.
It would be great if we could get this on the homepage:

1 Like

Is this what you might be looking for? https://dashboard.numeraipayouts.com/

2 Likes

This is great Aventurine, are you the developer behind this? Only thing missing for me is CORR + 2x MMC

I’m aware of this dashboard. It’s pretty good, but it doesn’t have cumulative returns. It’s still hard to compare return over let say 10 round. One model is better is some rounds, another model in other rounds. The cumulative return counts most…

Not me. I believe @ia_ai did it?

Did you see my post a few weeks back? I added a notebook that pretty much does all this:

If your point is though that this kind of functionality should be implemented into the website as its easier to access, I agree with that, as you’ll only come across my notebook if you know what you’re looking for.

3 Likes