[Beta Release] : Classics tournament reporting

Hi all,

As part of learning new tools / technologies and also to gain more insight into the classic tournament I have recently been busy with setting up a reporting solution.

There is of course already the numerai website which contains great information for your models and also the leaderboard for example and is basically the default to go to for live insights (next to the other dashboards mentioned on the site). In addition of course the excellent app from @ceunen for live payouts.

I was still missing other information though which is more useful for long-term analysis and could not find a complete report out on the web. Also I was anxious to learn more about certain tools like Power BI and decided that it was time create a new report with all the information that is available on https://api-tournament.numer.ai/. Also I learned a lot from the community, twitch, forum, rocket chat and decided it was time to give something back to the community :slight_smile:

Today the beta version (0.2.0) is released of the Numerai statistics report and can be found over here:

Numerai Statistics Retrieval and Report

Like I said its still beta and 80% finished I guess but someone (@arbitrage) told me I should release it soon and get some feedback from the community. I hope you will check it out and do please tell me what you think!

The report is based on PowerBI Desktop which is a free to use tool that can be installed by everyone with a Windows / Mac. All the instructions can be find on the repo readme on how to install and start using the report.

Alternatively you can also get just the pdf exports which are also on the repo, but then you will miss a lot of functionality that its in the report!. The report has the following:

  • General dashboard with quick round status: stakes, payouts, correlation
  • Top 10 page: medals scores, most earned
  • Leaderboard statistics
  • Option to select your models for detailed analysis
  • Option to use slicers on many pages for: dates, rounds, models, stakes
  • Rankings of your models
  • Round performances
  • Payouts for your models and for the whole tournament
  • Price data extraction from Coinbase

The data that is used in the reports are parquet files which are updated on a weekly basis (sunday) and can be used for refreshing the report. The parquet files are also in the repo. In the following weeks I will also put the C# project in the repo which is responsible for retrieving the statistics, but this needs to be a little bit cleaned up first :slight_smile:

Over here are some quick screenshots, please do use the tool and give me feedback.





11 Likes

I also put the reporting dashboard on PowerBI Service, today hone5com made it part of the Numerbay website! :slight_smile: thanks! https://numerbay.ai/

2 Likes

Amazing work! Sorry I jumped the gun on publishing it. Glad you liked it.

1 Like

Hi All, a quick update from me on the tournament reporting/dashboard. I now also added the visual studio solution and Python support files to the git repository. These are examples on how to retrieve the statistics from the graphql endpoint of numerai (and convert to parquet, push to git).

I didn’t do any update on the dashboard itself at the moment, but will start working on that again. I noticed that currently on Numerbay default page 3 is shown of the dashboard upon opening the website, which should actually default to page 1 (easy fix). In addition I will have a look at the following:

  • Add dates to visuals where roundnumber is used on X-axis
  • Visual to compare selected/all models against example_model (corr/mmc/fnc diff, performance diff)
  • Visual with ‘custom metamodels’ performance: metamodel existing of top users (e.g. top 100 users, most staked users, etc…)
  • Option to filter ranking increase statistics based on 20 rounds submissions
  • Visual cleanup of PowerBI report, simplify pages
  • Investigate option cloud-based instance (daily retrieval)

:slight_smile: This is going to take some time ofcourse, since its a personal project at the moment and I also want to spend (lots of) time on the upcoming data release!

4 Likes

Hi all, another quick update on the (classic) tournament reporting/dashboard. I didn’t do much lately, and was more occupied with work/study and ofcourse the new dataset. A minor beta release has been put online and is already available and part of the Numerbay website :smiley:

Besides minor fixes/improvements here and there, there is also a new page available: ‘Performance Scenarios’. The purpose of this page is quite easy, it gives you insights in your models performances when chosing for correlation only or with the various mmc multipliers. Ofcourse when TC is going to be introduced I will have to update this one again, but we will have to wait on that one.

Have fun!

3 Likes