Daily Tournament - Update #1

Daily tournament adoption
It has only been a week since the launch of the first daily round and we have already seen some great adoption.

In terms of staked models, 34% of Numerai Tournament and 20% of Numerai Signals are now submitting daily. In terms of total NMR staked, 44% for Numerai Tournament and 32% for Numerai Signals.

To everyone who has started submitting daily - great job and thank you! To everyone else - please let us know if there is anything we can do to assist you.

Priorities - Stabilization then Payouts
Our top priority now is to stabilize the daily tournament system. Many you have experienced issues with our data and submission APIs over the past week. Thank you for your bug reports and patience with us as we get these issues resolved.

Once the system has stabilized, we will look to roll out payouts for the new daily rounds. We have gotten both the supportive and critical feedback on the proposed payout system and we are working to release an updated rollout plan with details soon.

Weekend round scheduling changes on Nov 12
Starting on Nov 12, Saturday rounds will open 5 hours earlier on Saturday 13:00 UTC instead of 18:00 UTC, and the deadline will shift 30 minutes earlier on Monday 14:00 UTC instead of 14:30 UTC. For those of you on Compute, your models will be triggered as soon as the data is ready at this earlier open time.

And as a reminder, the open and close times of rounds are subject to delays based on our data vendors and may also change over time as we improve the speed of our own pipelines. The best way to make sure your model pipelines handle these changes is to integrate with Compute and just rely on our systems to inform you of when the data is ready.

9 Likes

my blocker to switch all my models (and staking any) to daily submission is int8 data support for compute-light.

2 Likes

In order to join the daily format quickly, I would need a submission window of 90-120 min …

2 Likes

The main reason I do not submit daily rounds is disappointment. Instead of making daily tournaments appealing due to a higher potential earning, Numerai decided to take the other direction and make the weekly tournament less remunerative (70% of the current payout) so that users must do more to get almost the same. I do not like this approach at all.

5 Likes

What’s the problem with int8? Can’t you download any dataset you want from compute?

Any update on accepting delayed submissions automatically for D+1? I have a few models which take way longer and I think this would really help/motivate me to participate in daily rounds.

1 Like

I am seeing an error for my daily submissions on numerapi 2.9.4

2022-11-09 13:38:07,147 ERROR numerapi.base_api: You must provide predictions for the current live IDs. Make sure you are using the latest live data.

Even though i am downloading round 351 tournament data.

Downloading numerai_tournament_data_351.parquet ⠙

Is anyone facing this issue as well?

You need to download numerai_live_data.parquet instead. numerai_tournament_data only gets updated weekly.

1 Like

i am pretty confused with all impacts after daily tournament started: no more daily/monthly/annual gain. weekly submission result was disconnected. no daily submission return estimation, etc. not sure how soon every will be back to normal.

3 Likes

Pretty sure all the functionality will return, it’s just going to take a little bit. The jump to daily put a massive load on the data-base. I think once they get things optimized we’ll see that data reappear.

We turned off returns because we thought daily rounds might be polluting the numbers (they were not) and the current returns numbers are not easily verifiable from the outside. We are doing some more validation before re-enabling (soon).

The current Numerai and Signals returns calculations live in separate pipelines with their own bespoke implementations. The code is complex and hard to debug because it essentially boils the ocean from the beginning of time re-simulating every tournament scoring change, recalculating every payout, multiplier change, etc to derive returns.

Returns will return soon, and I will publish an accompanying worksheet explaining how users can audit/verify 1d, 3mo & 12mo returns using Excel themselves.

Thanks for your patience.

8 Likes

@wigglemuse You can in compute-heavy. You cannot in compute-light. That is what I desire. I have been complaining about it in all the channels (RC, the Trello, now here) without any success so far.

Hmmm…makes no sense to me, but I guess I don’t understand it. You’ve got an environment right, can’t it just do any arbitrary internet (and therefore api) call you want? If I can download int8 outside of compute why can’t you do it inside? What is actually stopping you?

no, in compute-light you do not have an evironmanet, but you deploy a pickled model file, the version of data (v2,3,4) and the list of features the model expects to the numerai endpoint. The rest is handled by numerai.

The models predict method is called during live with a pd.DataFrame of the data. That’s it.
That being said, I could of course start my predict method by ignoring the passed data, creating an API link and downloading the data. Apart from technical issues (can the script write to where it is running??) that completely defeats the compute-light goal of maximum integration so what I am asking for is to have another deployment parameter for int8 or float data.

1 Like

Yeah, ok, pretty limited. I’m not saying they shouldn’t have int8 (and I’d need a whole lot more than that to make compute workable), but seems like you could get around it in the meantime as well. Can you just multiply the data * 4 to convert to int or does that cause memory issues?

any idea when the staking for daily tournament will be started ?

2 Likes

thank you for that tip. That is a reasonable workaround. I will try to do it that way!

When does the Friday round opens? I am getting ‘ValueError: Current round not open for submissions’

see it just opened, must be more patient. missing indication in gui

up ! I think this would be good to give an end this in-between period…
Please start daily payouts !