Prediction upload failing during crunch time using compute & numerapi

Hi all.

I have a compute node that calculates and submits all of my predictions, one per model, one after the other.

I recently added a few more models and with that addition I started seeing failures during the auto-triggered compute runs on Saturdays.

Specifically, the failures occur with the upload_predictions() call from numerapi. The return message only contains the mysterious single word “data” (it normally contains the submission ID).

When I trigger the compute node myself a few hours later, it runs fine.

I suspect that heavy submission traffic might be causing this failure, and that the upload is timing out or something like that. Has anyone else experienced this? Any suggestions for how to avoid it?

Thanks in advance if so.

prc

1 Like

I think there were some API issues on Saturday. A few of my predictions also failed to submit when I tried shortly after the round opened. There was also some discussion on RocketChat about this on the day, so I think quite a few people had the same issues.

I also had this issue several times over the last weeks.