Where do you train your models (if not Google Colab)

[nathanganser]

As I’m starting to work with the larger datasets, my Google Colab functions are taking hours to run and thus disconnecting me in between when I leave my laptop.

Is there another tool out there that can run a jupyter notebook in the background? What are you using?

Thanks!

[svendaj]

I am using and recommend Kaggle. Not only that you can run for free jupyter notebooks with 4 CPUs and 30GB RAM (max 12 hours run each, 5 notebooks in parallel), but thanks to Kaggle API you can fully automatize your pipeline. On top, there is great community and possibility to learn more from competitions.

I have created a few public notebooks and datasets to simplify first steps for Numerai Tournament participants:

• weekly updated dataset with latest data, so that you do not need to download them each time:
  • V4.3 Midnight - latest data, notebook producing the dataset: numerai data v4.3 Midnight (kaggle.com)
  • V4.2 Rain, notebook producing the dataset: numerai data v4.2 Rain (kaggle.com)
  • Older V4 and V4.1 and notebook producing the dataset: numerai data (kaggle.com)
• Example models - typically forks of Numerai example models with improvements for better results:
  • Hello Numerai automated (kaggle.com), with upload via NumerAPI, model is also staked so you can check its performance on leaderboard ( JOS_KAGGLE_HELLO - Numerai and improved version JOS_KAGGLE_SHATTER - Numerai)
  • numerai Feature Neutralization (kaggle.com) - example model with feature neutralization, JOS_KAGGLE_MEDIUM_FN - Numerai
  • numerai Target Ensemble (kaggle.com) - example model of target ensembling, JOS_KAGGLE_MEDIUM_TE - Numerai
  • Numerai Example Model Sunshine (kaggle.com) - older more complex example model, JOS_KAGGLE_SUNSHINE - Numerai
• Kaggle automation tips:
  • Triggering notebook execution by webhook with Flask and Kaggle API | Kaggle
  • Scheduling notebook execution with cron and Kaggle API | Kaggle
  • Running pipeline of notebooks with Kaggle API

[nathanganser]

Thank you! I’ll review this!!!

[dfrank]

Thank you! Also a new user exploring ways to train models (turns out my ~10 year old gaming rig can’t handle much past the “small” feature set)

[ambrul11]

Also using kaggle.com. Highly convenient option

[smilence666]

I train it on my local computer and use a batch script to submit dailly.

[datahunter]

This is very useful, thank you so much

[nathanganser]

Update on my end:

• I’ve tested many options (including Kaggle, but 30GM of RAM is limiting)
• I now exclusively use https://brev.dev/ which has super cheap GPUs and using screen you can easily leave your process running in the background for days and and get the model once the process is done.

Highly recommend that!

[svendaj]

Some guys are using Rent GPUs | Vast.ai as most price effective GPU option.

[gammarat]

I found it useful to have my programs periodically save relevant parameters to files. For example with my basic Numerai program I save parameters every 250 iterations to files named by the iteration number (like “XXXXX.mat”, where XXXXX is the iteration number, and “mat” because I use MatLab). In my case 250 iterations represents about 1/2 hour of processing, so there’s not much lost if for some reason or another processing is interrupted.

That also allows you to branch off downstream programs from pretty much where one likes.

FWIW I train my models at home, and the few times (for other projects) I’ve used Colab I used Google Drive for data storage.

[gregbowers]

This is very useful, thank you so much

[clot7213]

I’m experimenting with PaperSpace Gradient. It’s like Colab but with ‘unlimited’ GPUs with monthly subscription. I’m currently trying to build the correct virtual environments on Gradient so the pickled model is consistent with Numerai Evaluation. If it do work well, I’m guessing it would be a solid place for high-ram GPU training!