# Where do you train your models (if not Google Colab) **URL:** \ **Category:** Tournament\ **Created:** [March 31, 2024, 7:22pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194 "2024-03-31T19:22:04Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![nathanganser](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/nathanganser/32/2993_2.png) [@nathanganser](https://forum.numer.ai/u/nathanganser)\ **Post date:** [March 31, 2024, 7:22pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/1 "2024-03-31T19:22:04Z")
As I’m starting to work with the larger datasets, my Google Colab functions are taking hours to run and thus disconnecting me in between when I leave my laptop. Is there another tool out there that can run a jupyter notebook in the background? What are you using? Thanks! ---
**Author:** ![svendaj](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/svendaj/32/3285_2.png) [@svendaj](https://forum.numer.ai/u/svendaj)\ **Post date:** [April 5, 2024, 2:56pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/2 "2024-04-05T14:56:16Z")
I am using and recommend [Kaggle](https://www.kaggle.com/). Not only that you can run for free [jupyter notebooks with 4 CPUs and 30GB RAM (max 12 hours run each, 5 notebooks in parallel)](https://www.kaggle.com/docs/notebooks#technical-specifications), but thanks to [Kaggle API](https://www.kaggle.com/docs/api) you can fully automatize your pipeline. On top, there is great community and possibility to learn more from competitions. I have created a few public notebooks and datasets to simplify first steps for Numerai Tournament participants: - weekly updated dataset with latest data, so that you do not need to download them each time: - [**V4.3 Midnight**](https://www.kaggle.com/datasets/svendaj/numerai-v4-3-midnight) - latest data, notebook producing the dataset: [numerai data v4.3 Midnight (kaggle.com)](https://www.kaggle.com/code/svendaj/numerai-data-v4-3-midnight) - **V4.2 Rain** , notebook producing the dataset: [numerai data v4.2 Rain (kaggle.com)](https://www.kaggle.com/code/svendaj/numerai-data-v4-2-rain) - [**Older V4 and V4.1**](https://www.kaggle.com/datasets/svendaj/numerai-latest-tournament-data) and notebook producing the dataset: [numerai data (kaggle.com)](https://www.kaggle.com/code/svendaj/numerai-data) - Example models - typically forks of Numerai example models with improvements for better results: - [Hello Numerai automated (kaggle.com)](https://www.kaggle.com/code/svendaj/hello-numerai-automated), with upload via NumerAPI, model is also staked so you can check its performance on leaderboard ( [JOS\_KAGGLE\_HELLO - Numerai](https://numer.ai/jos_kaggle_hello) and improved version [JOS\_KAGGLE\_SHATTER - Numerai](https://numer.ai/jos_kaggle_shatter)) - [numerai Feature Neutralization (kaggle.com)](https://www.kaggle.com/code/svendaj/numerai-feature-neutralization) - example model with feature neutralization, [JOS\_KAGGLE\_MEDIUM\_FN - Numerai](https://numer.ai/jos_kaggle_medium_fn) - [numerai Target Ensemble (kaggle.com)](https://www.kaggle.com/code/svendaj/numerai-target-ensemble) - example model of target ensembling, [JOS\_KAGGLE\_MEDIUM\_TE - Numerai](https://numer.ai/jos_kaggle_medium_te) - [Numerai Example Model Sunshine (kaggle.com)](https://www.kaggle.com/code/svendaj/numerai-example-model-sunshine) - older more complex example model, [JOS\_KAGGLE\_SUNSHINE - Numerai](https://numer.ai/jos_kaggle_sunshine) - Kaggle automation tips: - [Triggering notebook execution by webhook with Flask and Kaggle API | Kaggle](https://www.kaggle.com/discussions/questions-and-answers/397468) - [Scheduling notebook execution with cron and Kaggle API | Kaggle](https://www.kaggle.com/discussions/product-feedback/371090) - [Running pipeline of notebooks with Kaggle API](https://www.kaggle.com/code/svendaj/running-pipeline-of-notebooks-with-kaggle-api) ---
**Author:** ![nathanganser](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/nathanganser/32/2993_2.png) [@nathanganser](https://forum.numer.ai/u/nathanganser)\ **Post date:** [April 5, 2024, 5:24pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/3 "2024-04-05T17:24:33Z")
Thank you! I’ll review this!!! ---
**Author:** ![dfrank](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/dfrank/32/2870_2.png) [@dfrank](https://forum.numer.ai/u/dfrank)\ **Post date:** [May 3, 2024, 7:10am UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/4 "2024-05-03T07:10:34Z")
Thank you! Also a new user exploring ways to train models (turns out my ~10 year old gaming rig can’t handle much past the “small” feature set) ---
**Author:** ![ambrul11](https://avatars.discourse-cdn.com/v4/letter/a/e68b1a/32.png) [@ambrul11](https://forum.numer.ai/u/ambrul11)\ **Post date:** [May 7, 2024, 1:18pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/5 "2024-05-07T13:18:30Z")
Also using [kaggle.com](http://kaggle.com). Highly convenient option ---
**Author:** ![smilence666](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/smilence666/32/2980_2.png) [@smilence666](https://forum.numer.ai/u/smilence666)\ **Post date:** [May 10, 2024, 4:44pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/6 "2024-05-10T16:44:03Z")
I train it on my local computer and use a batch script to submit dailly. ---
**Author:** ![datahunter](https://avatars.discourse-cdn.com/v4/letter/d/8e7dd6/32.png) [@datahunter](https://forum.numer.ai/u/datahunter)\ **Post date:** [August 26, 2024, 6:51pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/8 "2024-08-26T18:51:56Z")
This is very useful, thank you so much ---
**Author:** ![nathanganser](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/nathanganser/32/2993_2.png) [@nathanganser](https://forum.numer.ai/u/nathanganser)\ **Post date:** [August 26, 2024, 7:04pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/9 "2024-08-26T19:04:55Z")
Update on my end: - I’ve tested many options (including Kaggle, but 30GM of RAM is limiting) - I now exclusively use [https://brev.dev/](https://brev.dev/) which has super cheap GPUs and using `screen` you can easily leave your process running in the background for days and and get the model once the process is done. Highly recommend that! ---
**Author:** ![svendaj](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/svendaj/32/3285_2.png) [@svendaj](https://forum.numer.ai/u/svendaj)\ **Post date:** [August 27, 2024, 7:10pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/10 "2024-08-27T19:10:24Z")
Some guys are using [Rent GPUs | Vast.ai](https://vast.ai/) as most price effective GPU option. ---
**Author:** ![gammarat](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/gammarat/32/3281_2.png) [@gammarat](https://forum.numer.ai/u/gammarat)\ **Post date:** [August 28, 2024, 1:59pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/11 "2024-08-28T13:59:28Z")
I found it useful to have my programs periodically save relevant parameters to files. For example with my basic Numerai program I save parameters every 250 iterations to files named by the iteration number (like “XXXXX.mat”, where XXXXX is the iteration number, and “mat” because I use MatLab). In my case 250 iterations represents about 1/2 hour of processing, so there’s not much lost if for some reason or another processing is interrupted. That also allows you to branch off downstream programs from pretty much where one likes. FWIW I train my models at home, and the few times (for other projects) I’ve used Colab I used Google Drive for data storage. ---
**Author:** ![gregbowers](https://avatars.discourse-cdn.com/v4/letter/g/91b2a8/32.png) [@gregbowers](https://forum.numer.ai/u/gregbowers)\ **Post date:** [September 17, 2024, 8:29am UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/12 "2024-09-17T08:29:34Z")
This is very useful, thank you so much 🙂 ---
**Author:** ![shi\_luo](https://avatars.discourse-cdn.com/v4/letter/s/bc79bd/32.png) [@shi\_luo](https://forum.numer.ai/u/shi_luo)\ **Post date:** [September 19, 2024, 8:21pm UTC](https://forum.numer.ai/t/where-do-you-train-your-models-if-not-google-colab/7194/13 "2024-09-19T20:21:32Z")
I’m experimenting with PaperSpace Gradient. It’s like Colab but with ‘unlimited’ GPUs with monthly subscription. I’m currently trying to build the correct virtual environments on Gradient so the pickled model is consistent with Numerai Evaluation. If it do work well, I’m guessing it would be a solid place for high-ram GPU training!