Is anyone using a hosted environment (Colab, etc.) for their models? I am using the default Colab with 12GB of memory and it is a pain.
try out Kaggle notebooks
I will check that out. I think it has the same memory footprint as Google Colab.
Colab works great! We just need to optimize the code (this does take a lot of time).
My primary problem was limited amount of memory but I managed to get everything done under 3.0 GB.
I can get everything loaded from the files to my processed data frames. But, after that I can’t release the memory. I have to restart the session and then load the data frames with int16 columns and I am good the rest of the way
https://colab.research.google.com/drive/1D6krVG0PPJR2Je9g5eN_2h6JP73_NUXz This is the notebook that @surajp linked in rocket chat. Should give you a P100 and more RAM/Storage
Wow, this is really cool. Did some digging in the raw notebook, and it looks like adding
does the magic. Sure enough, found a tutorial on how you can do it with your own notebooks. Handy if you’ve already got some existing notebooks you want to add this feature to.