# What is your submission automation cost? **URL:** \ **Category:** Tournament\ **Created:** [October 18, 2022, 12:26am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755 "2022-10-18T00:26:40Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![ryo\_matsuzaka](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/ryo_matsuzaka/32/3049_2.png) [@ryo\_matsuzaka](https://forum.numer.ai/u/ryo_matsuzaka)\ **Post date:** [October 18, 2022, 12:26am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/1 "2022-10-18T00:26:40Z")
I am thnking about submission automation in cloud. ---
**Author:** ![jacob\_stahl](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jacob_stahl/32/881_2.png) [@jacob\_stahl](https://forum.numer.ai/u/jacob_stahl)\ **Post date:** [October 18, 2022, 12:59am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/2 "2022-10-18T00:59:23Z")
I’m considering moving my submission code to Azure Functions [Azure Functions Overview | Microsoft Learn](https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview). The cost is basically $0, I just haven’t gotten around to it yet. ---
**Author:** ![ryo\_matsuzaka](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/ryo_matsuzaka/32/3049_2.png) [@ryo\_matsuzaka](https://forum.numer.ai/u/ryo_matsuzaka)\ **Post date:** [October 18, 2022, 3:31am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/3 "2022-10-18T03:31:05Z")
Does it have enough computational resources? Similar service like GCP cloud functions has limatation for memory which is not suitable for training, I think. ---
**Author:** ![jacob\_stahl](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jacob_stahl/32/881_2.png) [@jacob\_stahl](https://forum.numer.ai/u/jacob_stahl)\ **Post date:** [October 18, 2022, 11:18pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/4 "2022-10-18T23:18:12Z")
Not for training, like not even close. Azure functions are limited too ~1.5GB of memory, which should be enough for most models to predict a single era. I think Google Cloud and AWS offer equivalent, but I don’t know what their limitations are. ---
**Author:** ![ryo\_matsuzaka](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/ryo_matsuzaka/32/3049_2.png) [@ryo\_matsuzaka](https://forum.numer.ai/u/ryo_matsuzaka)\ **Post date:** [October 19, 2022, 3:59am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/5 "2022-10-19T03:59:43Z")
Don’t you update trained model with updated dataset? ---
**Author:** ![jacob\_stahl](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jacob_stahl/32/881_2.png) [@jacob\_stahl](https://forum.numer.ai/u/jacob_stahl)\ **Post date:** [October 20, 2022, 1:26am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/6 "2022-10-20T01:26:07Z")
I have a lot of older models that I’m afraid to retrain. ---
**Author:** ![papaemman](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/papaemman/32/1453_2.png) [@papaemman](https://forum.numer.ai/u/papaemman)\ **Post date:** [November 30, 2022, 12:21am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/7 "2022-11-30T00:21:39Z")
@jacob_stahl @ryo_matsuzaka I have published an article describing “How I automated my Numerai weekly submission pipeline for free, using Azure functions and python”. I think you will find it useful! > **[How I automated my Numerai weekly submissions pipeline for free, using Azure...](https://medium.com/@papaemman.pan/how-i-automated-my-numerai-weekly-submissions-pipeline-for-free-using-azure-functions-and-python-9bcf8382af1c)** > > This guide describes how I set up my own weekly submission pipeline from scratch, using Microsoft Azure and python for free. 🚀 > [@\[Community Release\] Automated Numerai Tournament weekly submission pipeline for free, using Azure functions and python](http://forum.numer.ai/t/community-release-automated-numerai-tournament-weekly-submission-pipeline-for-free-using-azure-functions-and-python/5432): > > Hello guys, I know that many people are struggling to set up the Numerai Compute either because they don’t have an AWS account or because numerai-cli seems strange. Anyway, I just published a Medium article describing “How I automated my Numerai weekly submission pipeline for free, using Azure functions and python”. Here is the source code: [GitHub - papaemman/azure-functions-with-python: A complete guide on developing and deploying Azure functions with Python, using VSCode and Azure extensio…](https://github.com/papaemman/azure-functions-with-python) ---
**Author:** ![ryo\_matsuzaka](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/ryo_matsuzaka/32/3049_2.png) [@ryo\_matsuzaka](https://forum.numer.ai/u/ryo_matsuzaka)\ **Post date:** [November 30, 2022, 3:28am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/8 "2022-11-30T03:28:35Z")
Thank you very much for the information! I absolutely check it. ---
**Author:** ![hiydavid](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/hiydavid/32/2055_2.png) [@hiydavid](https://forum.numer.ai/u/hiydavid)\ **Post date:** [November 30, 2022, 5:17pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/9 "2022-11-30T17:17:34Z")
I use Deepnote. I’m able to submit daily predictions using their notebook scheduler feature. ---
**Author:** ![jacob\_stahl](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/jacob_stahl/32/881_2.png) [@jacob\_stahl](https://forum.numer.ai/u/jacob_stahl)\ **Post date:** [December 2, 2022, 1:09am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/10 "2022-12-02T01:09:20Z")
Thats a thing? ![d63c1ef766f4d12f14127148d3c0e45a](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/d/d5ae86a43a205cba51d4de50dcc0ed03d4c471af.gif) ---
**Author:** ![autratec](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@autratec](https://forum.numer.ai/u/autratec)\ **Post date:** [December 2, 2022, 1:23pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/11 "2022-12-02T13:23:54Z")
But now the submission changed to email triggered, how the schedule will work ? ---
**Author:** ![monticola](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/monticola/32/1449_2.png) [@monticola](https://forum.numer.ai/u/monticola)\ **Post date:** [December 2, 2022, 6:33pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/12 "2022-12-02T18:33:25Z")
My submission automation cost it around 0.6 €/Month on Azure. It runs on a container instance that loads the models from a cloud storage, predicts and stores the results on the same cloud storage until the next round. It runs very fast since it only predicts using live data, so no training is performed.It takes the container about 90 seconds to run my 12 models. My whole cloud infrastructure consist in the container instance, the cloud storage, one http trigger for the submissions, one http trigger for numerbay sales and two scheduled triggers, one for daily rounds and one for weekend rounds. The only small detail is that my container image is hosted on a free private repository outside Azure, so that cost it is not accounted for. Actually, that image repository would be the most expensive resource, costing about 4.2€ per month. Although I would not be surprised if there are less expensive alternatives out there. Edit: I forgot to mention that this represents the cost of daily and weekly predictions. Since this thread is relatively old, maybe I should remark that. I am bad at predictions, but been smoothly predicting daily since day 1 of daily predictions 🕶 😜 ![image](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/7/792fc5059bbdb42961090d8485588ae1a3747170.png) ---
**Author:** ![nyuton](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/nyuton/32/687_2.png) [@nyuton](https://forum.numer.ai/u/nyuton)\ **Post date:** [December 6, 2022, 8:42am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/13 "2022-12-06T08:42:31Z")
![image](https://canada1.discourse-cdn.com/flex009/uploads/numerai/original/2X/8/806eb9dd806cb9d4989455d993fa3d054572d767.png) I have some non-Numerai data in S3 as well, so compute for one model is less than this. ---
**Author:** ![svendaj](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/svendaj/32/3285_2.png) [@svendaj](https://forum.numer.ai/u/svendaj)\ **Post date:** [December 8, 2022, 4:42pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/14 "2022-12-08T16:42:19Z")
I am using Kaggle for training and for submissions, because they provide for free notebooks with 4 CPU cores and 30GB RAM. Their notebook scheduling capability is not usable for numerai submissions, so I have created [workaround solution with (always 😉) free Oracle Cloud compute node, running Kaggle public API and `cron` trigerring my numerai notebooks at announced round openning times](https://www.kaggle.com/discussions/product-feedback/371090). Inside of my notebooks I [wait for actual round openning](http://forum.numer.ai/t/server-errors-on-friday/5883/5). To sum it up: I remain at big fat **zero** automation costs 🤓 ---
**Author:** ![autratec](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@autratec](https://forum.numer.ai/u/autratec)\ **Post date:** [December 9, 2022, 1:54am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/15 "2022-12-09T01:54:50Z")
thanks for the sharing. any idea to go further, to get kaggle notebook be trigger by an external API call ? like web hooks, or something else ? @svendaj hi, i just had a thought to include your round readiness checking script in my notebook. and still leverage deepnote to run the notebook at every evening 9pm and check the round open status every 10mins. I think i will resolve the daily submission now. Again. thanks for the shasing. ---
**Author:** ![autratec](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@autratec](https://forum.numer.ai/u/autratec)\ **Post date:** [December 9, 2022, 3:13pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/16 "2022-12-09T15:13:26Z")
just tried script: napi.check\_round\_open() in deepnote. And realize that deepnote gives following error: AttributeError: ‘NumerAPI’ object has no attribute ‘check\_round\_open’ Same code run in colab is working fine. Any one encounter this issue before in deepnote ? ---
**Author:** ![svendaj](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/svendaj/32/3285_2.png) [@svendaj](https://forum.numer.ai/u/svendaj)\ **Post date:** [December 11, 2022, 11:01pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/17 "2022-12-11T23:01:06Z")
Sorry for later reply, so far I was working on running my notebook swarm in Kaggle. Today I have published [Kaggle notebook which can create and control complex pipeline of other notebooks](https://www.kaggle.com/code/svendaj/running-pipeline-of-notebooks-with-kaggle-api). This was needed for my Saturday night numerai fever, when I was manually running 18 notebooks. Starting with [weekly data update](https://www.kaggle.com/code/svendaj/numerai-data), immediately followed by 9 notebooks and 3 sub-pipelines with models to be retrained on weekly data. When this was ready (usually Sunday morning) I launched final stacking notebook. Now I am able to put everything to one master notebook and launch it with `cron` from cloud automatically 🕶 Webhooks will be my next step, but not sure when I will have some time to have a look at it. ---
**Author:** ![autratec](https://avatars.discourse-cdn.com/v4/letter/a/6bbea6/32.png) [@autratec](https://forum.numer.ai/u/autratec)\ **Post date:** [December 12, 2022, 3:47am UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/18 "2022-12-12T03:47:58Z")
thanks for the sharing - “one ring rule them all” ---
**Author:** ![nosaai](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/nosaai/32/2929_2.png) [@nosaai](https://forum.numer.ai/u/nosaai)\ **Post date:** [December 16, 2022, 1:31pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/19 "2022-12-16T13:31:33Z")
Would you mind enlightening on how you got the notebook to make submissions on Deepnote? What I really mean is getting the auto-submission to work. Using the Numerai submission API gives really funny errors ---
**Author:** ![nosaai](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/nosaai/32/2929_2.png) [@nosaai](https://forum.numer.ai/u/nosaai)\ **Post date:** [December 16, 2022, 1:48pm UTC](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755/20 "2022-12-16T13:48:21Z")
I have had exactly the same problem only this is in VSC on my computer. This must be something to do with the api. If you manage to solve the problem, please let me know. [Next page](https://forum.numer.ai/t/what-is-your-submission-automation-cost/5755.md?page=2)