My model performed well on the first few weeks, and then the performance degraded. I wonder how often should we re-train our model to incorporate the new data.
How sensitive you are to the most recent data may depend on your specific model. Some models do best with up-to-date data, others are looking more for stable signals that don’t change much over time. Models that use more stable signals would ideally have less volatility and so be less susceptible to drawdowns.
But ultimately you can figure out what works best for your model by doing step-forward training. Backtest on the validation data by training on the “freshest” data for a particular validation era, vs. data that is a few weeks old and see which does better.
I retrain my model weekly
I retrain weekly, namely because it’s for free. I am fully automated on Kaggle platform, so it costs me nothing and I can focus on experimentation. If there were some costs related, I would certainly think twice about retraining frequency.
The frequency of retraining is almost like trying to time the market - a futile effort. For example, my best public model JOS_KAGGLE_SUNSHINE Profile - Numerai has been trained 10 months ago (no retraining since) and has reasonable 1Y return of 61.7% and its CORR20 performance is at #152 of the models leaderboard.