Will early stopping in cross validation introduce overfitting?

Anyone has done experiments on early stopping and cross-validation? Does it give a better result using only cross-validation alone?

Also, how do you determine what n_estimator / num_of_epoch of your final model?

1 Like

Check out Optuna - A hyperparameter optimization framework. It makes picking hyper parameters a lot more efficient, and generally gives good results after ~30 trials.


Thanks for the sugguestion!

I’m mostly working with neural nets and so early stopping is very important. I am using ES + CV and while I don’t have enough live data to feel confident, I think it’s a principled approach that should at least allow me to evaluate a model with some degree of confidence.

Is your final model trained on train+valid dataset?

I don’t have a final model. I use the models trained during CV as an ensemble.

So you just average the predictions of the same model trained on different CV?

Yes, average the predictions of each of the models.