Will early stopping in cross validation introduce overfitting?

Data Science
1

Anyone has done experiments on early stopping and cross-validation? Does it give a better result using only cross-validation alone?

Also, how do you determine what n_estimator / num_of_epoch of your final model?

1 Like
2

Check out Optuna - A hyperparameter optimization framework. It makes picking hyper parameters a lot more efficient, and generally gives good results after ~30 trials.

2 Likes
3

Thanks for the sugguestion!

4

I’m mostly working with neural nets and so early stopping is very important. I am using ES + CV and while I don’t have enough live data to feel confident, I think it’s a principled approach that should at least allow me to evaluate a model with some degree of confidence.

5

Is your final model trained on train+valid dataset?

6

I don’t have a final model. I use the models trained during CV as an ensemble.

7

So you just average the predictions of the same model trained on different CV?

8

Yes, average the predictions of each of the models.