It led me to start making synthetic “training* (I use the word loosely as I’m not using NNs) data that on a broad statistical basis is the same as the real training data, but on a narrower basis is more closely aligned with the test data. The big thing that caught my eye was that the variance in the various feature groups in the training data is significantly different from that in the test and live data. I wrote a bit about it in this thread, and what I learned from that seems to have improved my modelling.
1 Like