I played around with your code here and got similar results. Do these models combine well when you add them together?
One thing I found generating models in this way with the PCA features is that although the predictions would be uncorrelated the performance would be very correlated. For example, model1 and model2 could have no prediction correlation but fail or win on the exact same eras meaning they didn’t stack well. I would re-phrase my original question as can you build two models whose era scores are uncorrelated. These models will tend to work very well when combined.
Era boosting (Era Boosted Models) works in some ways by building many of these performance uncorrelated models.
I think era boosting approaches outperform example predictions and the median staking Numerai user isn’t beating example predictions so I think many users could benefit from these techniques.