I think we should not evaluate 6-2 models on 22-2 basis. My view is the numerai team confused the issue by making this comparison the primary data point of their corr20 proposal.
What I inferred from Richard at last week’s roundtable is that numerai experimented with an internal model with both corr6 and corr20 targets, and what they saw excited them sufficiently to at least roll out the corr20 display.
Due to my previous (non-numerai) data for predicting ranked-returns of a large stock universe, I believe the relatively small predictive signal accumulates better over 20 days rather than 4 to help rise above the noise.
That said, I think the discussion on which time period works better for either hedge fund or the participants’ payout is premature until we build target20 models, see their corr20 performance, and compare with how life would be different compared to corr6. With 4 weeks for resolution, and maybe at least 10 rounds of live performance, I think we are easily looking at three months before a tournament change decision could be made.
Yes, especially as it is sort of a relative delta {(t6-t2)/t2 sort of thing}, rather than an absolute one.
Thinking about it overnight, it seems the issue opens up a whole range of interesting problems to look at. I’ve mentioned before I spent most of my career in target detection, tracking, and localization (ASW), which is really quite related to the market problem. And in that a significant “similar problem” was over/under estimation of a target track, which in turn is often related to the weighting one gives recent data over older data, given a particular method of analysis. Or it may point more towards using an adaptive prediction method (say, a la Kalman filtering) rather than a static, one off estimate.
I agree with this premise. The real added value of Signals is the ability to source your own data that is perhaps not in the 310 obsfucated features in Numerai classic albeit comes with a cost. The models can adjust with any predition time horizon if it’s worth more than salt. So with the DeFi revolution, data prices are starting to come down. So for me a better track to attract crowd source talent is to provide them a pool of free data they can pick and choose from. Then and only then will you see the talent unleash its creativity thus uniqueness!
Not necessarily, in the numeraI CLASSIC, we are given 310 features that are obfuscated so we have no idea what there are. We can pluck prices for free with Yahoo but fundamental data from FactSet or Morningstar would be good, also other alternative data like analyst estimates and broker recommendations. Sentiment indicators like Twitter or stocktwits, etc. This is what I mean. I am saying this because of my experience with Quantopian.