TC Feedback: Iterative Exhaustion and Impacts on Profitability

I can only speak for myself, but I doubt my experience is unique.
While I’m only engaged in the Signals side of things right now, I think this likely applies to both competitions.

When Numerai makes changes in targets, features, data, payable metrics, etc. It forces me back to an R&D phase and then a monitoring phase before I can engage in significant staking/risk-taking.

Last year I spent a lot of time on v.1 of my Signals pipeline. I launched those models around October, and after quite a few months of monitoring, finally felt I could stake on them. At the same time, I launched those models, I began working on v.2 of the pipeline, with a hopefully, much-improved feature set, that I spent A LOT of time researching before any development occurred.

The hope was, as v.1 made money through 2022 I could track v.2 and slowly allocate NMR to the new models. To be clear, despite my +100% returns in Signals, I’ve not made anything, truthfully, I’m in negative territory. Those returns all occurred in the monitoring phase and since staking, the models have largely wallowed around, until just recently.

Now I’m faced with a pending shift associated with TC. The models from v.1 are targeted toward Corr, so they will likely suck bigly on TC. V.2 models are targeted towards Corr and TB200 and I won’t be able to stake on those for some time, if at all now. The issue is, the risk of a shift to TC, for me, means I have to remove my stakes from v.1. I’ve made nothing in 2021 and I’ll likely not make anything in 2022. This has been a tone of work for literally nothing.

The rate of iteration, while exciting from a company/growth perspective, keeps many of us in a constant state of flux, where it’s difficult to know what has or will work, and we’re not given time to profit from our efforts (maybe that’s just me).

I’m for advancement but I feel like incentives might be breaking down (i.e. the incentive to participate…), I don’t know. What I do know is, if v.2 doesn’t somehow dumb luck into positive TC, it will likely signal the end of any serious participation for me. I can’t spend months in R&D and monitoring just to have the field goals moved as soon as I’m ready to start staking.

But maybe, that’s the point. Maybe TC is answering an internal problem we’re not aware of and the reduction of participation is the goal (higher PF?).

When the company appears to be making money, and the participants appear to be making money, one has to wonder why such a dramatic shift to a “make or break” metric is occurring, and why it’s even presented that way. And why isn’t there a more gradual implementation? The team could overlap TC with Corr and MMC for a year and allow for a natural shift by adjusting incentives to stake on TC. The move to this feels heavy-handed and doesn’t seem necessary based on what we know and have been told about current performance.

I don’t know, but I’m feeling pretty reluctant to reboot once again knowing some “new-new” is just one development window away and anything I do for the current version of TC will probably be worthless this time next year.

Sorry for the rant, disappointment, uncertainty, and iterative exhaustion is pretty high right now.

V.2 of Signals, targeting both Corr and TB200. Feels like it was for nothing tbh.

5 Likes

@objectscience one big important thing to note is that corr in signals is correlation to a feature neutral target. This is very different than corr in the main tournament and much closer to FNC. I fully expect that good corr and tb200 models on Signals will have good TC.

5 Likes

@mdo I appreciate that feedback and clarification. I really hope these models perform and I can justify staying on this journey.

3 Likes