I want to discuss a specific and fixable problem with TC that has not yet been brought up: that of optimizer misalignment.
We actually do know a lot about how TC works. To summarize it briefly, TC is a gradient with respect to user stake size of the following process:
- User predictions are gaussianized
- A stake-weighted mean prediction is taken, the meta-model prediction
- The meta-model prediction is normalized
- The meta-model prediction is raised to an exponent of 1.5 (mmp = mmp*abs(mmp)**0.5)
- The meta-model prediction is input to a cvxpy layer (“the optimizer”) with costs and constraints related to feature and factor exposure, allocation constraints, long-short neutrality, maximum position sizes, and whatever others.
- The output of the cvxpy layer is a theoretical portfolio that would adopted if not for a couple additional costs/constraints used in live trading that are excluded
- The return of our theoretical portfolio is calculated ( dot(position_sizes, market_returns) ). This is the value of which TC a gradient
Although the cvxpy layer is a bit of a black box, there is still quite a bit we can infer about it. Here is an example portfolio optimization problem from cvxpy that includes feature exposure and volatility costs as well as constraints related to asset allocation and long short neutrality:
# Factor model portfolio optimization.
w = cp.Variable(n)
f = cp.Variable(m)
gamma = cp.Parameter(nonneg=True)
Lmax = cp.Parameter()
ret = mu.T @ w
risk = cp.quad_form(f, Sigma_tilde) + cp.sum_squares(np.sqrt(D) @ w)
prob_factor = cp.Problem(
cp.Maximize(ret - gamma * risk),
[cp.sum(w) == 0, f == F.T @ w, cp.norm(w, 1) <= Lmax],
)
# Solve the factor model problem.
Lmax.value = 2
gamma.value = 0.1
prob_factor.solve(verbose=True)
We can see that this problem maximizes ( expected return – costs ) subject to some constraints. Expected return being equal to dot(stock_expected_return, stock_position_size). I think Numerai likely does something similar with the meta-model prediction in the place of expected return. The meta-model prediction isn’t necessarily being optimized to be a good estimate of expected return however which poses a problem. I will explain why I think that is and how to fix it next.
The problem is this: because TC optimizes only for return (and not return minus costs,) TC will optimize toward cvxpy layer inputs that maximize the expected return of the cvxpy layer output instead of inputs that best represent expected return. To understand the significance of this consider for example what would happen if user predictions and the meta-model were not normalized, in that case TC would drive the input to the cvxpy layer to have an arbitrarily large scale because a large enough input would render any costs in the optimizer irrelevant, then expected return would increase but at the expense of taking on arbitrarily high feature exposure and ignoring any other costs as well.
Of course the meta-model is normalized, but that does not prevent other adversarial inputs to the optimizer. The optimizer only adopts positions in a couple hundred stocks, so the input values for the majority of the stocks, those that do not get positions, are irrelevant. Therefore, although we cannot change the scale of the meta-model prediction (because it is normalized), we can allocate more of the scale towards those stocks that the optimizer will adopt positions in to the same effect (increased expected returns at the expense of ignoring costs). This suggests that TC will tend to punish users who allocate a lot scale to stocks that the optimizer does not adopt positions in. Thus it may be that TC punishes more novel predictions if they do not assign enough scale to the stocks that others do assign large scales to (as those are the stocks the optimizer is likely to take positions in.)
So far I have speculated about some of the qualities I expect that an adversarial input to the optimizer might have, however it can be very difficult to predict what an adversarial input will look like especially without knowing the exact costs that the optimizer uses. The fact that an adversarial input is possible is more important than the details of what such an input might look like, as whatever adversarial qualities TC rewards, it does so at the expense of failing to properly reward the best predictions.
How to Fix It
The source of our problem is that TC and the optimizer optimize for different things, TC maximizing return and the optimizer maximizing “expected return” minus costs. If TC were to maximize return minus costs instead, then the best possible input to the optimizer would be the best possible estimate of expected return and adversarial inputs are no longer possible. However normalizing the meta-model complicates this. Normalizing the prediction risks making the prediction magnitude too large and taking on too much cost, potential creating a situation where TC rewards a worse prediction direction that gets less cost. One solution might be to make TC based on normalized return minus costs, but then our expected return (meta-model prediction) would have to be normalized to a much smaller magnitude than actual returns because the fraction of returns that is actually predictable is very small. Alternatively, we could drop normalization entirely, but this would require a larger reworking of the Numerai system.