Understanding numeraire stakes and confidence


#1

Hello friends,

I am having trouble grokking the above example in the numeraire whitepaper, and I am hoping to get some guidance from this community,

If we only consider those who achieved logloss < -ln(0.5), we see that XIRAX, PHIL_CULLITON and ABRIOSI receives cash rewards. What I don’t understand is the intuition behind the s/c formula for calculating payouts. It seems strange to me that XIRAX gets paid less than ABRIOSI even though XIRAX staked more numeraire than ABRIOSI. What is the intuition behind the s/c formula?

Further, why is it that data scientists are compelled to reveal their true confidence level? It seems to me that if you have a great model that leads you to be VERY confident of your model, you will still think twice before submitting a high confidence score since it decreases your payout. If XIRAX somehow knew that his model was the best, wouldn’t he be motivated to submit the lowest confidence score possible? I understand the dutch auction mechanism here counters this somewhat because submitting a low confidence score means you might risk not getting paid at all. In practice, how has everyone been setting their confidence, and how does that relate to how much numeraire you stake?

The paper claims “The higher p, the higher c a data scientist will submit, and the more dollars the data
scientist can win from the auction.” Then why didn’t XIRAX stake more numeriare than PHIL_CULLITON?

Lastly, why are payouts not simply inversely proportional to the logloss? Is this just a convenience?

Thanks in advance!


#2

Yes, there is a tradeoff between payout and confidence. You want to submit the lowest C possible to maximize your payout, but at the same time if the staking pool is exhausted before your stake gets considered then you can’t win anything, so you want to maximize your C to ensure you get considered for payout. If payout wasn’t inversely proportional to C, then there wouldn’t be a compelling reason for participants to be realistic about their estimates of confidence, instead they would just submit higher and higher confidences to ensure they would be considered for payout.

The paper claims “The higher p, the higher c a data scientist will submit, and the more dollars the data scientist can win from the auction.” Then why didn’t XIRAX stake more numeriare than PHIL_CULLITON?

The amount of NMR that anyone is staking is obviously going to depend on how much they have, and their risk tolerance. If everyone had an unlimited amount of NMR, then of course, everyone is going to stake increasingly massive amounts, but that’s not the reality.

Lastly, why are payouts not simply inversely proportional to the logloss? Is this just a convenience?

This question is essentially asking “why isn’t the staking competition more like the main competition?” and the answer is because it’s a different competition. Log loss doesn’t matter for staking except in whether it’s better than chance or not. Log loss matters for the main competition, and in that case accounts are ranked by log loss with the top 100 getting payouts that decrease by rank. There are two different goals at work in these competitions. The main competition is about how much you can beat chance by. Unfortunately, the winners of this fluctuate massively from week to week because this isn’t an easy task with so much noise in the data. The staking competition favors consistency in performance, because the more often you beat chance, the higher your confidence will be and the more you can stake. There is more regularity in the winning accounts here as people are training models for consistency rather than low log loss.


#3

Thanks for the response daenris! I understand the tradeoffs better now.

18 AM

One followup question: given this formula, if NMR is worth a lot and e is small, so small that the entire right side tends to 1, then this would mean that players would need a confidence of 1 in order to play (regardless of the value of c). It makes sense to me that the more NMR is worth, the higher quality of predictions that the tournament will get. But is there a point where in practice no player would be willing to stake their NMR? I guess in this scenario it would make more sense to just play in the other tournament? In this sense, is the staking/cash tournament a bootstrapping mechanism for the numeraire tournament in the long term?

Also great to see you active on these forums :slight_smile: and sorry you got your NMR burned in the example :stuck_out_tongue: