Code to Calculate MMC vs Regular Payouts

[master_key]

After my MMC payout announcement in which I included some payout charts for various users comparing MMC and Regular Payouts, many users have been requesting their own charts, so I’m posting some code to help users generate their own.

First you need to get a pandas.DataFrame that looks like this
where index is round number, with each round’s correlation and mmc scores as columns.
This df is called user_df in my code.

You can get a similar DataFrame like this, but might have to rename a column or two.
(I’m having trouble with this right now but hopefully a user can reply saying how to actually do this correctly)

import numerapi
api = numerapi.NumerAPI()
user_df = pd.DataFrame(api.daily_submissions_performances("integration_test")).groupby("roundNumber").last()

Then running the following code will generate the stakes plot for MMC vs Primary Tourn.

end_round=204 # most recent resolved round
weekly_stakes_corr = {}
weekly_stakes_mmc = {}
stake_corr = 1 # initial stake
stake_mmc = 1
for r in range(168, end_round):
    if r in user_df.index:
        corr_score = user_df.loc[r, "correlation"]
        mmc_score = user_df.loc[r, "mmc"]
    else:
        corr_score = 0.0
        mmc_score = 0.0
    if corr_score:
            stake_corr *= 1 + corr_score*1
            stake_mmc *= 1 + mmc_score*2 #2x leverage for mmc
    weekly_stakes_corr[r] = stake_corr
    weekly_stakes_mmc[r] = stake_mmc

pd.DataFrame({"corr": weekly_stakes_corr, "mmc": weekly_stakes_mmc}).plot()

This code is slightly simplified because technically the stake selection would be from r-4 but it just makes the code a bit hard to read (r-4 and r+4s everywhere), and this is like 98% similar.

[hb_scout]

Probably doesn’t happen very often, if ever, but you could include the 25% caps on change in stake as well:

stake_corr *= 1 + (max(-0.25,min(0.25, corr_score*1)))
stake_mmc *= 1 + (max(-0.25,min(0.25, mmc_score*2))) #2x leverage for mmc

[jrdi]

Thanks for sharing @master_key! I added a clip (just in case) on (-0.2, 0.2) range for correlation and (-0.25, 0.25) for mmc.

I have been playing around with the code, the results aren’t as good as in @master_key original post but looks promising.

Also some of my new models are developed focusing on MMC (especially “mine4”) so, I would expect that after a few more rounds the numbers improve a bit!

[steppenwolf]

Hi, @master_key

Please, keep in mind, that daily_submissions_performances() function returns data in not strictly sorted by date order.

Look at the ‘date’ field in your DataFrame for rounds 176-179, they all have the same date.
To fix this we should sort rows by date before grouping:

napi.daily_submissions_performances(user)).sort_values(by=‘date’).groupby(“roundNumber”).last()

Maybe this bug can crawl out elsewhere, so be careful!

[jrdi]

@steppenwolf good point, I also miss it, and plots are affected significantly!

[jrdi]

Hi @master_key there’s something have been bothering me since the announcement of removing the leaderboard bonus and the introduction of the MMC payout. First of all, I understand the decision but I don’t think it will benefit the TOP N as it’s supposed. I have been making some calculation taking as an example niam model (sorry @mdo) since it’s usually on top 100 and has a huge stake and have been taking profit of the leaderboard bonus for a while.

I computed the return of this model with and without bonus and the difference is obviously huge (+256% vs +67%)

Since seems that MMC is coming to compensate removing the bonus, let’s see how it looks the payout comparison

If we take already resolved rounds niam can benefit from jumping to MMC leaderboard (only a 6% increase) but taking into account th,e upcoming rounds it doesn’t… (I know there aren’t resolved yet and the situation can change a lot but still…)

So, a model making huge profits (without exploiting anything) will experience a big cut on his profits once the bonus it’s removed. I’ve been wondering if the time spent on improving my models is going to be worthy.

Do you have any plan to compensate this situation? Or we just should expect a cut in our payouts?

[themicon]

On your graph 1.6 on the y-axis is a 60% increase from 1.0 not 6%. Or am I not reading this right?

Edit: I also don’t think it’s fair that you used an example model that has a 95% correlation with the metamodel. The original post by @master_key does also state that 84 out of the top 100 will have better returns using MMC. Did you perhaps pick a model that does not? Have you done the analysis on all the top 100? Does it lead to the same conclusion?

[jrdi]

You are right, it is not a 6% increase but a 25% at round 204 (the last completely resolved, dotted line), aprox numbers from 1.3 to 1.4 (from 30% to 40%), still far from 250%.

As far as I understand, they are presenting the MMC payout as a kind of alternative to the leaderboard bonus (or at least it’s what they are doing, removing the bonus and adding the mmc payout), even “dataman_ai” that was taking advantage of the bonus for only a few rounds…

@master_key statement says 84 out of the top 100 will have better returns using MMC comparing Corr payout vs MMC payout, and not Corr + Bonus payout vs MMC payout, that’s my point here.

[themicon]

OK, I get what you are doing. I’ve been looking at the MMC vs Corr payout for most of the top MMC models and there are very few that actually get to the 250% mark (which is what you are saying was the expected return for someone with Corr+Bonus in the top 100).