# How to estimate TC with the numerai meta model data

**URL:** <https://forum.numer.ai/t/how-to-estimate-tc-with-the-numerai-meta-model-data/6032>\
**Category:** Data Science\
**Created:** [January 12, 2023, 7:46am UTC](https://forum.numer.ai/t/how-to-estimate-tc-with-the-numerai-meta-model-data/6032 "2023-01-12T07:46:43Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![nyuton](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/nyuton/32/687_2.png) [@nyuton](https://forum.numer.ai/u/nyuton)\
**Post date:** [January 12, 2023, 7:46am UTC](https://forum.numer.ai/t/how-to-estimate-tc-with-the-numerai-meta-model-data/6032/1 "2023-01-12T07:46:43Z")

</div>

Hi,

TC is a metric that shows, how much your model improves the meta model.  
That means that ensembling the predictions of my model with the metamodel should improve the meta model metrics like corr and sharpe. The fund also picks trades from the stocks, where the metamodel has the most confidence (top/bottom 200).

With that in mind, we can estimate the (past) TC of a model with the following script:

> validation[‘my\_prediction’] = my\_model.predict(validation[features])
> 
> mm = pd.read\_parquet(‘v4.1\_meta\_model.parquet’)  
> validation= validation.join(mm[‘numerai\_meta\_model’])
> 
> validation.loc[:, “mm\_ensemble”] = validation[[‘era’, ‘my\_prediction’, ‘numerai\_meta\_model’]].dropna().groupby(‘era’).rank(pct=True).mean(axis=1)
> 
> validation\_stats = validation\_metrics(  
> validation,  
> [‘my\_prediction’, ‘mm\_ensemble’, ‘numerai\_meta\_model’],  
> example\_col=EXAMPLE\_PREDS\_COL,  
> fast\_mode=False,  
> target\_col=TARGET\_COL,  
> )  
> print(validation\_stats[[‘mean’, ‘sharpe’, ‘tb200\_mean’, ‘tb200\_sharpe’]])

So TC should be correlated with the gain of the meta model after ensembling.  
I guess, if the tb200\_mean of the ensemble is lower than the that of the metamodel, then there is no TC to expect from that model.  
Ideally, the ensemble should outperforms both of it’s components.

Do you think that the above estimation method is correct?  
Has anyone came up with a better estimation?

---

<div class="post-metadata">

**Author:** ![jmrichardson](https://avatars.discourse-cdn.com/v4/letter/j/eb8c5e/32.png) [@jmrichardson](https://forum.numer.ai/u/jmrichardson)\
**Post date:** [January 12, 2023, 4:19pm UTC](https://forum.numer.ai/t/how-to-estimate-tc-with-the-numerai-meta-model-data/6032/2 "2023-01-12T16:19:57Z")

</div>

That seems reasonable to me. I believe I remember hearing that it was actually top and bottom 500 (instead of 200)?

---

<div class="post-metadata">

**Author:** ![olivepossum](https://yyz1.discourse-cdn.com/flex009/user_avatar/forum.numer.ai/olivepossum/32/2259_2.png) [@olivepossum](https://forum.numer.ai/u/olivepossum)\
**Post date:** [January 12, 2023, 8:49pm UTC](https://forum.numer.ai/t/how-to-estimate-tc-with-the-numerai-meta-model-data/6032/3 "2023-01-12T20:49:04Z")

</div>

I think it’s 200 but MDO shared a post optimizing for 500 as mentioned it overfitted less [Optimizing for FNC and TB scores](http://forum.numer.ai/t/optimizing-for-fnc-and-tb-scores/5132)