Numerblox is a library to simplify software engineering around Numerai inference pipelines. Its older sibling is being used by the CrowdCent models, and @perfect_fit and @jrai are interested in feedback on this new version.
What is NumerBlox 1.0?
NumerBlox 1.0 focuses on:
End-to-end pipelines and full scikit-learn compatibility.
Simplification of the package structure, with fewer mandatory dependencies.
Leveraging new v4.2 data fully.
For more details on the library’s features and improvements, see the preview post.
The library is compatible with Python 3.9+. You install numerbloc v1 with: pip install -U numerblox
How to Participate in the Bounty:
Use the library
Provide constructive feedback on your experience, including any bugs or improvement suggestions in this bounty thread for a 1 NMR bounty (bounties are low, but their purpose is to give starting numerati outside of the crypto-sphere an opportunity to add some NMR to their accounts and get into the stake in the game mode).
Bonus NMR for pull requests and bug fixes on github.
If the system is not being gamed, and you want to get paid, add your discord link or an address capable of receiving NMR, and @bor1 will sporadically go through the posts and organize the payouts.
Thanks for participating in community-made tools! @bor1.
One recent fun thing we included in NumerBlox v1 is the ability to add benchmark models to your evaluation. If you include benchmark model columns in the evaluator it will return Corrv2 and Sharpe outperformance of your predictions vs. benchmark models and correlation of your predictions with benchmarks.
Here is a simple example for usage:
from numerblox.evaluation import NumeraiClassicEvaluator
# fast_mode will skip calculation of FNC, because this can take a while.
evaluator = NumeraiClassicEvaluator(fast_mode=True)
# Your validation data with columns prediction, era, target
# example_preds, meta_model_prediction and rain_ensemblev2 cols.
val_df = ...
# meta_model_col and benchmark_cols are optional
metrics = evaluator.full_evaluation(val_df,
# Pandas DataFrame with metrics
Good one! NumeraiClassicEvaluator will now give legacy EPC and MMC metrics with .full_evaluation. Legacy MMC will only be calculated if you define a meta_model_col and have meta model predictions in your validation DataFrame.
NOTE: Numerai hasn’t released the full details on these metrics yet so we use the “MMC2” calculation and deliberately call it “legacy”.
This quickstart notebook explains evaluation in step 3. Note that you also need to add meta model predictions and specify meta_model_col.