Python code to validate submissions by validation correlation to run after you attempt to submit

bensch · April 25, 2021, 3:34pm

benschreyer/NumeraiScripts/blob/main/verify_by_val_corr.py

#Bensch 4/20/21
#Script to check validation correlation of a list of Numer.ai models and print whether or not they changed from the previous round to the current
import requests



def getAccountValidations(name):
    req = requests.get("https://api-tournament.numer.ai/?query={userActivities(username:\""+name+"\",tournament:8){submission {validationCorrelation } } } ").json()
    return (req["data"]["userActivities"][1]["submission"]["validationCorrelation"],req["data"]["userActivities"][0]["submission"]["validationCorrelation"])

accounts = ["bensch","bensch_a"]

for account in accounts:
    get = getAccountValidations(account)
    #print(get)
    prev = get[0]
    curr = get[1]
    if(curr == prev):
        print(account+ " UNCHANGED " + str(prev))
    else:

This file has been truncated. show original

This could be useful to anyone that wants to check their underlying model didn’t change week to week for some unexpected reason by getting the validation correlations of the model for the two latest rounds and checking if they match. Note this has the blocking line input() at the end since I run it after all my other scripts when I do submissions locally.