What is the difference between feature exposure and regularization?

ryo_matsuzaka · September 24, 2022, 8:22am

Hello. I learned the concept of feature exposure thanks to several threads and a script:

numerai/example-scripts/blob/838bfd1788feaf40362d6bedb3e4683832a9dbb1/utils.py#L106


      
                  # To be consistent for all targets, let's embargo everything by 60/5 == 12 eras.
                  train_split = [e for e in train_split_not_embargoed if
                                 abs(int(e) - test_split_max) > embargo and abs(int(e) - test_split_min) > embargo]
                  train_splits.append(train_split)
          
              # convenient way to iterate over train and test splits
              train_test_zip = zip(train_splits, test_splits)
              return train_test_zip
          
          
          def neutralize(df,
                         columns,
                         neutralizers=None,
                         proportion=1.0,
                         normalize=True,
                         era_col="era"):
              if neutralizers is None:
                  neutralizers = []
              unique_eras = df[era_col].unique()
              computed = []
              for u in unique_eras:

When I saw the concepts, I remembered regularization.
The goal and basic concept of feature exposure is the same as the ones of regularization?

taori · September 24, 2022, 11:13am

In the post you linked, regularization and feature neutralization are two different methods used to reduce feature exposure.

regularization is used during model training to prevent the model from giving too much importance (and consequently exposure) to single features. This method depends on the particular model/algorithm in use and different models/algorithms might have (or not have) different parameters that control regularization.

feature neutralization is used on the model predictions (and so after training). It doesn’t depend on the model/algorithm in use. This method computes the exposure (e.g. using linear regression) of the predictions to a set of risky features and then removes this exposure while keeping the residuals.

Broadly speaking that is how I see the relationship between the two technincs.