# Feature Groups and Interactions
**URL:** https://forum.numer.ai/t/feature-groups-and-interactions/1324
**Category:** Data Science
**Created:** [December 17, 2020, 12:34pm UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324 "2020-12-17T12:34:10Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@shonumerai123](https://forum.numer.ai/u/shonumerai123)
#### Post date: [December 17, 2020, 12:34pm UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/1 "2020-12-17T12:34:10Z")
There are 6 groups of features in the dataset as everyone knows and I’ve been always thinking there should be some reasons behind that.
Followings are the avenues I’ve tried to explore so far:
1. Train models on a feature group (i.e. Dexterity only, Strength only etc), a combination of feature groups (i.e. Intelligence & Strength, Dexterity & Charisma & Constitution etc). There can be so many variations. Take subsets from each group and combine them, ensemble predictions etc etc.
2. Generate some representative features from each feature group (e.g. PCAs, correlations, stds…) and use them for predictions.
3. Limit interactions within a feature group and look at interactions with other feature groups only. (I’ve tried a XGBoost version below.)
[https://xgboost.readthedocs.io/en/latest/tutorials/feature\_interaction\_constraint.html](https://xgboost.readthedocs.io/en/latest/tutorials/feature_interaction_constraint.html)
None of these attempts have led to any meaningful improvement in the metrics so far, unfortunately…
Has anyone tried something similar? Will be great if you could possibly share you 2 cents. Thanks!
---
### Author:  [@arbitrage](https://forum.numer.ai/u/arbitrage)
#### Post date: [December 21, 2020, 6:25pm UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/2 "2020-12-21T18:25:10Z")
I tried the XGBoost feature interaction constraint and notice no discernable difference between using it and not using it. I have since abandoned the project. I do believe that the next frontier in model development will include some type of feature restriction, but so far, the frontier lies ahead…
---
### Author:  [@restrading](https://forum.numer.ai/u/restrading)
#### Post date: [December 23, 2020, 4:46am UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/3 "2020-12-23T04:46:32Z")
My understanding is that the interaction constraints implemented by XGBoost is to allow only interactions within groups, not across groups. This is the opposite of what we want.
---
### Author:  [@shonumerai123](https://forum.numer.ai/u/shonumerai123)
#### Post date: [December 23, 2020, 5:44am UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/4 "2020-12-23T05:44:37Z")
Thanks for your reply.
Yes, you are right about the way XGB implemented that. So I defined 310 lists and gave them as constraints. [feature\_dexterity1, all features EXCEPT other dexterity features], [feature\_dexterity2, all features EXCEPT other dexterity features], etc etc…
---
### Author:  [@restrading](https://forum.numer.ai/u/restrading)
#### Post date: [December 23, 2020, 6:49am UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/5 "2020-12-23T06:49:15Z")
> [@shonumerai123](#):
>
> all features EXCEPT …
So each constraint group has singled out one feature group into “quarantine”, but this still allows interactions among other feature groups.
Does having these many constraints affect training speed significantly?
---
### Author:  [@shonumerai123](https://forum.numer.ai/u/shonumerai123)
#### Post date: [December 23, 2020, 7:23am UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/6 "2020-12-23T07:23:44Z")
It wasn’t a problem but the result just wasn’t great unfortunately… I still don’t know how to make a good use of feature groups. The way I train my model at the moment is completely ignoring the groups so it’s no different even if they are given as feature1-feature310.
---
### Author:  [@shonumerai123](https://forum.numer.ai/u/shonumerai123)
#### Post date: [December 23, 2020, 8:08am UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/7 "2020-12-23T08:08:03Z")
Just realized this isn’t a proper way. Non-dex features can still interact within the group…
---
### Author:  [@restrading](https://forum.numer.ai/u/restrading)
#### Post date: [December 23, 2020, 8:08am UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/8 "2020-12-23T08:08:47Z")
Yes, that’s what I was referring to
---
### Author:  [@shonumerai123](https://forum.numer.ai/u/shonumerai123)
#### Post date: [December 23, 2020, 9:01am UTC](https://forum.numer.ai/t/feature-groups-and-interactions/1324/9 "2020-12-23T09:01:06Z")
Agreed. It’s impractical to define all possible allowed interactions…