Xgboost - How to use feature_importances_ with XGBRegressor()?

Question

How could we get feature_importances when we are performing regression with XGBRegressor()?

There is something like XGBClassifier().feature_importances_?

(XGBClassifier().feature_importances_) it is right , where is the problem ?? — Abhishek Verma, Jun 21 '17 at 16:30
@Abhishek I can't use feature_importances_ with XGBRegressor(), because it works only with XGBClassifier(). — Simone, Jun 21 '17 at 16:46
I had to use: model.get_booster().get_score(importance_type='weight') — RyGuy, Jun 07 '18 at 22:54

BenP · Answer 1 · 2019-05-07T07:50:20.157

13

from xgboost import XGBClassifier
model = XGBClassifier.fit(X,y)

# importance_type = ['weight', 'gain', 'cover', 'total_gain', 'total_cover']
model.get_booster().get_score(importance_type='weight')

However, the method below also returns feature importance's and that have different values to any of the "importance_type" options in the method above. This was raised in this github issue, but there is no answer [as of Jan 2019].

model.feature_importances_

edited May 07 '19 at 07:50

answered Nov 06 '18 at 10:53

BenP

317
2
8

2

The values in the list feature_importances_ equal the values in the dict get_score(importance_type='weight') where each element is divided by the sum of elements. – Anton Tarasenko Nov 22 '18 at 11:33
2

Which importance_type is equivalent to the sklearn.ensemble.GradientBoostingRegressor version of feature_importances_? My suspicion is total_gain – Keith Jan 18 '19 at 18:49

score 4 · Accepted Answer · answered Jun 23 '17 at 15:18

4

Finally I have solved this issue by:

model.booster().get_score(importance_type='weight')

answered Jun 23 '17 at 15:18

Simone

705
1
14
23

1

But mine returned an error : TypeError: 'str' object is not callable – Tony Wang Mar 20 '18 at 09:39
A bit off-topic, have you tried github.com/slundberg/shap for feature importance? It looks a bit complicated at first, but it is better than normal feature importance. – TwinPenguins Jun 08 '18 at 10:28
for me: model.get_score(importance_type='weight') – Catbuilts Oct 15 '18 at 07:06
@TonyWang try model.get_booster().get_score(importance_type='weight') instead. – Sndn Jan 26 '19 at 06:46
Sndn's solution worked for me as on 04-Sep-2019 – Tanveer Uddin Sep 04 '19 at 04:27

score 3 · Answer 3 · answered May 20 '18 at 02:36

3

In the past the Scikit-Learn wrapper XGBRegressor and XGBClassifier should get the feature importance using model.booster().get_score(). Not sure from which version but now in xgboost 0.71 we can access it using

model.feature_importances_

answered May 20 '18 at 02:36

byrony

131
3

I'm using from xgboost.sklearn import XGBRegressor in version 0.72.1 and this worked for me. Thanks! – Adam Jul 20 '18 at 17:57

Xgboost - How to use feature_importances_ with XGBRegressor()?

3 Answers3

Linked