What is Feature Importance?

A bank’s propensity model ranks call duration as its most important input. The marketing team must then ask whether that variable is known when staff decide whom to call.

The scikit-learn guide to permutation importance defines it as the decrease in a model score when a single feature value is randomly shuffled. The guide also warns that impurity-based importances are biased toward high-cardinality features and can rank features highly that do not predict unseen data when the model overfits.

Lundberg and Lee (2017) introduced SHAP, which assigns each feature an importance value for a particular prediction and unifies several earlier explanation methods.

The UCI Bank Marketing dataset page notes that call duration is unknown before a call and should be discarded for a realistic predictive model. CUBIG’s 2026 Bank Marketing experiment kept duration among its inputs, so its results do not show pre-call targeting performance.

Common methods include:

  • Impurity-based importance, which is built into tree models and quick to compute
  • Permutation importance, which can be measured on held-out data by shuffling one feature at a time
  • SHAP values, which explain individual predictions and can be averaged across a dataset
  • Drop-column importance, which retrains the model without a feature, similar to an ablation study

How is feature importance different from feature selection?

  • Feature importance describes how a trained model uses its inputs.
  • Feature selection decides which inputs the model should receive and can use importance scores as one signal.
  • A feature can be important to the model and still be unusable for the task, for example when it is recorded after the outcome.

Where does feature importance break in AI work?

Feature importance breaks when a top feature leaks the answer. A variable recorded after the outcome can dominate the ranking, and the model’s score can fall once that variable is unavailable in production, as described for data leakage.

Rankings can also mislead when features are correlated, because importance can be split between them or shift from one to the other between runs. Importance shows association with the model’s output, not cause in the business process, so teams should confirm the timing and meaning of each top feature with the people who own the data.

Frequently asked questions

What is the best way to measure feature importance?

Permutation importance is model-agnostic and can be computed on held-out data, so it suits most comparisons. SHAP values help when teams need explanations for individual predictions.

Does high feature importance mean the feature causes the outcome?

No. Importance measures how much the model relies on a feature, which can reflect correlation, leakage or proxy effects.

Why do impurity-based importances mislead?

scikit-learn notes that they can favor high-cardinality features and, when a model overfits, rank features that do not help on new data.