Feature importance estimation is crucial for model interpretability, but traditional permutation-based methods break down when features exhibit dependencies. Standard permutation importance shuffles features independently, creating out-of-distribution samples that don’t reflect realistic data relationships—leading to unreliable and often misleading importance scores. As warned by Hooker et al. (2021), “unrestricted permutation forces extrapolation.”

This talk introduces a conditional subgroup approach for computing model-agnostic feature importance that respects feature dependencies through row and column blocking strategies. The method combines two complementary Model-X techniques that model the joint feature distribution:

1. Conditional Imputation: Using Gaussian Copula and other statistical models to replace masked features while preserving the joint feature distribution, avoiding impossible feature combinations.
2. Restricted Permutations: Partitioning samples into blocks using Random Trees Embedding, then permuting features only within similar samples to maintain local feature dependencies.

The approach uses Fraction of Variance Unexplained (FVU) as a variance-based sensitivity measure with well-defined bounds [0,1], making it comparable across problems. Unlike SHAP or standard permutation importance, this method correctly handles multicollinear features without requiring model retraining or manual feature dropping.