SmoothHess: ReLU network feature interactions via Stein's Lemma Conference Paper

  


Authors: Torop, M.; Masoomi, A.; Hill, D.; Kose, K.; Ioannidis, S.; Dy, J.
Title: SmoothHess: ReLU network feature interactions via Stein's Lemma
Conference Title: 2023 Conference on Neural Information Processing Systems (NeurIPS)
Abstract: Several recent methods for interpretability model feature interactions by looking at the Hessian of a neural network. This poses a challenge for ReLU networks, which are piecewise-linear and thus have a zero Hessian almost everywhere. We propose SmoothHess, a method of estimating second-order interactions through Stein's Lemma. In particular, we estimate the Hessian of the network convolved with a Gaussian through an efficient sampling algorithm, requiring only network gradient calls. SmoothHess is applied post-hoc, requires no modifications to the ReLU network architecture, and the extent of smoothing can be controlled explicitly. We provide a non-asymptotic bound on the sample complexity of our estimation procedure. We validate the superior ability of SmoothHess to capture interactions on benchmark datasets and a real-world medical spirometry dataset.
Journal Title Advances in Neural Information Processing Systems
Volume: 36
Conference Dates: 2023 Dec 10-16
Conference Location: New Orleans, LA
ISBN: 1049-5258
Publisher: Neural Information Processing Systems Foundation  
Date Published: 2023-01-01
Language: English
ACCESSION: WOS:001229751904020
PROVIDER: wos
Notes: Proceedings Paper -- 37th Conference on Neural Information Processing Systems (NeurIPS) -- DEC 10-16, 2023 -- New Orleans, LA -- Source: Wos