With the rapid development of complex machine learning models, there is uncertainty on how these models truly work. Their black-box nature restricts experts from evaluating models solely on standard numerical metrics, which may result in a model performing seemingly well on a dataset but for the wrong reasons. It is particularly critical to have transparency in medical image analysis due to the importance those decisions have. This is because health care practitioners are hesitant to trust machines when there is no clear reason for the machine to use the appropriate logical steps for a decision, and most importantly, human life is at stake. Regulatory bodies, such as the U.S. Federal and Drug Administration, have recently started pushing guidelines to make AI medical devices more transparent so that they are safer to use.
To address this challenge, many researchers have proposed interpretability methods to generate explanations for the behavior of machine learning models. One popular interpretability method is Local Interpretable Model-agnostic Explanations (LIME), which can explain any black-box model trained on any type of data. Although LIME is a powerful tool, the explanations generated by it depend on how the hyper-parameters of the algorithm are set. If little care is given to the hyperparameters, the resulting explanations can be meaningless.
In this thesis, we first address the impact of a core component in LIME – weights. The weights impact the quality and variability of explanations generated by LIME. Since weights are determined by the distance metric, we compare the explanations of LIME for three cases; when weights are calculated using (1) an unnormalized distance metric, (2) a normalized distance metric (LIME authors default), and (3) set uniformly to a value of one. Through theoretical analysis and experimentation with different data sets and models, we demonstrate that using the unnormalized distance metric results in poor explanations, whereas the normalized metric and uniform weights give comparable high-quality explanations. We, therefore, propose ULIME (Uniform LIME), a variant of LIME that forgoes the weighted notion of locality by setting all weights to one. Our motivation behind ULIME is the simplification of the existing LIME algorithm by removing the weighting step, and consequently, removal of the distance metric hyper-parameter that is strongly coupled with explanation quality.
Secondly, we address the impact of superpixelization techniques on explanation intuitiveness – a method used to partition the image for explanation purposes. We develop a domain-specific superpixelization method that aids in generating more intuitive explanations and emphasize our results on two medical imaging datasets. Although LIME can be configured for any type of input and model, for this thesis, we restrict our investigation to image classifiers.
Our contributions aim to improve the current state-of-the-art LIME for image interpretability. We simplify LIME by removing the weighting step, reducing the possibility of error due to the distance hyper-parameter, and introducing the concept of domain-specific superpixelization verified by a radiologist. Taken together, our approach improves the quality of explanations, and, thereby, provides a mechanism for physicians and patients to place greater trust in the results of ML models.