Abstract
Part-prototype networks have recently become methods of interest as an interpretable alternative to many of the current black-box image classifiers. However, the interpretability of these methods from the perspective of human users has not been sufficiently explored. In addition, previous works have had major issues with following proper experiment design and task representation that limit their reliability and validity. In this work, we have devised a framework for evaluating the interpretability of part-prototype-based models from a human perspective that solves these issues. The proposed framework consists of three actionable metrics and experiments. The results of these experiments will reveal important and reliable interpretability related properties of such models. To demonstrate the usefulness of our framework, we performed an extensive set of experiments using Amazon Mechanical Turk. They not only show the capability of our framework in assessing the interpretability of various part-prototype-based models, but they also are, to the best of our knowledge, the most comprehensive work on evaluating such methods in a unified framework.