TY - GEN

T1 - Towards the Impact of Pruning Criteria and Tuning Approach on Model Interpretability and Quality

AU - Shcherbin, Andrey

N1 - Conference code: 25

PY - 2024

Y1 - 2024

N2 - Interpretation of deep neural networks in terms of human intuition is one of the most important research fields. To improve neural networks applicability, we need to interpret their predictions. Interpretation can help us understand the causes of some decisions and increase the quality of predictions. Feature attribution methods describe predictions using input data. They are applicable for any type of model and can help to improve the quality of model on some corner cases by modification of the training dataset. Neural networks are used for people identification tasks, such as face and voice recognition, so we process a lot of personal data with such models. To ensure privacy we need to run models on some low power devices: smartphones or video cameras. But neural networks architectures can be computationally expensive for such devices. So, we need to optimize model architectures for edge devices and distill knowledge from original model to optimized. In this work we focus on measurement of interpretability and tuning approaches to understand the impact of model architecture on its interpretability and quality. We use pruning as an optimization algorithm and Integrated gradients as an interpretation method. We check the impact of pruning criteria on model interpretability for ResNet50 and MobileNetV2 model architectures on ImageNet2012 dataset.

AB - Interpretation of deep neural networks in terms of human intuition is one of the most important research fields. To improve neural networks applicability, we need to interpret their predictions. Interpretation can help us understand the causes of some decisions and increase the quality of predictions. Feature attribution methods describe predictions using input data. They are applicable for any type of model and can help to improve the quality of model on some corner cases by modification of the training dataset. Neural networks are used for people identification tasks, such as face and voice recognition, so we process a lot of personal data with such models. To ensure privacy we need to run models on some low power devices: smartphones or video cameras. But neural networks architectures can be computationally expensive for such devices. So, we need to optimize model architectures for edge devices and distill knowledge from original model to optimized. In this work we focus on measurement of interpretability and tuning approaches to understand the impact of model architecture on its interpretability and quality. We use pruning as an optimization algorithm and Integrated gradients as an interpretation method. We check the impact of pruning criteria on model interpretability for ResNet50 and MobileNetV2 model architectures on ImageNet2012 dataset.

KW - deep neural networks

KW - feature attribution

KW - interpretability

KW - interpretability metric

KW - model tuning

KW - pruning

UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85201936775&origin=inward&txGid=211fb62d662ba1a60b128fe16e8d0d15

UR - https://www.mendeley.com/catalogue/5ad90424-a3be-33b8-8834-f8ba57632644/

U2 - 10.1109/EDM61683.2024.10615139

DO - 10.1109/EDM61683.2024.10615139

M3 - Conference contribution

SN - 9798350389234

T3 - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM

SP - 1800

EP - 1805

BT - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM

PB - IEEE Computer Society

T2 - 25th IEEE International Conference of Young Professionals in Electron Devices and Materials

Y2 - 28 June 2024 through 2 July 2024

ER -