Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review
Towards the Impact of Pruning Criteria and Tuning Approach on Model Interpretability and Quality. / Shcherbin, Andrey.
International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM. IEEE Computer Society, 2024. p. 1800-1805 (International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review
}
TY - GEN
T1 - Towards the Impact of Pruning Criteria and Tuning Approach on Model Interpretability and Quality
AU - Shcherbin, Andrey
N1 - Conference code: 25
PY - 2024
Y1 - 2024
N2 - Interpretation of deep neural networks in terms of human intuition is one of the most important research fields. To improve neural networks applicability, we need to interpret their predictions. Interpretation can help us understand the causes of some decisions and increase the quality of predictions. Feature attribution methods describe predictions using input data. They are applicable for any type of model and can help to improve the quality of model on some corner cases by modification of the training dataset. Neural networks are used for people identification tasks, such as face and voice recognition, so we process a lot of personal data with such models. To ensure privacy we need to run models on some low power devices: smartphones or video cameras. But neural networks architectures can be computationally expensive for such devices. So, we need to optimize model architectures for edge devices and distill knowledge from original model to optimized. In this work we focus on measurement of interpretability and tuning approaches to understand the impact of model architecture on its interpretability and quality. We use pruning as an optimization algorithm and Integrated gradients as an interpretation method. We check the impact of pruning criteria on model interpretability for ResNet50 and MobileNetV2 model architectures on ImageNet2012 dataset.
AB - Interpretation of deep neural networks in terms of human intuition is one of the most important research fields. To improve neural networks applicability, we need to interpret their predictions. Interpretation can help us understand the causes of some decisions and increase the quality of predictions. Feature attribution methods describe predictions using input data. They are applicable for any type of model and can help to improve the quality of model on some corner cases by modification of the training dataset. Neural networks are used for people identification tasks, such as face and voice recognition, so we process a lot of personal data with such models. To ensure privacy we need to run models on some low power devices: smartphones or video cameras. But neural networks architectures can be computationally expensive for such devices. So, we need to optimize model architectures for edge devices and distill knowledge from original model to optimized. In this work we focus on measurement of interpretability and tuning approaches to understand the impact of model architecture on its interpretability and quality. We use pruning as an optimization algorithm and Integrated gradients as an interpretation method. We check the impact of pruning criteria on model interpretability for ResNet50 and MobileNetV2 model architectures on ImageNet2012 dataset.
KW - deep neural networks
KW - feature attribution
KW - interpretability
KW - interpretability metric
KW - model tuning
KW - pruning
UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85201936775&origin=inward&txGid=211fb62d662ba1a60b128fe16e8d0d15
UR - https://www.mendeley.com/catalogue/5ad90424-a3be-33b8-8834-f8ba57632644/
U2 - 10.1109/EDM61683.2024.10615139
DO - 10.1109/EDM61683.2024.10615139
M3 - Conference contribution
SN - 9798350389234
T3 - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM
SP - 1800
EP - 1805
BT - International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices, EDM
PB - IEEE Computer Society
T2 - 25th IEEE International Conference of Young Professionals in Electron Devices and Materials
Y2 - 28 June 2024 through 2 July 2024
ER -
ID: 60549721