TY - JOUR

T1 - DIFFERENTIABLE PRUNING BY CONVOLUTIONAL PROJECTION

AU - Chudakov, Dmitry Sergeevich

AU - Berikov, Vladimir Borisovich

PY - 2025

Y1 - 2025

N2 - Structured pruning is a widely used technique for reducing the computational complexity of neural networks. The standard pruning process typically consists of two stages: first, identifying and removing the least significant blocks of parameters, and second, fine-tuning the network to restore its original accuracy. However, this separation of stages often results in suboptimal outcomes. In this paper, we propose a novel convolutional neural network pruning procedure that is fully end-to-end and differentiable. Our approach involves projecting convolutional kernels into a reduced number of channels through learnable linear transformations, implemented using 1x1 convolutional layers. Specifically, for each convolutional layer to be pruned, we insert learnable convolutions both before and after the layer. The pre-layer expands the number of channels to the original size, while the post-layer reduces the number of channels. Only these auxiliary convolutions are trained, after which the learned transformations are fused with the original layer to produce a pruned convolutional kernel. The proposed method demonstrates superior accuracy compared to traditional pruning techniques, particularly when removing a substantial number of parameters.

AB - Structured pruning is a widely used technique for reducing the computational complexity of neural networks. The standard pruning process typically consists of two stages: first, identifying and removing the least significant blocks of parameters, and second, fine-tuning the network to restore its original accuracy. However, this separation of stages often results in suboptimal outcomes. In this paper, we propose a novel convolutional neural network pruning procedure that is fully end-to-end and differentiable. Our approach involves projecting convolutional kernels into a reduced number of channels through learnable linear transformations, implemented using 1x1 convolutional layers. Specifically, for each convolutional layer to be pruned, we insert learnable convolutions both before and after the layer. The pre-layer expands the number of channels to the original size, while the post-layer reduces the number of channels. Only these auxiliary convolutions are trained, after which the learned transformations are fused with the original layer to produce a pruned convolutional kernel. The proposed method demonstrates superior accuracy compared to traditional pruning techniques, particularly when removing a substantial number of parameters.

KW - Computer Vision

KW - Deep Learning

KW - Pruning

UR - https://www.scopus.com/pages/publications/105054900770

UR - https://www.mendeley.com/catalogue/b99f3f60-c72c-3819-908a-d58b68ee55d6/

U2 - 10.33048/semi.2025.22.C02

DO - 10.33048/semi.2025.22.C02

M3 - Article

VL - 22

SP - C18-C29

JO - Сибирские электронные математические известия

JF - Сибирские электронные математические известия

SN - 1813-3304

IS - 2

ER -