TY - GEN

T1 - On Practical Approach to Uniform Quantization of Non-redundant Neural Networks

AU - Goncharenko, Alexander

AU - Denisov, Andrey

AU - Alyamkin, Sergey

AU - Terentev, Evgeny

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The neural network quantization is highly desired procedure to perform before running the neural networks on mobile devices. Quantization without fine-tuning leads to accuracy drop of the model, whereas commonly used training with quantization is done on the full set of the labeled data and therefore is both time- and resource-consuming. Real life applications require simplification and acceleration of the quantization procedure that will maintain the accuracy of full-precision neural network, especially for modern mobile neural network architectures like Mobilenet-v1, MobileNet-v2 and MNAS. Here we present two methods to significantly optimize the training with the quantization procedure. The first one is introducing the trained scale factors for discretization thresholds that are separate for each filter. The second one is based on mutual rescaling of consequent depth-wise separable convolution and convolution layers. Using the proposed techniques, we quantize the modern mobile architectures of neural networks with the set of train data of only ∼ 10% of the total ImageNet 2012 sample. Such reduction of the train dataset size and a small number of trainable parameters allow to fine-tune the network for several hours while maintaining the high accuracy of the quantized model (the accuracy drop was less than 0.5%). The ready-for-use models and code are available at: https://github.com/agoncharenko1992/FAT-fast-adjustable-threshold.

AB - The neural network quantization is highly desired procedure to perform before running the neural networks on mobile devices. Quantization without fine-tuning leads to accuracy drop of the model, whereas commonly used training with quantization is done on the full set of the labeled data and therefore is both time- and resource-consuming. Real life applications require simplification and acceleration of the quantization procedure that will maintain the accuracy of full-precision neural network, especially for modern mobile neural network architectures like Mobilenet-v1, MobileNet-v2 and MNAS. Here we present two methods to significantly optimize the training with the quantization procedure. The first one is introducing the trained scale factors for discretization thresholds that are separate for each filter. The second one is based on mutual rescaling of consequent depth-wise separable convolution and convolution layers. Using the proposed techniques, we quantize the modern mobile architectures of neural networks with the set of train data of only ∼ 10% of the total ImageNet 2012 sample. Such reduction of the train dataset size and a small number of trainable parameters allow to fine-tune the network for several hours while maintaining the high accuracy of the quantized model (the accuracy drop was less than 0.5%). The ready-for-use models and code are available at: https://github.com/agoncharenko1992/FAT-fast-adjustable-threshold.

KW - Distillation

KW - Machine learning

KW - Neural networks

KW - Quantization

UR - http://www.scopus.com/inward/record.url?scp=85072865641&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-30484-3_29

DO - 10.1007/978-3-030-30484-3_29

M3 - Conference contribution

AN - SCOPUS:85072865641

SN - 9783030304836

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 349

EP - 360

BT - Artificial Neural Networks and Machine Learning – ICANN 2019

A2 - Tetko, Igor V.

A2 - Karpov, Pavel

A2 - Theis, Fabian

A2 - Kurková, Vera

PB - Springer-Verlag GmbH and Co. KG

T2 - 28th International Conference on Artificial Neural Networks, ICANN 2019

Y2 - 17 September 2019 through 19 September 2019

ER -