Результаты исследований: Научные публикации в периодических изданиях › статья › Рецензирование
Contrastive fine-tuning to improve generalization in deep NER. / Bondarenko, Ivan .
в: Komp'juternaja Lingvistika i Intellektual'nye Tehnologii, № 21, 8, 15.06.2022, стр. 70-80.Результаты исследований: Научные публикации в периодических изданиях › статья › Рецензирование
}
TY - JOUR
T1 - Contrastive fine-tuning to improve generalization in deep NER
AU - Bondarenko, Ivan
N1 - The work is supported by the Mathematical Center in Akademgorodok under the agreement No. 075-15-2022—282 with the Ministry of Science and Higher Education of the Russian Federation.
PY - 2022/6/15
Y1 - 2022/6/15
N2 - A novel algorithm of two-stage fine-tuning of a BERT-based language model for more effective named entity recognition is proposed. The first stage is based on training BERT as a Siamese network using a special contrastive loss function, and the second stage consists of fine-tuning the NER as a "traditional" sequence tagger. Inclusion of the contrastive first stage makes it possible to construct a high-level feature space at the output of BERT with more compact representations of different named entity classes. Experiments have shown that this fine-tuning scheme improves the generalization ability of named entity recognition models fine tuned from various pre-trained BERT models. The source code is available under an Apache 2.0 license and hosted on GitHub https://github.com/bond005/runne_contrastive_ner
AB - A novel algorithm of two-stage fine-tuning of a BERT-based language model for more effective named entity recognition is proposed. The first stage is based on training BERT as a Siamese network using a special contrastive loss function, and the second stage consists of fine-tuning the NER as a "traditional" sequence tagger. Inclusion of the contrastive first stage makes it possible to construct a high-level feature space at the output of BERT with more compact representations of different named entity classes. Experiments have shown that this fine-tuning scheme improves the generalization ability of named entity recognition models fine tuned from various pre-trained BERT models. The source code is available under an Apache 2.0 license and hosted on GitHub https://github.com/bond005/runne_contrastive_ner
KW - распознавание именованных сущностей
KW - сопоставительное обучение
KW - Сиамские нейронные сети
KW - BERT
KW - NER
KW - named entity recognition
KW - contrastive learning
KW - Siamese neural networks
KW - BERT
KW - NER
UR - https://www.dialog-21.ru/media/5847/_-dialog2022scopus.pdf
UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85148050734&origin=inward&txGid=c1a2f8910ed0346faabe1a4929c6b7e2
UR - https://www.mendeley.com/catalogue/53c2698e-400d-312c-945d-b0e44dece9ed/
U2 - 10.28995/2075-7182-2022-21-70-80
DO - 10.28995/2075-7182-2022-21-70-80
M3 - Article
SP - 70
EP - 80
JO - Компьютерная лингвистика и интеллектуальные технологии
JF - Компьютерная лингвистика и интеллектуальные технологии
SN - 2221-7932
IS - 21
M1 - 8
ER -
ID: 36334160