Abstract
Automatic term extraction using deep learning models has shown promising results when sufficient training data is available. Unfortunately, in certain emerging domains and low-resource languages such data may be lacking, leading to reduced performance due to insufficient fine-tuning of these models. To address this issue, in this study we propose a multilingual transformer-based sequence labeling model for term extraction, trained on an original dataset in English and Kazakh. Our evaluation results demonstrate that the multilingual XLM-RoBERTa model achieves improvements in automatic term extraction compared to a model with BERT-BiLSTM-CRF architecture trained on limited training data. Although the degree of improvement varies across domains, languages and dataset sizes, our evaluation shows that in Kazakh subsets of the dataset the highest F1-score improvement reaches 20 % in the Blockchain domain, while in the domain of Materials Science the improvement amounts to 12.9 %. Furthermore, our multilingual model attains competitive results when compared to a model with popular and highly effective architecture trained on a small amount of training data.