Text data entity recognition based on muti-head convolution residual connections
Liu Wei, Li Bo, Yang Siyao
School of Information Science and Engineering, Shenyang University of Technology
Abstract： To construct a relational database for text data in work reports, and address the problem of extracting useful information entities from unstructured text and feature loss in traditional networks during information extraction, a deep learning-based entity recognition model, which is named RoBERTa-MCR-BiGRU-CRF is proposed. The model firstly uses the pre-trained model Robustly Optimized BERT Pretraining Approach (RoBERTa) as an encoder, feeding the trained word embeddings into the Multi-head Convolutional Residual network (MCR) layer to enrich semantic information. Next, the embeddings are input into a gated recurrent Bidirectional Gated Recurrent Unit (BiGRU) layer to further capture contextual features. Finally, a Conditional Random Field (CRF) layer is used for decoding and label prediction. Experimental results show that the model achieves an F1 score of 96.64% on the work report dataset, outperforming other comparative models. Additionally, for named entity categories in the data, the F1 score is 3.18% and 2.87% higher than BERT-BiLSTM-CRF and RoBERTa-BiGRU-CRF, respectively. The results demonstrate the model′s effectiveness in extracting useful information from unstructured text.
Key words : deep learning; named entity recognition; neural networks; data mining
