Research on Prediction of Sentiment Trend of Food Safety Public Opinion

Chaofan Jiang, Hu Wang, Changbin Jiang, Di Li

Management School, Wuhan University of Technology, Wuhan, China.

DOI: 10.4236/jcc.2023.113014                

Abstract

Emotion has a nearly decisive role in behavior, which will directly affect netizens’ views on food safety public opinion events, thereby affecting the development direction of public opinion on the event, and it is of great significance for food safety network public opinion to predict emotional trends to do a good job in food safety network public opinion guidance. In this paper, the dynamic text representation method XLNet is used to generate word vectors with context-dependent dependencies to distribute the text information of food safety network public opinion. Then, the word vector is input into the CNN-BiLSTM network for local semantic feature and context semantic extraction. The attention mechanism is introduced to give different weights according to the importance of features, and the emotional tendency analysis is carried out. Based on sentiment analysis, sentiment value time series data is obtained, and a time series model is constructed to predict sentiment trends. The sentiment analysis model proposed in this paper can well classify the sentiment of food safety network public opinion, and the time series model has a good effect on the prediction of food safety network public opinion sentiment trend.

Keywords

Network Public Opinion, Sentiment Analysis, Time Series Prediction, XLNet

1. Introduction

Emotional analysis is the process of analyzing, processing and extracting emotional subjective text using natural language processing and text analysis technology [1]. According to the type of algorithm used, there are three main methods of emotion analysis: emotion dictionary based method, machine learning based method and deep learning based method. The method based on emotion dictionary is mainly based on the construction of emotion dictionary, combining the comprehensive calculation rules of emotion words, degree adverbs, negative words, etc., to achieve the classification of text emotion [2]. Due to the high dependence on the creation of emotion dictionary, but the artificial emotion dictionary has the problem of incomplete emotion words, unable to identify the emotion tendency in different contexts, and unable to add new words in time, the expansion of emotion dictionary needs a lot of manpower and the system mobility is poor, so its robustness is not ideal [3]. The method based on machine learning mainly uses the text analysis technology to extract the relevant language features in the text, and uses the traditional machine learning method to treat the emotion analysis as a classification problem. Wawre [4] and others applied three classifiers to the emotional dichotomy of film reviews, providing ideas for the analysis of public sentiment. Abbasi [5] proposed a rule-based multivariate text feature extraction method, which considers the connection between semantic information and grammatical features, and can effectively remove the noise information and redundant features of the text. Dragoni [6] first extracts emotional words according to the emotional dictionary, and then combines these emotional words with SVM models in machine learning, and performs sentiment classification after extracting text features based on machine learning methods. Machine learning methods to achieve good classification effects require a lot of manual energy consumption, and the applicable domain of text is limited, that is, the feature set in one domain may not be adapted to another domain, and the portability of the model is poor. Based on the deep learning method, a network model is created with the function of automatic extraction of feature text classification, which can solve the shortcomings of the first two methods. Y. Kim [7] proposed to use CNN for text emotion analysis for the first time and obtained good results. Bahdanau [8] introduced the attention mechanism into the text translation task, and improved the effect of the text translation task by giving different weights to different features of the Chinese word vector of the text translation task. Deep learning has achieved remarkable results in various NLP tasks [9]. This method has become a very popular method in recent years because of its high performance. As many deep learning methods in natural language processing use word embedding learning algorithm to represent the vector of each word or sentence [10], word embedding method refers to the word vector obtained by using unsupervised learning expectation to convert text information into serialized vector, and then apply it to downstream problems. Google’s brain introduces the XLNet model, which uses the autoregressive language model and the replacement language model to solve the defects of the AR model. The AR model is a two-way model that cannot capture context information, and has achieved the best results of 18 NLP tasks [11]. Alshahrani [12] identified the optimism and pessimism in Twitter messages through finetune emotion analysis XLNet model, and achieved significant results. Sweidan [13] used XLNet in combination with the BiLSTM network for the aspect emotion classification in online user reviews, improving the overall accuracy of emotion classification. Use XLNet dynamic text representation instead of traditional static text representation to perform the initialization of downstream tasks. It can study extensive and in-depth text information at the same time. It can improve the accuracy of emotional analysis.

The emotional trend prediction studied in this paper mainly refers to the prediction and analysis of the changes in emotional attitudes contained in public comments with the development of a hot topic of a food safety event. Traditional time series prediction methods are mainly based on autoregressive (AR), autoregressive moving average (ARMA) and differential moving average autoregressive model [14] [15] [16]. In terms of the control of online public opinion, Ren Juwei [17] used the emotional time series analysis method to predict the social emotional trend of microblog events and analyze the emotional change trend of users on microblog topics in different time periods. Sun Jiaqi [18] used the Shanghai Stock Exchange Index Bar of Oriental Fortune as the data source to build investor comment data sets, build sentiment analysis models, propose time series prediction models, predict investor sentiment trends, and provide reference for investment decisions by judging the changes of investor sentiment trends.

2. Sentiment Analysis Model Based on XLNet and CNN-BiLSTM

2.1. Overall Structure of Algorithm

The XLNet-CNN-BiLSTM-Att model consists of four main components, namely the XLNet layer, CNN layer, BiLSTM layer, and Attention layer. The model architecture is shown in Figure 1. The overall process of the model is as follows. First, the XLNet layer is used to obtain word vectors containing contextual semantic information. Second, then local semantic features are extracted using CNN. Then, BiLSTM extracts contextually relevant features. Finally, the attention mechanism is introduced to assign weights to the extracted features to highlight the key information. The model utilizes a softmax classifier for sentiment classification.

XLNet layer: The XLNet layer is used to vectorize text. The text sequence $X(X_1,X_2,\dots ,X_n)$ with input length n is preprocessed, and the word vector corresponding to each word $g(g_1^{(2)},g_2^{(2)},\dots ,g_n^{(2)})$ is learned through the XLNet model. In which the word vector g encodes its context information and its own position information. The word vectors learned by the XLNet layer will be used as input data for the CNN layer.

CNN layer: The purpose of the CNN layer is to learn the local features of a text. It inputs the word vector calculated by the XLNet layer into the convolution layer and completes the feature extraction of the input text through three filters. The local feature matrix C_j obtained after the convolution layer is input into the pooling layer, and down-sampling is performed by the MaxPooling technique to obtain the optimal solution of the local value.

Figure 1. XLNet-CNN-BiLSTM architecture.

BiLSTM layer: The BiLSTM layer aims to learn the contextual information features of words in sequence data. The feature vector calculated by the CNN layer is input into the hidden layer of BiLSTM. A standard LSTM computes a sequence of hidden layer vectors from one direction, while a bidirectional LSTM computes from two different directions and finally combines the results from both directions for output. The forward hidden layer vector is $h\to$ , the backward hidden layer vector is $h\leftarrow$ , and the output vector of the hidden layer at the moment is $v_i=h\leftarrow +h\to$ .

Attention layer: The Attention layer is to assign different weights to the feature vectors. The Attention mechanism assigns different weights to the feature vectors selected by the BiLSTM layer according to different degrees of influence a (a₁, a₂, …, a₃), so as to improve the attention of the model to important features. Finally, the text sentiment is calculated by the softmax activation function.

2.2. Experimental Design and Result Analysis

2.2.1. Datasets

In June 2020, four departments, including the State Administration for Market Regulation and the Ministry of Education of the People’s Republic of China, jointly issued the “Campus Food Safety Protection Action Plan (2020-2022)”, which clearly proposes to strictly prevent and control campus food safety risks and prevent the occurrence of major food safety accidents. Based on the background, this paper uses Python and XPath technology to customize a web crawler to crawl a totally of 86,670 related blog posts from several food safety events, including “Mouldy and rotten meat found in a primary school canteen of Chengdu”, “Dead rats found in school meals in Xinzhou”, “Food poisoned in the school canteen in Bazhou”, and “Handan school students have diarrhea”. After preprocessing such as deleting duplicates and objective invalid remarks, 41,420 annotated Weibo data with balanced positive and negative sentiments are finally obtained as training data. At the same time, we crawled three other events, including “Mass vomiting of students in Henan”, “The kitchen of one Tianjin school is untidy”, and “Students vomited after dinner in Anhui”, that broke out on Weibo and had a wide range of effects, as test data. To ensure the validity of the data, the text content was first classified by the Baidu Sentiment Analysis API. Second, we manually annotated the text content and classified it as positive or negative. Finally, a more accurate positive and negative corpus is obtained by comparing and correcting the annotations obtained by the manual and the Baidu API. Table 1 shows the descriptive statistic of test datasets.

2.2.2. Experimental Setup

This study uses the jieba package in python to segment the training data and download the base version of the Chinese XLNet pre-training model released by the Harbin Institute of Technology Xunfei Joint Laboratory. The XLNet model that had been pre-trained on the general-purpose dataset was further pre-trained on our training dataset. The parameters of the XLNet model were fine-tuned using CNN, BiLSTM, and attention mechanisms, which is equivalent to transferring the model from the general-purpose domain to food safety sentiment analysis. After obtaining the word vectors generated by XLNet pre-training model, the microblog sentiment analysis model is trained in the environment with the deep learning framework Tensorflow 2.0.0. A grid search method is used to adjust the main parameters and obtain the ideal set of parameters. The values of the model parameters are shown in Table 2.

2.3. Experimental Evaluation Indicators

The common metrics used to evaluate the performance of sentiment analysis models are Precision (P), Recall (R), F1-score (F₁), Accuracy (Acc) and Loss rate (L).

$Acc=\frac{T_P+T_N}{T_P+T_N+F_P+F_N}\times 100\%$ $P=\frac{T_P}{T_P+F_P}\times 100\%$ (1)

$R=\frac{T_P}{T_P+F_N}\times 100\%$ $F_1=\frac{2\times P\times R}{P\times R}\times 100\%$ $L=-\left[\log \hat{y}+(1-y)\log (1-\hat{y})\right]$ (2)

Table 1. Test event data.

Table 2. Model parameters.

2.4. Experimental Results

In this experiment, three models were chosen as experimental control models: Word2Vec-BiLSTM, XLNet and XLNet-CNN. The sentiment analysis experiments were conducted on three test events, including Henan event, Tianjin event, and Anhui event. Figure 2 depicts the dynamic changes in the accuracy and loss rate of the model training over the 10 epochs. According to the accuracy and loss rates of the validation set data during the training phase, we may conclude that the model proposed in this paper outperforms the other five models.

In this study, the optimal parameter values of XLNet-CNN-BiLSTM were utilized to classify Internet users’ sentiments, and the results are presented in Table 3. The average accuracy of the three test events is 94.12%, with the Anhui event having the highest accuracy at 94.75%. This demonstrates that the XLNet-CNN-BiLSTM model proposed in this research is capable of achieving outstanding performance in Internet public opinion sentiment analysis. Besides, the model outperformed the other five models in terms of accuracy in all three events tested. This implies that the dynamic text representation approach is superior to the static text representation method Word2Vec for classification tasks.

3. Prediction of Public Sentiment Trend of Food Safety Network Based on LSTM Network

3.1. Construction of Emotional Value Sequence of Food Safety Network Public Opinion

In this paper, the emotional value of online public opinion text on food safety is constructed in the same way as the bullish index used by Antweiler in the stock market, which is used to reflect the emotional trend of netizens in public opinion events over a period of time, and can integrate emotional information. The calculation method of the bullish index combines the bullish and bearish sentiment of the investors in the stock market. It is a classic calculation method in the

Figure 2. Variation of accuracy (left) and loss (right) of the model during training.

Table 3. Models test result.

measurement of investor sentiment in the stock market. It is similar to the positive and negative emotional tendencies of the Internet users of the food safety network public opinion on public opinion events. Therefore, this method is used for reference to construct the emotional value of the food safety network public opinion in this paper. Referring to the bullish index formula, the emotional value calculation formula is $S_t=\ln [(1+n_t^{pos})/(1+n_t^{neg})]$ , $n_t^{pos}$ indicates the number of blog posts with positive emotions in the food safety network public opinion event in a certain period, $n_t^{neg}$ indicates the number of posts with negative emotions in a certain period.

In order to use the time series model to predict the emotional trend more accurately, the emotional value calculated for each blog article is stored in chronological order. The emotional value data of “Henan Event”, “Anhui Event” and “Tianjin Event” are selected as the data source of the emotional trend prediction model, Take the average of the emotional values of the online public opinion blog posts corresponding to these food safety events every 30 minutes and give the emotional value sequence $\{S_1,S_2,\dots ,S_t\}$ that constitutes the corresponding time point. The emotional value time series is divided into training sets and test sets. A set of training samples is used to train LSTM to generate memory, and then the prediction set is input into the training LSTM to obtain the output of the prediction set.

3.2. Wavelet Analysis

Because the emotional tendency of blog posts is easily affected by many factors such as opinion leaders, Internet users’ relevance and the nature of events, it usually contains a lot of noise, so the constructed temporal emotional value also contains noise. The wavelet analysis can refine the signal analysis in many degrees through the operation functions such as expansion and translation. It can eliminate the noise in the time series data, but also cannot change the characteristics of the original time series data. Therefore, this paper uses wavelet denoising to eliminate the noise components in the time series data of the emotional value of the public opinion of the food safety network, reduce the interference impact of noise on the LSTM model, and improve the prediction function of the model. Wavelet analysis can decompose each layer of input data into a low frequency signal and a high frequency signal, and can further decompose the low frequency signal into a low frequency signal and a high frequency signal, and so on.

3.3. LSTM Time Series Prediction Method

The LSTM model can mine the information in the long time series data. Compared with the problem of gradient disappearance or explosion that may occur in RNN, LSTM has effectively solved the problem of long-term dependence and gradient disappearance by deliberately improving the internal structure of RNN and adding three gating units, namely input gate, forgetting gate and output gate. The purpose of this paper is to predict the emotional trend of food safety network public opinion through a neural network model. Considering the impact of historical emotional value time series data on the emotional trend, LSTM can effectively use historical information and has advantages in processing long-term dependent data, and build an emotional value time series prediction model based on LSTM model.

3.4. Experimental Design and Result Analysis

3.4.1. Experiment Description and Parameter Setting

1) Wavelet analysis

The db3 wavelet base is used to decompose the time series data of the public opinion sentiment value of the food safety network by three layers of wavelet, and reconstruct that the public opinion sentiment value of the food safety network is the time series data, so as to improve the extrapolation ability of the subsequent LSTM model. The emotional value time series data of the three original food safety network public opinion events and the reconstructed emotional value time series data are shown in Figures 3-5. From the original emotional value time series data chart of the three events and the reconstructed emotional value time series data chart, it can be seen that the emotional value time series data of the food safety network public opinion after the db3 wavelet decomposition and reconstruction can effectively raise the original data noise and retain its approximate signal to the maximum extent, Therefore, it is theoretically scientific to establish LSTM prediction model based on the reconstructed emotional value time series data.

2) Parameter description

Tensorflow is currently the most popular open source machine learning framework based on Python development language. This paper uses the grid search method to find the main parameters of the LSTM model, and obtains the optimal parameter set of the model as shown in Table 4.

Figure 3. Henan event raw sentiment value time series data (left), reconstructed data (right).

Figure 4. Tianjin event raw sentiment value time series data (left), reconstructed data (right).

Figure 5. Anhui event raw sentiment value time series data (left), reconstructed data (right).

Table 4. Parameters table.

3.4.2. Experimental Evaluation Indicators

This paper adopts the Evaluation indicators commonly used in the prediction model as the way to measure the error between the predicted value and the real value, including the average absolute percentage error (MAPE), the average absolute error (MAE), and the root mean square error (RMSE).

3.4.3. Experimental Results and Discussion

When designing the comparative experiment, the previous experience was used for reference. This experiment uses time series data to train the trend model of public sentiment. Therefore, this paper designs three experimental schemes based on LSTM neural network model for prediction and comparison: 1) LSTM Shenjing network model; 2) BP neural network model; 3) The random forest (RF) model draws the comparison chart of the actual value and predicted value of the above three experimental schemes for three public opinion events, as shown in Figure 6.

The ordinate in the figure above represents the processed emotional value, the abscissa represents the amount of data, the purple line is the actual value, and the other lines are the predicted values of each model. From the figure, we can clearly see the fitting degree of the actual value and the predicted value of the three schemes. The prediction effect of RF model is general, and there are many places with large errors; The fitting effect of BP model is good, but it is not accurate enough; The LSTM model can reach a relatively accurate degree, and it can be seen from the image that only local small errors exist. In this paper, MAPE, MAE and RMSE are selected as the evaluation criteria for the model performance. The model performance evaluation of the three experimental schemes designed is listed in Table 5.

Table 5. Model performance evaluation table.

Figure 6. Comparison of predicted values of three models: Henan incident, Tianjin incident and Anhui incident.

According to the comparison results in Table 5, the values of the three evaluation criteria of LSTM model are significantly different from those of BPNN model and RF model. This shows that the prediction effect of emotional value time series data in LSTM prediction model is better than that of BPNN and RF models.

4. Conclusion

Aiming at the problem of traditional word vector model, a classification algorithm of public sentiment on food safety network based on XLNet model is proposed. Taking full advantage of the advantage that XLNet model can represent Chinese text at the character level and the accuracy is not affected by the word segmentation effect, XLNet model is used to convert the word vector of text information. The structure of CNN-BLSTM emotion classification algorithm based on XLNet model proposed in this paper is shown in detail, including the introduction of each layer of the network and the training of the model. By setting up contrast experiments, the effectiveness of the proposed method is proved, and a relatively ideal effect of emotion classification is obtained. Based on the results of emotion classification, the emotional value of fixed time unit is calculated and the emotional value time series is constructed. Based on the advantages of LSTM model in time series data, LSTM model is proposed to predict emotional trend, and good prediction results are achieved.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Pang, B. and Lee, L. (2008) Opinion Mining and Sentiment Analysis. Foundations and Trends? in Information Retrieval, 2, 1-135. https://doi.org/10.1561/1500000011
[2]	Xu, L.H. and Lin, H.F. (2008) Constructing the Affective Lexicon Ontol-ogy. Journal of the China Society for Scientific and Technical Information, 27, 180-185.
[3]	Shi, W. and Wang, H.W. (2013) Sentiment Analysis of Chinese Online Reviews Based on Semantics. Journal of the China Society for Scientific and Technical Information, 32, 860-867.
[4]	Wawre, S.V. and Deshmukh, S.N. (2016) Sentiment Classification Using Machine Learning Techniques. International Journal of Science and Research (IJSR), 5, 819-821. https://doi.org/10.21275/v5i4.NOV162724
[5]	Abbasi, A., France, S., Zhang, Z., et al. (2010) Selecting Attributes for Sentiment Classification Using Feature Relation Networks. IEEE Transactions on Knowledge and Data Engineering, 23, 447-462. https://doi.org/10.1109/TKDE.2010.110
[6]	Dragoni, M. and Petrucci, G. (2018) A Fuzzy-Based Strategy for Multi-Domain Sentiment Analysis. International Journal of Approximate Reasoning, 93, 59-73. https://doi.org/10.1016/j.ijar.2017.10.021
[7]	Kim, Y. (2014) Convolutional Neural Networks for Sentence Classification. 1746-1751. https://doi.org/10.3115/v1/D14-1181
[8]	Bahdanau, D., Cho, K. and Bengio, Y. (2014) Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:1409.0473.
[9]	Cliche, M. (2017) BB_twtr at SemEval-2017 Task 4: Twitter Sentiment Analysis with CNNs and LSTMs. arXiv preprint arXiv:1704.06125. https://doi.org/10.18653/v1/S17-2094
[10]	Yadav, A. and Vishwakarma, D.K. (2020) Sentiment Analysis Using Deep Learning Architectures: A Review. Artificial Intelligence Review, 53, 4335-4385. https://doi.org/10.1007/s10462-019-09794-5
[11]	Yang, Z., Dai, Z., Yang, Y., et al. (2019) Xlnet: Generalized Autoregressive Pretraining for Language Understanding. Advances in Neural Information Processing Systems, 32.
[12]	Alshahrani, A., Ghaffari, M., Amir, K., et al. (2020) Identifying Optimism and Pessimism in Twitter Messages Using Xlnet and Deep Consensus. 2020 International Joint Conference on Neural Networks (IJCNN), 1-8. https://doi.org/10.1109/IJCNN48605.2020.9206948
[13]	Sweidan, A.H., El-Bendary, N. and Al-Feel, H. (2021) Sentence-Level Aspect-Based Sentiment Analysis for Classifying Adverse Drug Reactions (ADRs) Using Hybrid Ontology-XLNet Transfer Learning. IEEE Access, 9, 90828-90846. https://doi.org/10.1109/ACCESS.2021.3091394
[14]	Chisci, L., Mavino, A., Perferi, G., et al. (2010) Real-Time Epileptic Seizure Prediction Using AR Models and Support Vector Machines. IEEE Transactions on Biomedical Engineering, 57, 1124-1132. https://doi.org/10.1109/TBME.2009.2038990
[15]	Faruk, D.?. (2010) A Hybrid Neural Network and ARIMA Model for Water Quality Time Series Prediction. Engineering Applications of Artificial Intelligence, 23, 586-594. https://doi.org/10.1016/j.engappai.2009.09.015
[16]	Rojas, I., Valenzuela, O., Rojas, F., et al. (2008) Soft-Computing Techniques and ARMA Model for Time Series Prediction. Neurocomputing, 71, 519-537. https://doi.org/10.1016/j.neucom.2007.07.018
[17]	Ren, J.W., Yang, L., Wu, X.F., et al. (2017) Public Sentiment Trend Prediction of Microblog Events Based on Affective Commonsense Knowledge. Journal of Chinese Information Processing, 31, 169-178.
[18]	Sun, J.Q., Wang, X., Yang, P., et al. (2021) Predicting Sentiment Trend by Time Series Model and Sentiment Analysis. Computer Engineering and Design, 42, 2938-2945.