Building a Neural Network Model to Analyze Teachers’ Satisfaction with Online Teaching during the COVID-19 Ravages

Gangxin Wen^1,2, Quanlong Guan^1,2,3*, Xiaofeng Wu^4*, Weiqi Luo^1,2,3

¹College of Information Science and Technology, Jinan University, Guangzhou, China.
²College of Cyber Security, Jinan University, Guangzhou, China.
³Guangdong Institute of Smart Education, Jinan University, Guangzhou, China.
⁴Guangzhou Polytechnic of Sports, Guangzhou, China.

DOI: 10.4236/jcc.2022.101005                

Abstract

The ravages of COVID-19 have forced schools in countries around the world to make a temporary shift from traditional, face-to-face teaching to online teaching. Are teachers in schools prepared to deal with this change? We conducted a survey in which we distributed questionnaires to primary and secondary school teachers in Guangdong Province, China, asking them about their views on various aspects of online education. We received 498,481 questionnaires back, and over 80% of teachers were satisfied with the online resources, and over 68% of teachers were satisfied with the online platform and software. Immediately afterward, we analyzed the differences between urban and rural teachers on specific issues using cross-sectional analysis and chi-square tests and built a neural network model to achieve predictions of teacher satisfaction with an accuracy of nearly 90%. Finally, we analyzed the features that influence the decisions of the neural network. This epidemic has prompted the widespread use of online learning, and the insights we gain today will be helpful in the future.

Keywords

COVID-19, Online Education, Neural Networks, Satisfaction

1. Introduction

In December 2019, a new coronavirus causes acute infectious pneumonia all across the world [1]. Many institutions have eliminated face-to-face classes and use online platforms for distance learning [2]. In February 2020, China announced its strong support for information-based teaching and large-scale online education. Schools across the country’s provinces are gradually launching online teaching through software such as Dingding, Tencent Meetings, and Zoom, and online education is seeing good growth opportunities.

The growth of online education has also brought challenges. The first challenge is the disparity between urban and rural areas. In China, the disparity between urban and rural areas in education is an old and real problem. The policy orientation has resulted in an uneven distribution of public resources, with most quality education resources concentrated in the cities [3]. The advent of internet technology has exposed more educators to the possibilities of teaching and learning in ways other than the traditional classroom [4]. Whether online education can narrow the gap in the quality of education between urban and rural China is yet to be studied.

Secondly, whether teachers and students are satisfied with the online classroom experience is critical. In previous research, many scholars have studied students’ satisfaction with their participation in online classes, while few have bothered to study teachers’ satisfaction with online classes. However, teachers play an essential role in distance learning. In addition to having the relevant knowledge, they need to design and develop interactive courseware appropriate to each new technology, organize teaching resources in a format suitable for independent learning, and assess student performance, attitudes, and perceptions at the distance site [5]. Moreover, distance learning systems were initially developed at the tertiary level and are gradually being used at the K-12 level. As students in the early grades cannot perceive changes in the learning environment, studying, teacher satisfaction can provide insight into the strengths and weaknesses of distance learning systems for students in the early grades. In addition to this, it is worth looking at whether online teaching platforms can meet the needs of educators, whether online teaching can fulfill the mission of teaching and learning, and whether online education can effectively replace traditional education during the epidemic.

Our primary target audience was teachers, who had previously been neglected in our work. We first designed a questionnaire about online education experience during the epidemic and sent it to all primary and secondary school teachers in Guangdong Province, China, after which we received nearly half a million responses. The questionnaires were collated and cleaned in preparation for the study. We analyzed the data in two ways, the first was an analysis of the differences between urban and rural teachers in the online delivery process, and the second was an analysis of teacher satisfaction.

The main methods used to analyze the differences between urban and rural teachers are cross-tabulations and chi-square tests, and explanations and recommendations are given alongside the comparative analysis. The satisfaction analysis is not a new direction, as many scholars have done similar work, but they have mainly focused on students participating in online education. They have mainly used statistical methods or traditional machine learning algorithms, and the sample sizes of their studies are relatively small. However, satisfaction is influenced by multiple factors and is a multivariate non-linear problem, and traditional statistical models have difficulty fitting them [6]. Furthermore, many additional factors have not been considered before in this epidemic. Combining these reasons and the data size, we decided to use a deep learning approach to study teachers’ satisfaction with online education.

We constructed a neural network model based on the residual structure to train teachers’ data, and in the test set, our neural network model demonstrated a high prediction accuracy. In addition to this, we believe that the decision-making process of neural networks should not be mysterious and black-box, and therefore we visualize the crucial features that influence the decisions of neural networks through Gradient-weighted Class Activation Mapping.

The components of the whole paper are as follows: Section 2 states the work related to teacher satisfaction and the variability of teachers between urban and rural areas. Section 3 describes the process of collecting and processing the questionnaire data. Section 4 uses statistical methods to compare the differences between urban and rural teachers and provides a crude analysis of the results. Section 5 describes our neural network model and conducts related experiments. Section 6 contains discussion and recommendations, and Section 7 concludes the paper and provides an outlook for future work.

2. Related Work

2.1. Online Education Satisfaction

Online learning is a network learning method with connectivity, accessibility, and flexibility [7]. For some, it offers the potential for learning for new audiences; for others, it offers the opportunity to change the dynamics of learning delivery and competition fundamentally [8]. With the development of online education, students, parents, and educators about online teaching satisfaction have been the focus of scholars around the world.

Before research into neural networks became hot, many scholars studied the factors affecting teaching satisfaction by various statistical methods and traditional machine learning methods. Some typical studies are as follows: Through regression analysis, Kuo et al. [9] explored the contribution of independent variables to student satisfaction and the effect of student background on dependent variables. The results showed that learner-instructor interaction, learner-content interaction, and Internet self-efficacy were good predictors of student satisfaction. Cole et al. [10] used three years to conduct an e-learning satisfaction survey on 553 students, and the results showed no statistically significant difference in satisfaction based on gender, age, or study level. “Convenience” and “Lack of interaction” are the two most influential factors on student satisfaction. Bolliger and Wasilik [11] developed and validated a tool that can be used to perceive teacher satisfaction in online learning. Eom et al. [12] used structural equation modeling to study the determinants of student satisfaction in the context of online courses, and the structural model results reveal that instructor feedback and learning style are significant predictors of learning outcomes. Gunawardena et al. [13] designed a hybrid method including qualitative and quantitative methods to explore the factors affecting employee satisfaction in a multinational company’s online education project. Lee [14] used factor analysis, structural equation modeling, independent sample t-test, and other techniques to investigate the potential differences in online learning acceptance and satisfaction between Korean and American students. Simpson [15] studied a sample of 157 students participating in online education and found that teaching content that passed the teacher review can improve student satisfaction. Roach et al. [16] believe that educators should always pay attention to students’ progress when conducting online education. Eom [17] analyzes the causes and consequences of teacher-student interaction in asynchronous learning courses and finds that curriculum structure, students’ self-motivation, and learning style all affect teacher-student interaction.

Wang et al. [18] used the structural equation model to explore the relationship between student characteristics, self-regulated learning, self-efficacy, and course results in an online learning environment. The model results found that students with previous online learning experience tend to have more effective learning strategies when taking online courses. In order to reduce the sense of isolation that online learners may experience, which in turn affects their satisfaction with online learning, McInnerney and Roberts [19] point out, the authors offer several suggestions:

1) more use of synchronous rather than asynchronous communication facilities.

2) adding a “Warm-up” phase to the curriculum structure.

3) focusing on communication in the teaching process.

With the maturity of neural network technology, various neural network models have been applied to academic research in education. Guo [6] uses the three-layer multilayer perceptron (MLP) models to build a prediction model of student’s curriculum satisfaction that is more accurate than linear regression. The study shows that the number of students enrolled in courses and the high distinction rate in the final exam is the two most important factors affecting course satisfaction. Kardan et al. [20] developed a neural network model for student satisfaction and course selection, and the results show that the model is superior to three famous machine learning algorithms and two previous naive algorithms.

Aydogdu [21] used a neural network model to study the effects of gender, course time, and homework completion on students’ final course scores. The prediction accuracy was 80.47%. Agaoglu [22] uses artificial neural networks to mine teacher performance data. Yukselturk et al. [23] use a genetic algorithm-based feature selection algorithm and a neural network to predict who will drop out of school during online education.

2.2. Urban-Rural Differences in Education

Internationally, online education is seen as an opportunity to improve primary education in rural areas. A large body of literature discusses the contribution of different online education models to narrowing the socio-economic gap between developing and developed regions.

A survey by Jinqiu et al. [24] shows that communities in northwest China have benefited enormously from Internet and communications technology programs. Zhang et al. [25] designed and used scaffold strategies to conduct a cross-regional collaborative learning activity among secondary schools. The study found that systematic scaffold strategies were essential for students to have compelling cross-regional online collaborative learning experiences. Wang and Zhao [26] summed up the achievements of fundamental education reform in rural China, analyzed the problems existing in the reform, and put forward countermeasures to solve these problems. Research by Wei et al. [27] has shown a significant positive relationship between education and income in rural areas.

3. Data Collection and Processing

3.1. Survey Design

Several departments designed the questionnaire for teachers. First, three university educators and three provincial education department staff designed the first draft of the questionnaire. This first draft was then distributed to 30 teachers from different schools. Feedback and suggestions were received from these thirty teachers so that the questionnaire could be revised and adjusted. As shown in Table 1, the final teacher questionnaire consisted of twenty questions, which can be grouped into four categories as follows.

3.1.1. Personal Information

In order to compare the variability of teachers in different regions and age groups in the questionnaire, information was collected on the teachers’ region (Q1), the teachers’ age (Q2), the teachers’ working hours (Q3), the teachers’ highest level of education (Q4), the teachers’ title (Q5), the teachers’ teaching audience (Q6) and the teachers’ teaching content (Q7).

3.1.2. Online Teaching Behavior

Six questions were designed to collect information about the teacher’s teaching behavior or behavior-related. These include Q8, Q10, Q11, Q12, Q13, and Q14. For example, the teaching platform used by the teacher (Q11), the way the lesson is organized (Q12), the content of the lesson (Q13).

3.1.3. Online Teaching Experience

The questionnaire was designed with six questions to capture teachers’ teaching experiences online. These included teachers’ concerns before implementing teaching (Q9), their satisfaction with online education (Q15), what competencies they think online education can develop in students (Q16), the advantages of

Table 1. Contents of the teacher questionnaire.

online education (Q17), the key factors that make online education perfect (Q18), and the problems that still exist with online education (Q19).

3.1.4. Attitudes to Online Teaching

At the end of the questionnaire, teachers were asked about their future teaching options (Q20) to gather their attitudes towards online teaching.

3.2. Data Collection

The target group for this study was primary, middle, and high school teachers in a Chinese province, covering all stages of K-12 education. The questionnaire was used to investigate the target group’s participation in online education during the COVID-19 outbreak. A questionnaire was anonymous and voluntary to ensure the confidentiality and reliability of the data and the privacy of the respondents. The questionnaire was released online on 23 March 2020, and 498,481 questionnaires were returned as of 4 April 2020.

3.3. Data Cleaning

In order to reduce the influence of dirty data on the results and to ensure the validity of the experimental results, we performed a clean data filter. The simplest and most effective way is to analyze the set of values for individual features. Assuming that x is a feature with continuous values and that $x^{*}$ is a valid value in that feature, it should satisfy the following relationship.

$\bar{x}-3\sigma \le x^{*}\le 3\sigma +\bar{x},\bar{x}=\frac{{\displaystyle {\sum }_{i=1}^n{x}_i}}{n},\sigma =\sqrt{\frac{{\displaystyle {\sum }_{i=1}^n{\left(x_i-\bar{x}\right)}^2}}{n}}.$ (1)

In addition, we believe that the span of grades taught by the same teacher should not exceed 6. If a teacher teaches both Year 1 and Year 7 students, we would consider this figure unreasonable. Similarly, we believe that the total number of subjects taught by the same teacher should not exceed three. If a teacher teaches four subjects simultaneously—language, mathematics, English, and physics—we do not consider the data reasonable.

After data cleaning, the statistics showed that the number of valid questionnaires in the teachers’ questionnaire was 493,747, with an effective rate of 99.05%.

4. Analysis of the Differences between Urban and Rural Teachers

In this section, we analyze the data collected using cross-tabulations and chi-square tests to examine the differences in the online education experiences of rural and urban teachers. Of all the data collected, the number of rural teachers was 290,890, or 58.91%. The number of urban teachers was 202,853, or 41.09% of the total number of teachers.

4.1. Comparison of the Basic Attributes of Urban and Rural Teachers

As shown in Table 2, in three age ranges of 29 and below, 40 to 49, and 60 and above, the proportions of rural and urban teachers are similar. In contrast, in the age range of 30 to 39, the proportion of teachers in urban areas is higher than in rural areas, and in the age range of 50 to 59, the proportion of urban teachers is lower than the overall proportion, while the proportion of rural teachers is higher than the overall proportion.

One of the most critical factors hindering the development of primary education in Chinese townships is the lack of quality teachers [28]. Looking at rural teachers compared to urban teachers in terms of degree and age, the proportion of teachers with a Ph.D. is smaller in both rural and urban areas, but of the proportion of teachers with a master’s degree, the proportion of rural teachers is only one-third that of urban teachers. In addition, the proportion of teachers

Table 2. Basic personal information of urban and rural teachers.

with a bachelor’s degree in townships is nearly ten percentage points lower than that of urban teachers.

In urban schools, nearly 90% of teachers teach only one course during their employment, while this proportion is only 76.91% of teachers in rural areas. Moreover, we can see from the table that the proportion of rural teachers who teach two or three courses simultaneously is about double that of urban teachers.

In township schools, the shortage of teachers in traditional subjects (such as Chinese and mathematics) is not apparent, but there is an increasing lack of talents to teach modern subjects, such as English, computer science, music, and physical education [28]. As a result, a teacher often has to teach more than one course, even if he has not mastered some of them.

4.2. Concerns of Urban and Rural Teachers Prior to Online Teaching

From Table 3, both urban and rural teachers are concerned that student behavior is challenging to monitor and teaching is ineffective when teaching online. In addition, rural teachers differ from urban teachers in the proportional distribution of the following four options: students are not equipped to teach online, parents do not support and cooperate with online teaching, insufficient capacity to teach online, and immature conditions for teaching online.

The Chi-square test showed significant association between school districts and teachers’ worries, ${\chi }^2\left(N=493743\right)=201.003$, $p<0.01$. The Cramer’s V is 0.08 which suggests a small effect [29].

The implementation of online education has put forward higher requirements for teachers who can master and apply modern educational technology. However, many teachers in villages and towns are old teachers, most of them can’t use electronic devices to teach, which seriously restricts the further development of

Table 3. Urban and rural teachers’ worries before starting online teaching.

online education in villages and towns [30].

4.3. Difficulties That Urban and Rural Teachers Used to Encounter in the Process of Online Teaching

Interestingly, there was not much difference in the challenges encountered by urban and rural teachers in online classes during the epidemic, as can be seen in Table 4. ${\chi }^2\left(N=493743\right)=39.211$, $p<0.01$. The Cramer’s V is 0.009 which suggests a small effect [29].

When teaching online, teachers have a range of problems such as difficulty monitoring students’ learning behavior, distracted students, inconvenient teacher-student communication, and less effective teaching than in a physical classroom. These problems may be related to the medium of learning used by students. The most portable internet tool available to us in our daily lives is the smartphone. The ubiquity, versatility and connectivity of smartphones provide a new and potentially powerful online learning environment [31]. For younger students, the games on the smartphone may be more attractive than the teacher’s teaching content, thereby distracting them. In addition, the small screen of a smartphone makes it more suitable for reading short texts quickly, rather than lengthy materials [32]. As a result, students’ prolonged use of small-screen mobile phones for learning can lead to eye strain and poor concentration, affecting the effectiveness of online classes.

4.4. Factors Affecting Urban and Rural Teachers’ Choice of Online Teaching Platform

Besides having essential online teaching functions for a well-established online teaching platform, it should also support functions such as assignment posting, in-class questioning, and student management. These functions are implemented on mainstream platforms, so there is not much difference in the choice of rural teachers and urban teachers on this issue. ${\chi }^2\left(N=493743\right)=277.494$,

Table 4.Difficulties encountered by urban and rural teachers teaching online.

$p<0.01$. The Cramer’s V is 0.024 which suggests a small effect [29].

As illustrated in Table 5, Compared to urban teachers, rural teachers are more concerned with the following factors when choosing an online teaching platform, such as whether the platform has rich course resources, whether the content of the platform’s courses is of higher quality, whether the platform’s courses meet current teaching needs, whether the platform’s teaching methods are flexible, and whether the platform’s teaching resources are more easily accessible. However, urban teachers are more concerned about the platform’s interactivity and whether it can provide a relaxed and interactive environment.

4.5. Key Factors Affecting Online Education

Table 6 above shows the similarities between rural and urban teachers on factors that influence online education. Both urban and rural teachers agree that the two most important factors affecting online education are the students’ ability to learn independently and whether their parents help support them.

Similarly, as shown in Table 4, when asked about the difficulties encountered in online teaching, both urban and rural teachers agreed that one of the imperfect aspects of online education was the inability to monitor students effectively.

In addition, if the government is to implement online education on a large scale, it should have supporting hardware and software facilities, and teachers involved in online education need to have some ability to use multimedia equipment for teaching.

4.6. The Teaching Methods Chosen by Urban and Rural Teachers after the End of COVID-19

As can be seen from Table 7 below, for the choice of teaching method after the epidemic, more than half of the rural teachers opted for the previous offline teaching mode. Similarly, about 46% of urban teachers chose to revert to their

Table 5. Comparison of factors driving teachers to choose different teaching platforms.

Table 6. Teachers’ perceptions of key factors affecting online education.

Table 7. Teacher teaching format preferences after the end of the epidemic.

previous mode of teaching. It shows that the time is not fully ripe for the implementation of online education.

For the option “Combine online and offline teaching”, urban teachers are about seven percentage points more likely than rural teachers. Therefore, we suggest that urban teachers are more open to online education than rural teachers.

Teachers who chose to adopt a fully online teaching mode after the epidemic ended represent only a small proportion of teachers in towns and villages.

5. Building a Neural Network Model to Analyze Teacher Satisfaction

5.1. Building Our Neural Network

The statistical analysis above shows the differences between urban and rural teachers when teaching online, but it would make more sense to use neural networks if we had a large amount of data. Since the emergence of online education, many scholars have researched teachers’ and students’ satisfaction in online education, primarily using traditional machine learning methods such as logistic regression and random forest. Considering the current technological hotspots and the size of our data, we decided to use a neural network model to analyze teacher satisfaction.

Since the creation of Alexnet [33] in 2012, the number of layers in artificial neural networks has been moving more profoundly, and it is intuitively assumed that as the number of layers increases, the network’s ability to fit the features will improve. Nevertheless, it was not to be. It has been found that the accuracy of the model does not constantly improve with increasing network depth.

The residual neural network was proposed by Kaiming He [34] and others in 2015. Compared with traditional neural networks, deep residual neural networks add a jump layer connection structure to the forward propagation, which effectively improves the performance of deep neural networks and enhances the feature fitting ability of deep neural networks. Given the excellent performance of the residual structure, we also decided to use the residual structure when building our neural network.

The left-hand side of Figure 1 shows the bottleneck, a component of the residual neural network, which can be defined as:

Figure 1. The basic residual unit in ResNet (left), the basic constituent unit of our neural network (centre) and the structural model of our neural network (right).

$y=F\left(x,W_i\right)+x,$ (2)

where x and y are the input and output vectors of the layers to be considered, W_i is the weight in the weight matrix, and F represents the residual mapping to be learned. For an example that has three layers, its residual mapping function is as follows:

$F=W_3\cdot g\left(W_2\cdot g\left(W_{1}x+b_1\right)+b_2\right)+b_3,$ (3)

$g\left(x\right)=\max \left(x,0\right),$ (4)

where g denotes the Rectified Linear Unit activation function, and W₁, W₂, W₃, b₁, b₂, and b₃ are the weights and biases of the first layer, the second layer and the third layer, respectively.

Deep residual neural networks are generally used to process high-latitude image data, and questionnaire data containing just over 100 features is straightforward compared to image data. Therefore, we choose the classical residual neural network structure with the least number of neural network layers, the 18-layer network structure, as our template, and we will further improve on the 18-layer residual neural network.

However, even if it is the 18-layers residual neural network with the smallest number of layers, each of its layers contains four 3 × 3 convolutional layers to achieve feature extraction. Compared with image data, our questionnaire data has fewer features, and continuous convolution may lead to the loss of essential features. Therefore, we need to modify the traditional 18-layers residual neural network. We used a 1 × 1 convolutional layer and two bottleneck structures to form the building block (Figure 1. Centre) of our neural network. We then implemented our own network (Figure 1. Right) using building blocks instead of the two basic block structures for each layer of the classic 18-layer ResNet. It significantly weakens the feature extraction capability of the traditional 18-layer residual neural network, thereby retaining essential features.

In our neural network, Rectified Linear Unit (ReLU) is used as the activation function. The output neuron of the j_th layer can be expressed as:

$O_j=\text{ReLU}\left(S_j\right)=\max \left(0,W_j^{\text{T}}\cdot x_j+b\right),$ (5)

where w and b denote the weight and bias, respectively. Use real numbers as parameters and return real values within $\left[0,+\infty \right)$. For the output layer, use the softmax function as the activation function. Similar to the above, the output neuron of the j_th layer can be expressed as:

$O_j=\sigma \left(S_j\right)=\frac{{\text{e}}^{S_j}}{{\displaystyle {\sum }_{k=1}^m{\text{e}}^{S_k}}},$ (6)

m is the number of output neurons. In our neural network structure, the value of m is 3. The softmax function takes real numbers as parameters and maps them to real values between 0 and 1, and makes the sum equal to 1. Since the sum of the output is 1, the softmax layer can be regarded as a probability distribution, and the O_j value can be interpreted as the estimated probability of the input classification by the network.

We use the commonly used Adam as the optimizer during model training. When training the model, momentum and adaptive learning rate can be used to speed up the convergence speed. We choose Categorical Cross entropy as the loss function. Cross entropy can be used to evaluate the difference between the probability distribution obtained by the current training and the true distribution. It describes the distance between the actual output and the expected output, that is, the smaller the value of cross entropy, the closer the two probability distributions are.

After the basic neural network framework has been built, the question arises as to how to process the data to meet the input requirements of the neural network. The teacher questionnaire consists of 20 questions, including 8 single-choice and 12 multiple-choice questions. Each option for a single-choice question is represented by a number that has a practical meaning, for example, for question 4: the highest qualification obtained by the teacher. The number 1 represents a doctorate, the number 2 a master’s degree, the number 3 a bachelor’s degree, and the number 4 a college degree. We chose the answer to each multiple-choice question as one of our independent variables. We use 0 and 1 to indicate whether the teacher selected multiple-choice options when completing the questionnaire. If this option was selected, we use 1 to indicate this, and if not, we use 0. We, therefore, use a vector of length the number of multiple-choice options to represent the results of the multiple-choice questions.

It is worth mentioning that in multiple-choice questions, there are some meaningless options, such as “other,” and after removing some of these options, we combine all the independent variables to form a 117 × 1 vector data. To make the data satisfy the input conditions of our neural network, we added four zeros to the end of the original data and adjusted them to an 11 × 11 numerical matrix. We then greyed out the digital matrix to obtain a grey-scale image, the input data for our neural network.

We chose teacher satisfaction with online education as the dependent variable. Teacher satisfaction with online education is a three-category variable with three values. Zero indicates that teachers are satisfied with online education, one indicates that teachers are generally satisfied with online education, and two indicates that teachers are dissatisfied with online education. We use this value as the output data of the neural network.

5.2. Experimentation and Performance Evaluation

The 18-layer residual neural network is the classical residual neural network structure. However, it is unknown whether an 18-layer residual neural network with modified essential components of the neural network can still maintain high prediction accuracy. In order to verify the rationality of our neural network structure, we conducted a comparison experiment with the traditional 18-layers residual neural network.

We conducted experiments with a data volume of 30,000, and compared two neural networks in terms of prediction accuracy and training time, and set the number of iterations to 128.

As can be seen from Table 8, our neural network model is more accurate than ResNet-18 in terms of prediction and our neural network requires less training time.

After comparing the impact of different layer neural network structures on our dataset, we compared the performance of our neural network with traditional machine learning algorithms (Logistic Regression, SVM, Naive Bayesian, Decision Tree, Random Forest) on three datasets with different data sizes of 50,000, 150,000 and 450,000.

In addition, we performed a cross-sectional comparison of the models, replicating the artificial neural network model from the Bang Won Seok paper [35] (named ANN) and the backpropagation neural network model from the Tinggui [36] Chen paper (named BP), and experimented with them on our dataset. The experimental results are shown in Table 9.

True positive (TP), false negative (FN), false positive (FP), and true negative (TN) are the four kinds of outcomes (TN). TP denotes successfully projected positive group samples, FN denotes a positive group that was wrongly forecasted as unfavorable, FP denotes an antagonistic group that was incorrectly predicted as positive, and TN denotes a correctly predicted negative group. We utilize the following assessment criteria to forecast the performance of our approach and compare it to other ways based on these indices. The proportion of the correct number of positive samples in the overall number of positive samples computed by the classifier is referred to as precision.

${\text{Precision}}_i=\frac{{\text{TP}}_i}{{\text{TP}}_i+{\text{FP}}_i}.$ (7)

The proportion of the right number of positive samples in the total number of positive samples is referred to as recall.

${\text{Recall}}_i=\frac{{\text{TP}}_i}{{\text{TP}}_i+{\text{FN}}_i}.$ (8)

The harmonic mean of accuracy and recall is used to get the F1 score.

$\text{F}{1}_i=2\cdot \frac{{\text{Precision}}_i\cdot {\text{Recall}}_i}{{\text{Precision}}_i+{\text{Recall}}_i}.$ (9)

In our dataset, the output is triple classification rather than binary classification,

Table 8. The prediction accuracy of neural networks with different structures.

Table 9. Comparison of our neural network model with other algorithms.

so we need to calculate Precision, Recall and F1 separately for each output type and then average them.

The experimental results in the Tableshow that our neural network model has a higher prediction accuracy than the traditional machine learning model for data amounts of 50,000, 150,000, and 450,000. For 150,000 and 450,000, our neural network model performs well in Precision, Recall, and F1-Score, and is far better than other traditional machine learning models.

5.3. Visual Explanation of the Model’s Decision-Making Process

In the previous experiment, we built a neural network model and used several features to predict teacher satisfaction accurately. The black-box nature of neural networks makes it difficult to analyze which features play a key role in the decision-making process of neural networks. We changed the representation of the data before feeding it into the neural network, which allowed us to label the key features in the decision-making process of the neural network by means of a heat map, via the Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm [37]. For every target concept, Grad-CAM employs gradients of that concept to build a coarse localization map that highlights the locations in the picture where the concept may be predicted.

In order to obtain the location mapping for the classification discriminations of class c, we first compute the gradient of the score for class c, $y^c$ (before the

softmax), with respect to feature maps $A^k$ of a convolutional layer, i.e. $\frac{\partial y^c}{\partial A^k}$.

Neuron significance weights ${\varpi }_k^c$ are derived from these gradients by averaging the global-average-pooled values:

${\varpi }_k^c=\frac{1}{z}{\displaystyle \sum {\displaystyle \sum \frac{\partial y^c}{\partial A^k}}}.$ (10)

We combine forward activation maps with a ReLU and weight them to get,

$L_{\text{Grad-CAM}}^c=\text{ReLU}\left({\displaystyle {\sum }_k{\varpi }_k^c\cdot A_{ij}^k}\right).$ (11)

The ReLU function is used to remove the influence of negative values on the feature map on the classification results, and the final classification task is expressed by Equation (11).

We integrate the Grad-CAM algorithm into our overall process to obtain the reasons for the decisions made by the neural network model for each input data. Figure 2 illustrates our complete work, including data collection, filtering, building the neural network model, and using the Grad-CAM algorithm to label the essential features of the neural network model for making decisions.

The Grad-CAM technique makes our model more transparent by generating a visual interpretation of which regions play an essential role in the decision-making of the neural network. We, therefore, selected six individuals from the data of teachers successfully predicted by the neural network model: three teachers who were satisfied, generally satisfied, and dissatisfied with online education in the urban teacher, and three teachers who were satisfied, generally satisfied, and dissatisfied with online education in the rural teacher. The data from the six teacher samples were processed using Grad-CAM technology, and Figure 3 below shows the results of our experiment.

Among the teachers who were satisfied with online education, features 46, 47, 48, 57, 58, and 59 in the urban teachers’ data played an essential role in the decision-making process of the neural network. By reviewing the questionnaire, we found that these features correspond to the usability of the online education platform and the abundance of online course resources, respectively. Features 82, 83, 84 and 93, 94 and 95 in the data on rural teachers play an essential role in the decision-making process of the neural network. The practical implication of these features is that online education improves the ability of students to use digital devices.

Figure 2. Overall architecture.

Of those teachers who were generally satisfied with online education, features 84, 85, 86 and 95, 96 and 97 in the urban teacher data played a significant role in the decision-making process of the neural network, with the corresponding questions in the questionnaire being the impact of online education on students’ expression and communication respectively. Features 2 and 3 in the township teacher data significantly influenced the decision-making of the neural network, corresponding to the questions in the questionnaire about the teacher’s age, educational background, and the subject taught.

Some of the teachers were dissatisfied with this online education practice. By analyzing the heat map, we can learn that features 92, 103, 111, and 112 in the urban teachers’ data played an essential role in the decision-making process of the neural network, and these features correspond to problems such as poor online education in the questionnaire, respectively. Features 24, 25, 35, 36, 46, and 47 of the rural teacher data play an essential role in the decision-making process of the neural network, and these features correspond to issues such as inadequate

Figure 3. The heat map of city teachers and rural teachers.

online facilities in the questionnaire.

6. Discussion and Suggestion

The primary purpose of this study was to explore the differences between urban and rural teachers’ online education and examine the factors that influence teachers’ satisfaction with online teaching. The study results show some differences between urban teachers and rural teachers in their participation in online education.

The relative prosperity of cities provides a good material guarantee for citizens [38]. The economic differences between urban and rural areas and the imbalance in educational resources lead to differences in educational infrastructure between urban and rural areas, affecting teacher satisfaction.

Most teachers believe that offline learning is more effective than online learning. Online learning mainly faces the problems of network technology resources, the lack of interaction with students, and the lack of classroom atmosphere [39]. In our study, the above phenomenon was also confirmed. Some 53% of teachers felt that the web-based resources were limited and prone to clogging and freezing, and some 26% felt that communication with students was not smooth enough.

Online education during the epidemic has a more significant impact on subjects that need to exercise their hands-on skills [40]. After the outbreak, we found that most urban and rural teachers returned to their previous physical classroom teaching style. On the one hand, it may be that these teachers felt that online education blocked student-to-student communication and student-to-teacher interaction, and on the other hand, it may be related to the fact that teachers were required to conduct a range of hands-on demonstrations and related to, for example, physics, chemistry, art, music.

Online education is a complex task with high requirements for teachers and is easy to make people burn out [41]. Therefore, the government must provide a range of training for teachers before implementing online education, as some teachers have difficulty meeting the IT standards and adaptability required for online teaching. Therefore, whether the school organizes relevant training is an essential factor affecting teacher satisfaction. Teachers with distance education experience and teachers without experience have significant differences in their views on distance education [42]. In general, teachers with relevant multimedia teaching experience were more confident in delivering online education. This was confirmed in our study, where many teachers had concerns and anxieties prior to delivering online because they had never been exposed to multimedia teaching before.

The grade of the teacher’s teaching object also affects the teacher’s satisfaction. Teachers believe that online education needs students’ self-discipline to maintain [43]. Therefore, the students in the upper grades may be more self-disciplined so that teachers do not need to spend too much energy, thereby improving teacher satisfaction.

While online education is seen as an effective means of breaking regional monopolies in educational resources and enhancing equity in education for individual students and families, participation in online learning requires additional payments for equipment and communication, increasing the financial burden on families. Thus, while online learning reduces the education gap between districts, it does not effectively reduce educational resources between students in the same district. On the contrary, it may further widen the gap. Therefore, local governments should actively promote the construction and improvement of Internet infrastructure. For remote and rural areas, Internet tariffs can be appropriately reduced.

Teachers in primary and secondary schools should pay more attention to poor students during particular times and give extra financial or academic help. When promoting online education, schools should find out in advance whether online resources are available for students, impoverished students, and children left behind to prevent students from being left behind due to financial problems.

The raging epidemic has accelerated the growth of online education. Through this brief period of online education practice, the education sector and educators should be aware of potential problems in the current online education model and take relevant measures to address them. As distance education becomes increasingly accepted by the general public, best practices in online education must continue to be explored for students of all ages, cultures, and socio-economic statuses.

7. Conclusion

In this work, we first designed and distributed a questionnaire for teachers who participated in online classes during the COVID-19 epidemic to primary and secondary school teachers in Guangdong Province, China, and then received nearly 500,000 questionnaire returns. After cleaning the data collected, we analyzed the differences between urban and rural teachers using statistical methods such as cross-tabulation and chi-square tests. To make more profound use of the data we collected, we constructed a neural network model to predict teacher satisfaction, which in comparative experiments outperformed many traditional machine learning algorithms and some models designed by previous scholars. We conclude with a transparent demonstration of the decision process of the neural network model using Grad-CAM techniques, as we believe that a model should not be mysterious and black-box. Future work includes optimization of the model using the attention mechanism and homomorphic encryption algorithms for model security.

Funding

This research is grateful to the National natural Science Foundation of China (NSFC, 62077028), the Science and Technology Planning Project of Guangdong (2021B0101420003, 2020B0909030005, 2020B1212030003, 2019A050510024, 2020ZDZX3013, 2019B1515120010, 2018KTSCX016, 2021KTSCX264), the Science and Technology Planning Project of Guangzhou (201902010041), the teaching reform research projects of Jinan University (JG2021112) for the funding support, which enable the successful completion of this research.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Bao, Y.P., Sun, Y.K., Meng, S.Q., Shi, J. and Lu, L. (2020) 2019-nCoV Epidemic: Address Mental Healthcare to Empower Society. The Lancet, 395, E37-E38. https://doi.org/10.1016/S0140-6736(20)30309-3
[2]	Hodges, C., Moore, S., Lockee, B., Trust, T., Bond, A., et al. (2020) The Difference between Emergency Remote Teaching and Online Learning. Educause Review, 27, 1-12.
[3]	Bao, C.Y. (2006) Policies for Compulsory Education Disparity between Urban and Rural Areas in China. Frontiers of Education in China, 1, 40-55. https://doi.org/10.1007/s11516-005-0003-y
[4]	Coyner, S.C. and McCann, P.L. (2004) Advantages and Challenges of Teaching in an Electronic Environment: The Accommodate Model. International Journal of Instructional Media, 31, 223-229.
[5]	Sherry, L. (1995) Issues in Distance Learning. International Journal of Educational Telecommunications, 1, 337-365.
[6]	Guo, W.W. (2010) Incorporating Statistical and Neural Network Approaches for Student Course Satisfaction Analysis and Prediction. Expert Systems with Applications, 37, 3358-3365.
[7]	Moore, J.L., Dickson-Deane, C. and Galyen, K. (2011) E-Learning, Online Learning, and Distance Learning Environments: Are They the Same? The Internet and Higher Education, 14, 129-135.
[8]	Poehlein, G.W. (2007) Universities and Information Technologies for Instructional Programmew Issues and Potential Impacts. Technology Analysis & Strategic Management, 8, 283-290. https://doi.org/10.1080/09537329608524251
[9]	Kuo, Y.C., Walker, A.E., Belland, B.R. and Schroder, K.E.E. (2013) A Predictive Study of Student Satisfaction in Online Education Programs. The International Review of Research in Open and Distributed Learning, 14, 16-39. https://doi.org/10.19173/irrodl.v14i1.1338
[10]	Cole, M.T., Shelley, D.J. and Swartz, L.B. (2014) Online Instruction, E-Learning, and Student Satisfaction: A Three Year Study. The International Review of Research in Open and Distributed Learning, 15, 111-131. https://doi.org/10.19173/irrodl.v15i6.1748
[11]	Bolliger, D.U. and Wasilik, O. (2009) Factors Influencing Faculty Satisfaction with Online Teaching and Learning in Higher Education. Distance Education, 30, 103-116. https://doi.org/10.1080/01587910902845949
[12]	Eom, S.B., Wen, H.J. and Ashill, N. (2006) The Determinants of Students’ Perceived Learning Outcomes and Satisfaction in University Online Education: An Empirical Investigation. Decision Sciences Journal of Innovative Education, 4, 215-235. https://doi.org/10.1111/j.1540-4609.2006.00114.x
[13]	Gunawardena, C.N., Linder-VanBerschot, J.A., LaPointe, D.K. and Rao, L. (2010) Predictors of Learner Satisfaction and Transfer of Learning in a Corporate Online Education Program. The American Journal of Distance Education, 24, 207-226. https://doi.org/10.1080/08923647.2010.522919
[14]	Lee, J.W. (2010) Online Support Service Quality, Online Learning Acceptance, and Student Satisfaction. The Internet and Higher Education, 13, 277-283.
[15]	Simpson, J.M. (2012) Student Perceptions of Quality and Satisfaction in Online Education. The University of Alabama, Tusca-loosa.
[16]	Roach, V., Lemasters, L., et al. (2006) Satisfaction with Online Learning: A Comparative Descriptive Study. Journal of Interactive Online Learning, 5, 317-332.
[17]	Eom, S. (2009) Effects of Interaction on Students’ Perceived Learning Satisfaction in University Online Education: An Empirical Investigation. International Journal of Global Management Studies, 1, 60-74.
[18]	Wang, C.H., Shannon, D.M. and Ross, M.E. (2013) Students’ Characteristics, Self-Regulated Learning, Technology Self-Efficacy, and Course Outcomes in Online Learning. Distance Education, 34, 302-323. https://doi.org/10.1080/01587919.2013.835779
[19]	McInnerney, J.M. and Roberts, T.S. (2004) Online Learning: Social interaction and the Creation of a Sense of Community. Journal of Educational Technology & Society, 7, 73-81.
[20]	Kardan, A.A., Sadeghi, H., Shiry Ghidary, S. and Sani, M.R.F. (2013) Prediction of Student Course Selection in Online Higher Education Institutes Using Neural Network. Computers & Education, 65, 1-11.
[21]	Aydogdu, S. (2020) Predicting Student Final Performance Using Artificial Neural Networks in Online Learning Environments. Education and Information Technologies, 25, 1913-1927. https://doi.org/10.1007/s10639-019-10053-x
[22]	Agaoglu, M. (2016) Predicting Instructor Performance Using Data Mining Techniques in Higher Education. IEEE Access, 4, 2379-2387. https://doi.org/10.1109/ACCESS.2016.2568756
[23]	Yukselturk, E., Ozekes, S. and Turel, Y.K. (2014) Predicting Dropout Student: An Application of Data Mining Methods in an Online Education Program. European Journal of Open, Distance and E-Learning, 17, 118-133. https://doi.org/10.2478/eurodl-2014-0008
[24]	Zhao, J.Q., Hao, X.M., and Banerjee, I. (2006) The Diffusion of the Internet and Rural Development. Convergence, 12, 293-305. https://doi.org/10.1177/1354856506067202
[25]	Zhang, W.Y., Chen, L. and Feng, X.Y. (2008) Distance Education in Rural China Achieves Inter-School Collaboration and Increased Access to Education. Asian Journal of Distance Education, 6, 27-38.
[26]	Wang, J.Y. and Zhao, Z.C. (2011) Basic Education Curriculum Reform in Rural China: Achievements, Problems, and Solutions. Chinese Education & Society, 44, 36-46. https://doi.org/10.2753/CED1061-1932440603
[27]	Wei, X., Tsang, M.C., Xu, W.B. and Chen, L.K. (1999) Education and Earnings in Rural China. Education Economics, 7, 167-187. https://doi.org/10.1080/09645299900000014
[28]	McQuaide, S. (2009) Making Education Equitable in Rural China through Distance Learning. The International Review of Research in Open and Distributed Learning, 10, 1-21. https://doi.org/10.19173/irrodl.v10i1.590
[29]	Cohen, J. (2013) Statistical Power Analysis for the Behavioral Sciences. Academic Press, Cambridge.
[30]	Wu, Y.Q., He, W. and Tan, J.S. (2009) Status, Function and Development Trend of Distance Education in the Rural Areas of China. 2009 International Conference on E-Learning, E-Business, Enterprise Information Systems, and E-Government, Hong Kong, 5-6 December 2009, 181-184.
[31]	Woodcock, B., Middleton, A. and Nortcliffe, A. (2012) Considering the Smartphone Learner: An Investigation into Student Interest in the Use of Personal Technology to Enhance Their Learning. Student Engagement and Experience Journal, 1, 1-15.
[32]	Anthony, S.G. and Keating, M.S. (2013) The Difficulties of Online Learning for Indigenous Australian Students Living in Remote Communities—It’s an Issue of Access. Online Journal of Distance Learning Administration, 16, 1-10.
[33]	Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012) Imagenet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 25, 1097-1105.
[34]	He, K.M., Zhang, X.Y., Ren, S.Q. and Sun, J. (2016) Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 27-30 June 2016, 770-778. https://doi.org/10.1109/CVPR.2016.90
[35]	Seok, B.W., Wee, K.H., Park, J.Y., Kumar, D.A. and Reddy, N.S. (2021) Modeling the Teacher Job Satisfaction by Artificial Neural Networks. Soft Computing, 25, 11803-11815. https://doi.org/10.1007/s00500-021-05958-0
[36]	Chen, T.G., Peng, L.J., Yin, X.H., Rong, J.T., Yang, J.J. and Cong, G.D. (2020) Analysis of User Satisfaction with Online Education Platforms in China during the Covid-19 Pandemic. Healthcare, 8, Article No. 200. https://doi.org/10.3390/healthcare8030200
[37]	Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D. and Batra, D. (2017) Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, 22-29 October 2017, 618-626. https://doi.org/10.1109/ICCV.2017.74
[38]	Shigemura, J., Ursano, R.J., Morganstein, J.C., Kurosawa, M. and Benedek, D.M. (2020) Public Responses to the Novel 2019 Coronavirus (2019-nCoV) in Japan: Mental Health Consequences and Target Populations. Psychiatry and Clinical Neurosciences, 74, 281-282. https://doi.org/10.1111/pcn.12988
[39]	Adnan, M. and Anwar, K. (2020) Online Learning amid the Covid-19 Pandemic: Students’ Perspectives. Online Submission, 2, 45-51.
[40]	Castelli, G., Diab, D., Scimeca, A., Mehta, C., Payne, N., Mershon, J., Ray, S., DeLancey, J., Lee, C. and Posid, T. (2021) Assessing the Impact of the Covid-19 Pandemic on Medical Student Education. Research Square. https://doi.org/10.21203/rs.3.rs-131836/v1
[41]	Hogan, R.L. and McKnight, M.A. (2007) Exploring Burnout among University Online Instructors: An Initial Investigation. The Internet and Higher Education, 10, 117-124. https://doi.org/10.1016/j.iheduc.2007.03.001
[42]	Ulmer, L.W., Watson, L.W. and Derby, D. (2007) Perceptions of Higher Education Faculty Members on the Value of Distance Education. Quarterly Review of Distance Education, 8, 59-70.
[43]	Allen, I.E. and Seaman, J. (2006) Growing by Degrees: Online Education in the United States, 2005. The Sloan Consortium.