Assessment of WeChat Work Online Teaching Modes under COVID-19: Based on AHP and Fuzzy Comprehensive Evaluation Method

Cejun Cao^1*, Yuting Xie¹, Yunhong Zhou¹, Ying Gong¹, Min Gao²

¹Department of Logistics Management, School of Management Science and Engineering, Chongqing Technology and Business University, Chongqing, China.
²Department of Marketing, School of Business Administration, Chongqing Technology and Business University, Chongqing, China.

DOI: 10.4236/jss.2020.87028                

Abstract

To cope with the unexpected situation in the field of education even online education under novel coronavirus epidemic, different online teaching modes are proposed and implemented. However, how to improve the quality and effectiveness of online teaching is still challenging. This paper makes a quantitative assessment about different online teaching modes of WeChat Work, puts forward a comprehensive evaluation system framework, and uses Analytic Hierarchy Process to determine the weights of various factors. Then, fuzzy comprehensive evaluation method is applied to quantitatively analyze the performance of each online teaching mode so as to obtain the optimal scheme. Results demonstrated that live broadcast of WeChat Work was the best option because of its overall advantages in different aspects. In other words, live broadcast teaching was recommended to the teachers and students in learning and teaching regarding Third-party Logistics Management.

Keywords

Novel Coronavirus, WeChat Work, Online Teaching Modes, Analytic Hierarchy Process, Fuzzy Comprehensive Evaluation Method

1. Introduction

It is universal that the quality of teaching plays an indispensable role in educational progress. In other words, the advantages and disadvantages of teaching directly affect the survival of universities as well as the performance of teachers and students. To cope with the novel coronavirus epidemic, the majority of universities begin to adopt online teaching modes so as to accomplish the corresponding tasks for teachers and students. The main reasons include the following points. On the one hand, it is the essential implementation of epidemic prevention and control. On the other hand, it is in response to the call entitled “ensuring learning undisrupted when classes are disrupted” proposed by Ministry of Education of the People’s Republic of China. However, the quality of large-scale online teaching is difficult to be guaranteed due to various reasons in practice. Additionally, there are many different online teaching modes, such as voice teaching, personal live recording teaching, MOOC, and so on. As a result, how to use effective methods to identify the best mode to ensure the quality of teaching is currently an urgent and important issue for online education in the context of the coronavirus epidemic.

In recent years, many scholars regardless of at home and abroad conducted a series of explorations concerning the issue related to online teaching modes. For instance, Ni (2013) claimed the persistence in an online environment may be more challenging in research methods classes than in other public administration classes. Similarly, Tynan et al. (2015) contended that since E-learning would increase teaching workload, they established a workload allocation model with considering contemporary teaching mode. Brancaccio-Taras et al. (2016) thought that the webinar-based teaching theme provided participants with a sense of being part of the educational community and more confidence in various teaching modes. Wang et al. (2020) established an online teaching system of medical immunology based on small private online course (SPOC) model, which was supplemented by “video courses as the main body, online Q&A as the supplement”. In addition, Chen (2015) used literature research, case analysis and questionnaire survey methods to identify the elements of the flipped classroom teaching mode based on micro video. Li (2016) proposed an EVA evaluation model to assess the enterprise value of online education. Liu & Zhou (2019) also developed BP neural network model to investigate teaching quality. Xu (2018) applied AHP to study online teaching performance of higher vocational colleges.

By reviewing the existing literature, it is reported that the previous studies mainly conducted comparative analysis and performance evaluation regarding online education platforms. It is difficult to investigate more detailed aspects of online teaching modes. Furthermore, most the foregoing studies mainly used qualitative analysis approaches. No one can deny the fact that AHP or fuzzy comprehensive evaluation is rarely employed to assess online teaching modes. In this context, given the current problems in the field of online teaching, this paper takes WeChat Work as an example to study the performance of four teaching modes. They are respectively voice, conference, live broadcast, and video mode. Firstly, this paper adopts AHP to determine the weights of indicators to evaluate online teaching modes in WeChat Work. Then, fuzzy comprehensive evaluation method is used to deal with fuzzy conceptions that cannot be accurately defined by numbers, thus to a certain extent eliminating the subjectivity and one-sidedness caused by the qualitative method (Ning, 2005). Finally, the best online teaching mode for Third-party Logistics Management in Chongqing Technology and Business University is found.

The arrangement of the rest of this paper is as follows. Section 2 introduces the indicators to assess online teaching modes on WeChat Work. In Section 3, AHP is used to determine the weights of the indicators to evaluate online teaching modes. Section 4 presents the application of fuzzy comprehensive evaluation into online teaching modes on WeChat Work for Third-party Logistics Management. Section 5 concludes this paper.

2. Construction of the Indicators to Assess Online Teaching Modes on WeChat Work

2.1. Online Teaching Modes on WeChat Work

As mentioned above, this paper takes the course titled “Third-party Logistics Management” in Chongqing Technology and Business University as an example. This course is regarded as the core course of Logistics Management discipline. The most popular mode of teaching is offline. However, with the evolving of the novel coronavirus epidemic, a new teaching mode differing from the traditional one is necessary. According to a recent survey, it can be seen that this course adopts online teaching platform, which is WeChat Work. In addition, it is widely used in a plenty of courses in Chongqing Technology and Business University. The main reason is that it provides teachers and students with a wide range of advantages and resources. For example, it can be applied for free and safety.

In particular, WeChat Work has four online teaching modes, namely voice, video, live broadcast, and conference. Although all the modes have the same basic functions, they have different advantages from different perspectives such as interaction, internet stability, playback status, clarity of sound quality and operational convenience. Besides, an appropriate mode is conducive to enhance the quality of online teaching regarding Third-party Logistics Management. Therefore, how to choose the best mode from the four ones for this course is significantly critical.

2.2. Evaluation Indicators System Regarding Online Teaching Modes

Practical cases and theoretical studies demonstrate that there are many factors to influence the performance of online teaching platforms. In this paper, the insights of the existing literature are leveraged and extended to construct the indictors (Liu & Peng, 2020; Li et al., 2020b). Interaction, internet stability, playback status, clarity of sound quality and operational convenience are finally selected to evaluate the performance of different online teaching modes. Specifically, interaction refers to a series of activities including online tests, wheat-based quizzes, speaking, and so on. It can make online teaching more realistic, supervise the behaviors of the students, and improve the performance of online learning. Internet stability is the key factor to determine which mode should be chosen on WeChat Work. For instance, video mode poses higher requirements on internet. Interrupt would be occurred if internet is in a bad state. On the contrary, such situation is seldomly found in voice mode. In terms of live playback, it can help students who do not attend classes in time and study hard in the class to learn what the teachers teach and review the knowledge. With respect to clarity of sound quality, it would exert positively influences on the perceived satisfaction of students towards the online teaching platform. Moreover, operational convenience also has the impacts on the selection of online teaching modes.

In order to make the established indicators system more reasonable, the questionnaire regarding this topic is also conducted among the students who admitted in 2018, majoring Logistics Management in Chongqing Technology and Business University. In detail, 57 effective questionnaires are returned to check the importance of the proposed indicators. In addition, other relevant data are also used in this paper.

3. Weights of Determination of the Indicators to Assess Online Teaching Modes on WeChat Work Based on AHP

3.1. Procedure of Determining the Weights of Indicators

Analytic Hierarchy Process (AHP) is an evaluation method which is proposed by Saaty in 1970s (Yang & Mu, 2010). It is widely used in various fields including economy, management, and so on. In terms of online teaching modes assessment on WeChat Work, the weights of the indicators cannot be quantitatively and objectively determined based on fuzzy comprehensive evaluation. To deal with such case, AHP is applied to determine the weights of the indicators. The specific procedure is as follows.

Step 1: Establish a hierarchical structure model. Target level D is the goal of evaluation, which aims to choose the best mode from WeChat work. Criterion layer B concerns and includes all indicators to assess different online teaching modes. That is, interaction, internet stability, playback status, clarity of sound quality and operational convenience are all considered here. Layer C contains all potential schemes, such as voice, video, live broadcast, conference.

Step 2: Construct the judgment matrixes. The relative importance between any two indicators for target and criterion layer based on 1 - 9 scale method (Cao, 2012). Each element in judgement matrix can be denoted by $a_{ij}$. Thus, all elements are integrated into a judgement matrix (namely $A={\left(a_{ij}\right)}_{n\times n}$ ).

Step 3: Calculate the relative weights and the eigenvalues. A square root is used to calculate the weights of each indicator. And then, eigenvalues of each index can be found. Thus, the maximal eigenvalues can be determined, which is denoted by ${\lambda }_{\max }$.

Step 4: Consistency check for each judgement matrix. In order to avoid the unexpected cases, consistency check should be conducted. Only if the consistency ratio is located within certain a level, the corresponding results is reasonable. The specific process can be concluded as follows.

1) Consistency index (C.I.)

$C.I.=\frac{{\lambda }_{\max }-n}{n-1}$ (1)

Particularly, if C.I. = 0, it indicates that it is completely consistent.

2) Another important term is the average random consistency index (R.I.), which is always known. The corresponding value is shown in Table 1.

3) The consistency ratio is calculated based on the following equation.

$C.R.=\frac{C.I.}{R.I.}$ (2)

What should be addressed is that if C.R. = 0, it indicates that it is completely consistent. The results are recommended to be accepted when C.I. < 0.1, otherwise, the judgement matrix should be reconstructed.

3.2. Determination of the Weights of the Indicators for Third-Party Logistics Management

Based on the recent survey, all elements in the judgment matrix A can be obtained.

$A=\left[\begin{array}{ccccc}1& 2& 2& 1& 1\\ \frac{1}{2}& 1& 1& \frac{1}{2}& \frac{1}{2}\\ \frac{1}{2}& 1& 1& \frac{1}{2}& \frac{1}{2}\\ 1& 2& 2& 1& 1\\ 1& 2& 2& 1& 1\end{array}\right]$

According to the judgment matrix A, the weights of the indicators and eigenvalues are obtained by using AHP, which is depicted in Table 2.

Table 1. Average random consistency index.

Table 2. The weights of indicators to evaluate online teaching modes on WeChat work.

Then, consistency ratio can be calculated by combing with the aforementioned results in Table 2. More specifically, C.R. = 0 < 0.1. In other words, the consistency check of judgement matrix A is passed. Finally, the corresponding weights of the indicators are as follows. w⁰ = [0.2500, 0.1250, 0.1250, 0.2500, 0.2500].

4. Application of Fuzzy Comprehensive Evaluation into Online Teaching Modes on WeChat Work

4.1. Fuzzy Comprehensive Evaluation Method

The idea of fuzzy comprehensive evaluation method which is proposed by American auto-control expert in 1965 originate from fuzzy mathematics. It aims to explicitly and systematically solve the problems that cannot be quantified in some terms (Li et al., 2020a). This paper uses fuzzy comprehensive evaluation method to assess different teaching modes on WeChat Work. Specifically, a series of factors should be pre-determined, thus forming factor set that is denoted by U = [U₁, U₂, U₃, U₄, U₅]. And then, the comment set is also established as V = [V₁, V₂, V₃, V₄, V₅]. Next, all experts assess each indicator with certain a comment. Finally, the overall scores of each online teaching mode on WeChat Work can be obtained.

4.2. Procedure of Assessing Online Teaching Modes Based on Fuzzy Comprehensive Evaluation Method

The specific procedure of assessing online teaching modes adopted in Third-party Logistics Management is as follows.

Step 1: Build a set of evaluation factors or indicators. It is obvious that the factors that affect the performance of online teaching modes have been obtained in Subsection 2.2. Hence, let “U” denote factor set, which is depicted in Table 3.

Step 2: Establish the corresponding comment sets for each factor or indicator. In a similar way, it is also constructed based on the questionnaire. Besides, let ‘V’ represent comment set. Meanwhile, the comment set of each factor is subdivided into five level. In other words, V = [very poor, poor, average, good, very good].

Step 3: Determine the weights of each factor or indicator. As we know, it can be obtained from Table 2 in Subsection 3.2.

Table 3. Factor set for each online teaching mode.

Step 4: Establish membership matrix for each online teaching mode on WeChat Work. All membership matrixes are generated once the experts accomplished the judgement for each indicator. Let “R” denote the membership matrix, which is written as $R={\left(r_{ij}\right)}_{m\times k}$. Wherein, $r_{ij}$ represent the membership of factorion level j of comment set. In addition, $m$ is the number of indicators, and k denotes the number of levels of the comment.

Step 5: Calculate comprehensive membership vector. In terms of each level of comment, the overall evaluation can be achieved. Finally, the comprehensive membership vector is obtained. The equation is denoted as follows.

$Q=w^{0}R=\left[w_1^0,w_2^0,w_3^0,w_4^0,w_5^0\right]\times \left[\begin{array}{ccc}{r}_{11}& \cdots & r_{1k}\\ ⋮& \ddots & ⋮\\ r_{m1}& \cdots & r_{mk}\end{array}\right]$

Wherein, Q represents comprehensive membership vector, R denotes the normalized membership matrixes.

Step 6: Calculate the overall scores and choose the best mode. Firstly, different scores are assigned to the different levels of the comments. That is, $P=\left[1,2,3,4,5\right]$. And then, the overall scores can be obtained by the product of comprehensive membership vector and the scores of the comment. It is denoted as follows.

$L=Q\times P^T$ (3)

4.3. Evaluation of Online Teaching Modes on WeChat Work for Third-Party Logistics Management

Based on the results of the previous survey, the following membership matrixes for each online teaching mode on WeChat Work for Third-party Logistics Management are obtained. It is depicted in Table 4.

According to Table 4, the comprehensive membership vector for each online teaching mode on WeChat Work. In this context, it can be calculated by:

$Q_1=w^0\times R_1=\left[0.1074,0.1135,0.1991,0.3542,0.2176\right]$

$Q_2=w^0\times R_2=\left[0.1019,0.1134,0.2384,0.1821,0.2130\right]$

$Q_3=w^0\times R_3=\left[0.0648,0.0695,0.2640,0.3241,0.2778\right]$

$Q_4=w^0\times R_4=\left[0.0782,0.1065,0.2153,0.3310,0.2523\right]$

Wherein, R₁, R₂, R₃, R₄ respectively denote the membership matrix of different online teaching modes including voice, video, live broadcast, and conference on WeChat Work. Finally, the ultimate scores of each online teaching mode can be obtained.

$L_1=Q_1\times P^T=3.4361$

$L_2=Q_2\times P^T=2.8373$

$L_3=Q_3\times P^T=3.6812$

$L_4=Q_4\times P^T=3.5226$

Table 4. Membership matrixes for each online teaching mode based on questionnaire.

According to the foregoing results, the following conclusions can be drawn. Live broadcast is the best mode for teachers and students on WeChat Work. The recommended order of the rest of online teaching modes is as follows: conference, voice and video.

Results indicated that live broadcast was the best online teaching mode for teachers for Third-party Logistics Management in the context of the novel coronavirus epidemic. In addition, such mode can impose favorable influences on preparing lessons, addressing a lecture, and immediately answering the questions of students, thus making up for the disadvantages of online teaching. Furthermore, it can also exert desirable impacts on encouraging the students to give a feedback on this course in a timely manner. Besides, students can fill in the gaps through live broadcast playback after class to consolidate the knowledge. Therefore, in order to ensure the perfect performance of online teaching regarding Third-party Logistics Management, it is recommended that both teachers and students adopt live broadcast on WeChat Work.

5. Conclusion

This paper concentrates on the application of AHP and fuzzy comprehensive evaluation method into the evaluation of online teaching modes on WeChat Work in terms of Third-party Logistics Management. Results indicated that live broadcast may achieve the best performance for online teaching. In this sense, this mode is recommended to both teachers and students. In accordance with the recent survey, playback option does not be included in voice, video and conference mode on WeChat Work, which cannot furnish students with a wide range of resource. Increasingly, it has the potential advantages in the aspects of interaction, participation of students, immediate feedback by comparing with other three online teaching modes. Although live broadcast mode is superior to others according to the current results, some issues are still open. In the future, the results and conclusions should be validated by more data from different groups of students and different courses in different courses.

Acknowledgements

This paper was supported by Education and Teaching Reform Research Project of Chongqing Municipal Education Commission Grant No. 171606, Graduate Education and Teaching Reform Research Project of Chongqing Technology and Business University Grant No. 2015YJG0106.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Brancaccio-Taras, L., Gull, K. A., & Ratti, C. (2016). The Science Teaching Fellows Program: A Model for Online Faculty Development of Early Career Scientists Interested in Teaching. Journal of Microbiology & Biology Education, 17, 333-338. https://doi.org/10.1128/jmbe.v17i3.1243
[2]	Cao, M. L. (2012). Hierarchical Analysis Determines the Weight of Evaluation Indicators and Excel Calculations. Jiangsu Science and Technology Information, No. 2, 39-40. (In Chinese)
[3]	Chen, H. Y. (2015). Research on the Flip Classroom Teaching Mode Based on Microvideo. Shandong: Shandong Normal University. (In Chinese)
[4]	Li, J., Zu, J. Y., Wang, Y. C. et al. (2020a). Children’s Early Education Machine Design Based on AHP and Fuzzy Comprehensive Evaluation Methods. Packaging Works, 1-5. (In Chinese)
[5]	Li, X. H. (2016). Research on the value evaluation method of online education enterprises. Beijing: Beijing Jiaotong University. (In Chinese)
[6]	Li, X. H., Liu, Z. T., Tang, M. H. et al. (2020b). Online Teaching Discovery Based on Teaching Cube and Live Video. University Chemistry, 1-5. (In Chinese)
[7]	Liu, Q., & Peng, N. (2020). Demonstration of Online Teaching Based on “Superstar Live Broadcast + Learning Pass”. University Chemistry, 1-6. (In Chinese)
[8]	Liu, Y. W., & Zhou, J. Q. (2019). Study on the Teaching Quality Evaluation Method of the BP Neural Network Model Combined with the Main Component Analysis. Mapping Geographic Information, 44, 107-109. (In Chinese)
[9]	Ni, A. Y. (2013). Comparing the Effectiveness of Classroom and Online Learning: Teaching Research Methods. Journal of Public Affairs Education, 19, 199-215. https://doi.org/10.1080/15236803.2013.12001730
[10]	Ning, X. Q. (2005). Fuzzy Mathematical Principles and Methods. Xuzhou: China Mining University Press. (In Chinese)
[11]	Tynan, B., Ryan, Y., & Lamont-Mills, A. (2015). Examining Workload Models in Online and Blended Teaching. British Journal of Educational Technology, 46, 5-15. https://doi.org/10.1111/bjet.12111
[12]	Wang, W., Zhang, Y. S. et al. (2020). Exploration and Practice of Online Teaching System of Medical Immunology Based on Small Private Online Course (SPOC) Model during Coronavirus Disease 2019 (COVID-19) Pandemic. Chinese Journal of Cellular and Molecular Immunology, 376-382.
[13]	Xu, W. (2018). Research and Practice of Online Education Performance Evaluation Model in Higher Vocational Colleges. Zhejiang: Zhejiang University of Technology. (In Chinese)
[14]	Yang, Z., & Mu, Z.J. (2010). Study on the Evaluation of Teaching Quality of College Teachers Based on Hierarchical Analysis. Journal of Inner Mongolia University of Technology (Social Science Edition), 19, 111-115-122. (In Chinese).