Evaluating Pedagogical Quality of Learning Activities Using Fuzzy Evaluation Mappings: The Case of Pedagogical Games of Mathematical Proof ()

Pascal Stiefenhofer^{}

Newcastle University Business School, Newcastle upon Tyne, UK.

**DOI: **10.4236/am.2022.135029
PDF HTML XML
65
Downloads
269
Views
Citations

Newcastle University Business School, Newcastle upon Tyne, UK.

This paper introduces a conceptual framework for developing pedagogical games of mathematical proof (PGMP) designed to help non-STEM students learn mathematical reasoning in a playful manner and without “fear”. Within the constructivist learning paradigm it develops an in-class learning activity where social science students learn the concept of mathematical deduction playfully using toys to construct tables from which formal proofs of propositions are derived without calculations. A PGMP pedagogy quality assurance evaluation method based on fuzzy evaluation mappings capturing learning effectiveness, learning efficiency, and learning satisfaction is introduced. Our results from an in-class experiment show that pedagogical games of mathematical proof help non-STEM students to effectively engage with mathematical reasoning playfully. The results are consistent according to a quality assurance consistency index.

Share and Cite:

Stiefenhofer, P. (2022) Evaluating Pedagogical Quality of Learning Activities Using Fuzzy Evaluation Mappings: The Case of Pedagogical Games of Mathematical Proof. *Applied Mathematics*, **13**, 432-452. doi: 10.4236/am.2022.135029.

1. Introduction

Self-efficacy is a student’s set of beliefs in her or his capacity to successfully execute a behavior necessary to produce a positive learning performance attainment [1] [2] [3]. Students with high self-efficacy beliefs allocate more effort when they encounter learning difficulties compared to students with low self-efficacy who often don’t make any notable attempt to achieve a learning goal, as they a priori belief that they will fail to achieve it [4] [5].

A unique type of anxiety is mathematics and statistics anxiety. It is unique in this regard, as there are no widespread anxiety conditions for other specific learning content areas such as reading, writing, or history^{1}. Mathematics and statistics anxiety is a negative emotional reaction to the manipulation of numbers and symbols that can be debilitating. It has been defined as “a feeling of tension and anxiety that interferes with the manipulation of numbers and the solving of mathematical problems in…ordinary life and academic situations [6] [7].” Recent research shows that a key factor negatively impacting mathematics anxiety is self-efficacy beliefs toward learning mathematics [8] [9]. This finding is in accordance with the observation that students with lower self-efficacy beliefs toward learning mathematics have higher levels of mathematics anxiety [10] [11] [12] [13] [14], a relation consistent across gender groups, grade-level groups, ethnic groups, and instruments measuring mathematics anxiety [15] [16].

To help students overcome anxious feelings toward mathematics educators recently started to incorporate active learning elements in their instruction with pedagogical successes reported in attitudes towards mathematics studies [16] [17] [18] [19]. Active learning in mathematics is a recent pedagogical development in response to a “call on institutions of higher education, ..., to ensure that effective active learning is incorporated into post-secondary mathematics classrooms”.^{2} Prior to this call a land marking paper showed in a meta analysis of 225 studies that active learning improves learning and course performance in undergraduate STEM courses relative to traditional lecturing [20]. This research further suggests that active learning does not harm, and may further benefit, already high-achieving students. It also shows that students in classes with traditional lecturing were more likely to fail compared to students in classes with active learning. Active learning helps students effectively engage in their learning process and improve their performance attainment [21]. The basic premises of active learning include instructional techniques reinforcing higher-order thinking, requiring students to co-create their own knowledge through active participation in the learning process [22] [23].

Game based learning is an emerging active learning instructional method in mathematics with several advantages [24]: it increases student engagement [25] [26], enhances self-efficacy [18], improves student achievement [27], motivation and resilience [28]. The literature on pedagogical games supporting teaching and learning of mathematics in higher education courses is scarce, and largely focuses on online settings [29] [30]. The game introduced in this paper, however, uses a “toy” to help students put hands on mathematics to physically flip and or rotate wooden objects to fit their counter-shape holes in a box. Students then write down their observations into a table from which mathematical proofs can be deduced. Even advanced mathematical concepts such as Abelian group properties of symmetries can be deduced from this method by playfully constructing a Cayley table. The activity considered here, enables students to derive proofs without calculations. There is currently little evidence in the literature about the effectiveness of using toys as an active learning activity to derive proofs “playfully”. This paper sets out to investigate this by developing a formal framework for constructing pedagogical games of mathematical proof and their pedagogical quality evaluations.

It is generally difficult to determine if a pedagogical game is devised with the explicit intention of meeting educational criteria, or whether researchers are appropriating existing games to test their possible educational impact [29] [31] [32]. Moreover, how would one compare the effectiveness of alternative pedagogies relative to each other? Naik (2017) suggests that the effectiveness of pedagogical games should ultimately be measured in terms of student learning outcomes [29]. In this paper, we consider a geometric approach based on fuzzy mappings to establish a pedagogical quality assurance system in which pedagogical effectiveness is only one dimension. We determine the quality of a pedagogical game by considering three main learning dimensions including: learning effectiveness, learning efficiency, and learning satisfaction. We utilize the concepts of fuzzy variables and metric functions to obtain a pedagogical quality assurance evaluation method.

The organization of this paper is as follows. Section two provides a formal framework for developing pedagogical games of mathematical proof. We apply this pedagogical framework in section four where we provide an example. Section three introduces the PGMP pedagogical quality assurance evaluation model using fuzzy variables. It provides a process for calculating a pedagogical quality measure supported by a consistency measure to judge pedagogical quality assurance of PGMPs. Section four provides the results of an in-class experiment utilizing both, the PGMP framework and its pedagogical evaluation model. Section five is a conclusion followed by an appendix.

2. Constructing a PGMP Learning Activity

We define a pedagogical game of mathematical proof expressed as

${\models}_{A}{\left(\Gamma \u22a2\varphi \right)}_{G}\mathrm{.}$ (1)

A pedagogical game of mathematical proof, hence, consists of a teaching model
${\models}_{A}$ defined by a set of pedagogies *A*, a set of rules *G* defining the game structure, and the mathematical proof
$\Gamma \u22a2\varphi $ , consisting of a set of assumptions
$\Gamma $ , a true statement
$\varphi $ or set of statements
$\Phi $ , and the method of deduction
$\u22a2$ . Table 1 below summarizes the steps in the development of a PGMP as described in (1).

Implicit assumptions: Students have been introduced to basic abstract algebra and are familiar with the concepts of a group, Abelian group, and (non-)symmetry of mathematical objects such as those shown in Figure 1.

The theoretical model summarized in Table 1 requires educators to define the

Table 1. How to construct a PGMP.

Figure 1. PGMP “Toy”.

sets
$\Gamma \mathrm{,}\Phi \mathrm{,}G$ , and *A*. We use the notation
$\Phi $ to indicate a set of mathematical statements
$\Phi \mathrm{:}=\left\{{\varphi}_{1}\mathrm{,}{\varphi}_{2}\mathrm{,}\cdots \mathrm{,}{\varphi}_{n}\right\}$ . For example
$\Phi $ may represent a set of exercises, and
${\varphi}_{i}$ a specific statement
$i=1,2,\cdots ,n$ to be proven.

Step 1: Define $\Gamma $ and $\Phi $ , and formulate $\left(\Gamma \u22a2\varphi \right)$ as in Equation (1). We define the premises $\Gamma $ . That is

$\Gamma \mathrm{:}=\left\{{\gamma}_{1}\mathrm{,}\cdots \mathrm{,}{\gamma}_{4}\right\}\mathrm{,}$ (2)

where

${\gamma}_{1}$ : $\Delta $ is an equilateral triangle.

${\gamma}_{2}$ : $\square $ is a box with a whole $\Delta $ in it such that a block $\Delta $ fits in $\square $ . See Figure 1.

${\gamma}_{3}$ : There are two transformations called rotation $\Theta $ and flip $\Psi $ .

${\gamma}_{4}$ : There is a composition $\circ $ of transformations $\Theta $ and $\Psi $ acting on the block $\Delta $ .

Next, let there be a set of statements to be proven, where each statement ${\varphi}_{i}$ corresponds to an exercise.

$\Phi \mathrm{:}=\left\{{\varphi}_{1}\mathrm{,}\cdots \mathrm{,}{\varphi}_{6}\right\}\mathrm{,}$ (3)

where

${\varphi}_{1}$ : $\Delta $ fits in $\square $ in 6 different ways.

${\varphi}_{2}$ : $\Delta ,\text{\hspace{0.17em}}\square \mathrm{,}\Theta \mathrm{,}\Psi $ and $\circ $ form a group $\left(\mathcal{G}\mathrm{,}\ast \right)$ .

${\varphi}_{3}$ : $\Delta \mathrm{,}\text{\hspace{0.17em}}\square \mathrm{,}\Theta $ and $\circ $ form subgroups $\left({\mathcal{S}}_{i}\text{,}\ast \right)$ of $\left(\mathcal{G}\mathrm{,}\ast \right)$ , for some $i=1,2,\cdots ,n$ .

${\varphi}_{4}$ : $\left({\mathcal{S}}_{i}\text{,}\ast \right)$ are Abelian.

${\varphi}_{5}$ : $\Delta ,\text{\hspace{0.17em}}\square \mathrm{,}\Theta \mathrm{,}\Psi $ and $\circ $ form subgroups ${\mathcal{T}}_{i}$ of $\left(\mathcal{G}\mathrm{,}\ast \right)$ , for some $i=1,2,\cdots ,n$ .

${\varphi}_{5}$ : $\left({\mathcal{T}}_{i}\mathrm{,}\ast \right)$ are Abelian.

${\varphi}_{6}$ : $\left(\mathcal{G}\mathrm{,}\ast \right)$ is not Abelian.

Step 2: Choose *G* over
$\left(\Gamma \u22a2\varphi \right)$ . This requires to define the set of rules of the mathematical game.

$G\mathrm{:}=\left\{{g}_{1}\mathrm{,}{g}_{2}\mathrm{,}{g}_{3}\right\}\mathrm{,}$ (4)

where

${g}_{1}$ : $\Theta $ is a right (or left) rotation of 120 degrees with fixed center in $\Delta $ .

${g}_{2}$ : $\Psi $ is a right or left flip of 180 degrees with fixed center in $\Delta $ (imagine a line from any vortex through the center of $\Delta $ ).

${g}_{3}$ : $\circ $ is a composition of transformations acting on $\Delta $ .

The definition of the set *G* must be in accordance with the guidelines set out in the pedagogy *A*. In our case, the elements of *G* are supposed to support a constructivist learning approach, where students actively co-create knowledge by physically rotating and flipping a block
$\Delta $ to match a hole of same shape
$\Delta $ in a box
$\square $ to derive a table of all possible successful combinations of flips and rotations of
$\Delta $ from which proofs are derived.

Step 3: We now define the pedagogy of our game of mathematical proof. With an increasing awareness that many undergraduates are passive during seminar sessions [33] [34], we define a learning activity that allow students to actively engage in co-creating knowledge. Each student is provided a wooden block (an equilateral triangle)
$\Delta $ and a wooden box
$\square $ with a hole of shape
$\Delta $ in it. The student is then asked to use the building block to physically rotate and flip it to fit the hole and to therewith derive symmetries and present observations in form of a table from which proofs of propositions are then derived. We defined the set *A* which includes the following conditions.

1) Students are provided with a wooden box and a symmetric wooden object as shown in Figure 1.

2) Students are provided with a problem set which contains a set of instructions for each problem (Appendix).

3) Students work independently on the problem set (single player game).

4) Students are provided with 50 minutes of time to complete a set of tasks (Theorems 1 and 2 in Appendix).

5) Students can ask the instructor for definitions (loosing 5 points each).

6) Students compete with each other on time (loosing 5 points each rank) and points (each task is worth some points).

7) A total point score is calculated and appropriate reductions are made. The winner is announced to the class.

The pedagogical model ${\models}_{A}$ consists of a mixed pedagogy including elements from active learning and competitive learning both based within the constructivist paradigm with students co-creating their own knowledge playfully.

3. Pedagogy Quality Assurance Evaluation Method

In order to assess the usefulness of a PGMP as a learning activity we develop a pedagogy quality assurance evaluation method (PEM). We construct fuzzy variables [35] combined with a hierarchical model [36] to obtain a numerical value expressing pedagogical quality [37] [38] [39]. In our PEM, quality is defined as a weighted average of learning effectiveness, learning efficiency, and learning satisfaction as shown in Figure 2.

Let there be *n* evaluation factors represented by a set

$U=\left\{{u}_{1}\mathrm{,}{u}_{2}\mathrm{,}\cdots \mathrm{,}{u}_{n}\right\}\mathrm{,}$ (5)

where
${u}_{i}$ is the *i*^{th} evaluation factor in
$u\in U$ and
$u=\left({u}_{1},{u}_{2},\cdots ,{u}_{n}\right)$ a vector. There are *m* levels of appraisal grades represented by a set

$V=\left\{{v}_{1}\mathrm{,}{v}_{2}\mathrm{,}\cdots \mathrm{,}{v}_{m}\right\}\mathrm{,}$ (6)

where
${v}_{k}$ is the *k*^{th} appraisal grade in
$v\in V$ on a Likert scale and
$v=\left({v}_{1}\mathrm{,}{v}_{2}\mathrm{,}\cdots \mathrm{,}{v}_{m}\right)$ is a vector.
${v}_{1}$ represents “strongly disagree” and gradually increasing to
${v}_{m}$ representing “strongly agree”. A mapping
$U\to V$ is a fuzzy evaluation mapping if for each evaluation factor
${u}_{i}\in U$ there is a mapping

${\mu}_{{\Pi}_{i}}\mathrm{:}U\to \left[\mathrm{0,1}\right]$ (7)

where
${\Pi}_{i}$ is a fuzzy set associated with evaluation factor
${u}_{i}\in U$ . Alternatively, for every
${u}_{i}\in U$ the mapping
${\Pi}_{i}\mathrm{:}U\to \left[\mathrm{0,1}\right]$ yields a row vector
${\pi}_{i}=\left({\pi}_{i1}\mathrm{,}{\pi}_{i2}\mathrm{,}\cdots \mathrm{,}{\pi}_{im}\right)$ , where
${\pi}_{ik}$ represents the fuzzy membership degree of appraisal factor *i* to grade *k*. The general fuzzy appraisal matrix
$\left[{\pi}_{ik}\right]$ for all evaluation factors
$i=1,\cdots ,n$ and appraisal grades
$k=1,\cdots ,m$ is denoted by

${\Pi}_{\left(n\times m\right)}=\left[\begin{array}{cccc}{\pi}_{11}& {\pi}_{12}& \cdots & {\pi}_{1m}\\ {\pi}_{21}& {\pi}_{22}& \cdots & {\pi}_{2m}\\ \vdots & \vdots & \ddots & \vdots \\ {\pi}_{n1}& {\pi}_{n2}& \cdots & {\pi}_{nm}\end{array}\right]\mathrm{.}$ (8)

For simplicity, we employ a triangular distribution function $\mu $ in the construction of the mapping $\Pi $ characterizing the fuzzy measure values ${\pi}_{ik}$ for $i=1,2,\cdots ,n$ and $j=1,2,\cdots ,m$ [37] [38]. In order to obtain a comprehensive

Figure 2. Pedagogy quality assurance evaluation method.

overall quality assurance evaluation vector
${Q}_{\left(1\times m\right)}=\left({q}_{1},{q}_{2},\cdots ,{q}_{m}\right)$ we construct a weight vector
${W}_{\left(n\times 1\right)}$ by the AHP method [36] assigning relative weight
${w}_{i}$ to evaluation factor
${u}_{i}\in U$ for every
$i=1,2,\cdots ,n$ . The overall quality assurance evaluation vector *Q* is obtained by

$Q=W\Pi ,$ (9)

where

${q}_{k}=\mathrm{min}\left\{1,{\displaystyle \underset{i=1}{\overset{n}{\sum}}}\text{\hspace{0.05em}}\text{\hspace{0.05em}}{W}_{i}{\pi}_{ik}\right\},\text{\hspace{0.17em}}\text{\hspace{0.17em}}\text{for}\text{\hspace{0.17em}}k=1,\cdots ,m.$ (10)

Figure 2 outlines the overall methodology of the quality assurance evaluation process. Pedagogical quality is constructed as a measure of learning performance, defined by the degree of which students can successfully perform tasks within given time constraints, and learning satisfaction. A Post-Study System Usability Questionnaire (PSSUQ) ( [40], p.75-77) with respect to all three dimensions is employed to obtain survey data. The goal of this quality assurance system is to provide educators with a measure for the overall quality *Q* of a pedagogy.

We begin by constructing the fuzzy sets
${\Pi}_{i}$ from student survey (PSSUQ) and expert survey data. The later can be obtained via qualitative interview with subject experts. For simplicity, we assume a triangular distribution function
${\mu}_{i}$ to construct the mapping functions characterizing fuzzy measure values
$\left[{\pi}_{ik}\right]$ . We conduct a focus group with education experts to obtain the data points
${v}_{i1}\mathrm{,}{v}_{i2}\mathrm{,}\cdots \mathrm{,}{v}_{i8}$ for each evaluation factor
${u}_{i}$ in the product quality evaluation factor set *U* and appraisal intervals
$\left({v}_{i1}\mathrm{,}{v}_{i2}\right)\mathrm{,}\left({v}_{i2}\mathrm{,}{v}_{i3}\right)\mathrm{,}\cdots \mathrm{,}\left({v}_{i7}\mathrm{,}{v}_{i8}\right)$ . Then for all
${\stackrel{\xaf}{v}}_{i}\in \left[{v}_{i1}\mathrm{,}\cdots ,{v}_{im}\right]$

${\mu}_{i}\left({\stackrel{\xaf}{v}}_{k}\right)=(\begin{array}{ll}{p}_{k}\left(\stackrel{\xaf}{v}-{v}_{k}\right)+\mathrm{1,}\hfill & \text{when}\text{\hspace{0.17em}}{\stackrel{\xaf}{v}}_{k}\in \left[{v}_{k}-\frac{1}{{p}_{k}},{v}_{k}\right]\hfill \\ {p}_{k}\left({v}_{k}-\stackrel{\xaf}{v}\right)+\mathrm{1,}\hfill & \text{when}\text{\hspace{0.17em}}{\stackrel{\xaf}{v}}_{k}\in \left[{v}_{k},{v}_{k}+\frac{1}{{p}_{k}}\right]\hfill \\ \mathrm{0,}\hfill & \text{otherwise}\mathrm{,}\hfill \end{array}$ (11)

where each ${\stackrel{\xaf}{v}}_{k}$ is some value provided by the student survey data (PSSUQ). From Equation (11) we use the values $\frac{{\mu}_{i}\left({\stackrel{\xaf}{v}}_{k}\right)}{{\displaystyle {\sum}_{k=1}^{m}}\text{\hspace{0.05em}}\text{\hspace{0.05em}}{\mu}_{ik}\left({v}_{ik}\right)}$ for $k=1,2,\cdots ,m$ to calculate the normalized appraisal vector ${\Pi}_{i}$ .

${\Pi}_{i}=\left\{{\pi}_{i1},{\pi}_{i2},\cdots ,{\pi}_{im}\right\},$ (12)

with appraisals grades
${v}_{i}\in V$ for evaluation factor *i*.
${\Pi}_{i}$ can then be normalized in the usual way yielding the values
${\pi}_{ik}$ .

Some factors
${u}_{i}\in U$ according to our product quality definition provided above contribute with different importance to the overall measure of pedagogical quality. We now discuss the construction of the vector *W* using the AHP method [41]. The AHP method requires experts to make pair-wise comparesons between evaluation factors
${A}_{1}\mathrm{,}{A}_{2}\mathrm{,}\cdots \mathrm{,}{A}_{n}$ and assigning numerical values
${a}_{ij}$ for
$i,j=1,2,\cdots ,n$ to them. This yields a square matrix

${A}_{\left(n\times n\right)}=\begin{array}{cc}& \begin{array}{cccc}{A}_{1}\text{\hspace{0.05em}}\text{\hspace{0.05em}}& {A}_{2}& \text{\hspace{0.05em}}\text{\hspace{0.05em}}\text{\hspace{0.05em}}\cdots & \text{\hspace{0.05em}}{A}_{n}\end{array}\\ \begin{array}{c}{A}_{1}\\ {A}_{2}\\ \vdots \\ {A}_{n}\end{array}& \left(\begin{array}{cccc}{a}_{1,1}& {a}_{1,2}& \cdots & {a}_{1,n}\\ {a}_{2,1}& {a}_{2,2}& \cdots & {a}_{2,n}\\ \vdots & \vdots & \ddots & \vdots \\ {a}_{n,1}& {a}_{n,2}& \cdots & {a}_{n,n}\end{array}\right)\end{array}$ (13)

of relative weights ${W}_{1}\mathrm{,}{W}_{2}\mathrm{,}\cdots \mathrm{,}{W}_{n}$ with the following properties:

1) ${a}_{ij}\approx \frac{{W}_{i}}{{W}_{j}}$ , for $i,j=1,2,\cdots ,n$ .

2) ${a}_{ii}=1$ , for all $i=1,2,\cdots ,n$ .

3) If ${a}_{ij}=\alpha $ for $\alpha \ne 0$ , then ${a}_{ji}=\frac{1}{\alpha}$ , for $i=1,2,\cdots ,n$ .

4) If ${A}_{i}$ is more important than ${A}_{j}$ then ${a}_{ij}\cong \left({W}_{i}/{W}_{j}\right)>1$ .

These properties yield a positive definite and reciprocal matrix with 1’s on the main diagonal. The experts only need to provide data for the upper triangle of the matrix, that is $L=n\left(n-1\right)/2$ data points. Property one above suggests a relation

$AW=nW$ (14)

One can use Saaty’s method to compute *W* [41]. The final step is to check that human judgments (expert data, *A*) are consistent. For that purpose, we calculate a consistency index (*CI*) measuring how a given matrix *A* compares to a purely random matrix in terms of their consistency indices. Let *CI* be the consistency index

$CI=\frac{{\lambda}_{\mathrm{max}}-n}{n-1}$ (15)

where *n* is the size of the pair-wise comparison matrix *A* and
${\lambda}_{\mathrm{max}}$ represents the maximum eigenvalue of the matrix. Let *RI* be the average random index which can be computed as shown in [41]. A matrix *A* is consistent if and only if
${\lambda}_{\mathrm{max}}=n$ and that we always have
${\lambda}_{\mathrm{max}}>n$ when matrix *A* is a positive reciprocal one. If CI is less than 0.1, the numerical judgments will be considered acceptable [41].

4. Results from an In-Class Experiment

An in-class experiment with 10 third year undergraduate economics students taking an advanced mathematics course was conducted at the University of Exeter. The student cohort attended two different one hour sessions plus 15 minutes introduction to the learning activity. In session one, they conducted the learning activity “Theorem 1” and after 7 days, session two covered learning activity “Theorem 2”^{3}. The precises activities are discussed in section three and the appendix of this paper. Student performance was incentivized with a competitive game element, distributing points to correct answers and penalizing students with negative points who requested definitions from the tutor. A total score was determined and the best performer was announced to the cohort.

Students reported Subjective Task Time, measured on a Likert scale 1 - 10 with 1 long subjective time perception and 10 short subjective time perception. Both Subjective Task Completion and Subjective Task Time was reported in a USQ survey [40], with Task Completion determined by a tutor who marked the completed tasks to obtain an overall assessment score for each student. Together, Task Time and Task Completion determine a weighted measure of Learning Performance.

A focus group consisting of five mathematics education experts was conducted to determine the bounds of the metric function in Equation (11). As a rule, experts had to jointly agree about the metric bounds, which were determined in a verbal 30 minutes debate after being introduced to the specifics of the experiment and PGMP. Expert decisions were purely based on the description of the PGMP and they were not privy to student survey results. The same cohort of experts also provided numerical values (matrix (17)), based on a pairwise comparison of the variables “fun to play and usability”, “instruction quality”, and “game quality” of the PGMP to yield the matrix *A* in Equation (13).

Table 2 shows the individual average task success and self evaluated task time data for both sets of exercises resulting in the proofs of theorems 1 and 2. Associated with each theorem, an USQ survey collected data on task completion and task completion time for six tasks (see Appendix) [40]. On average task success was 8.308 (Learning Effectiveness) with associated average task time 8.6 (Learning Efficiency) for theorem 1. For theorem 2, average task success was 8.617 (Learning Effectiveness) with associated average task time of 8.057 (Learning Efficiency). average task success was higher for theorem 2, students completed the six tasks While associated with theorem 1 faster. From this table an average

Table 2. Individual task success and task time for theorems 1 and 2.

Learning Performance of 8.454 for theorem 1 compared to 8.337 for theorem 2 suggests that students performed better on proving theorem 1.

Table 3 reports individual subject data from the PSSUQ survey on three variables: average “fun to play and usability” (PGMP play) 4.250, average instruction quality (InstQual) 4.129, and average game quality (PGMPQual) 4.100. From this table, an overall average Learning Satisfaction value of 4.160 is obtained.

We now determine the parameters of the fuzzy evaluation mapping given in Equation (11). For simplicity, we assume a triangular metric. From a focus group consisting of five mathematics education experts we obtain the boundary values of each fuzzy variable reported in Table 4. The experts agree to apply the same metric boundaries for Task Success and Task Time, but propose different metric

Table 3. Individual pedagogy satisfaction measured by PGMP play, InstQual, and PGMPQual.

Table 4. Expert metric evaluation matrix.

boundaries for PGMP Satisfaction. Figure 3 graphically shows the suggested metric for PGMP Satisfaction and Figure 4 graphically represents the metric for Task Success and Task Time and Table 5.

TableA1 (appendix) and Table5 report the individual fuzzy values for Task Success for Theorems I and II. TableA2 (appendix) and Table6 report the individual fuzzy values for Task Time for Theorems I and II.

From Table 5, Table 6 and Table 3 we obtain the matrix

$\Pi =\left(\begin{array}{ccccc}0& 0& 0.13462& 0.776883& 0.88495\\ 0& 0& 0.244403& 0.755597& 0\\ 0& 0& 0.177471& 0.75793& 0.064599\end{array}\right).$ (16)

The following matrix *A* is constructed with input from experts:

$\begin{array}{cc}& \begin{array}{ccc}\text{InstQual}& \text{GameQual}& \text{PGMPplay}\end{array}\\ \begin{array}{c}\underset{}{\overset{}{}}\text{InstQual}\\ \text{\hspace{0.17em}}\text{\hspace{0.05em}}\text{\hspace{0.05em}}\text{\hspace{0.05em}}\text{GameQual}\\ \underset{}{\overset{}{}}\text{PGMPplay}\end{array}& \left[\begin{array}{ccccccc}1\text{\hspace{0.17em}}\text{\hspace{0.17em}}& & & \frac{1}{2}\text{\hspace{0.17em}}\text{\hspace{0.17em}}& & & 2\\ 2\text{\hspace{0.17em}}\text{\hspace{0.17em}}& & & 1\text{\hspace{0.17em}}\text{\hspace{0.17em}}& & & 3\\ \frac{1}{2}\text{\hspace{0.17em}}\text{\hspace{0.17em}}& & & \frac{1}{3}\text{\hspace{0.17em}}\text{\hspace{0.17em}}& & & 1\end{array}\right]\end{array}$ (17)

Figure 3. Satisfaction metric.

Figure 4. Task success, and task time metric.

Table 5. Task success theorem 2.

Table 6. Task time theorem 2.

Using the average of normalized columns (ANC) method

${{a}^{\prime}}_{i}=\frac{1}{n}{\displaystyle \underset{j=1}{\overset{n}{\sum}}}\frac{{a}_{ij}}{{\displaystyle {\sum}_{i=1}^{n}{a}_{ij}}},\text{\hspace{0.17em}}\text{\hspace{0.17em}}\text{for}\text{\hspace{0.17em}}i,j=1,2,\cdots ,n.$ (18)

where
${a}_{ij}$ are the entries of the matrix *A* we obtain a normalized matrix

Table 7. Satisfaction evaluation.

${A}^{\prime}=\left(\begin{array}{ccc}0.2857& 0.2727& 0.3333\\ 0.5714& 0.5454& 0.5\\ 0.1429& 0.1818& 0.166\end{array}\right).$ (19)

Averaging ${A}^{\prime}$ over rows yields a vector ${W}_{\left(1\times n\right)}$ defined by

${w}_{j}=\frac{{\displaystyle {\sum}_{j=1}^{n}{a}_{ij}}}{n}\text{\hspace{0.17em}}\text{\hspace{0.17em}}\text{for}\text{\hspace{0.17em}}i\mathrm{,}j=\mathrm{1,2,}\cdots \mathrm{,}n\mathrm{.}$ (20)

Hence $W=\left(\mathrm{0.297,0.539,0.164}\right)$ . We calculate the overall evaluation vector

$Q=\left(\mathrm{0.297,0.539,0.164}\right)\cdot \left(\begin{array}{ccccc}0& 0& 0.13462& 0.776883& 0.88495\\ 0& 0& 0.244403& 0.755597& 0\\ 0& 0& 0.177471& 0.75793& 0.064599\end{array}\right)\mathrm{.}$ (21)

That is
$Q=\left(\mathrm{0,0,0.20,0.762,0.273}\right)$ . We defuzyfy *Q* to obtain an overall measure of quality
$\stackrel{\xaf}{Q}$ using

$\begin{array}{c}\stackrel{\xaf}{Q}=\frac{{\displaystyle {\sum}_{i=1}^{n}{b}_{i}^{2}{a}_{i}}}{{\displaystyle {\sum}_{i=1}^{n}{b}_{i}^{2}}}\\ =\frac{{0}^{2}\times 31+{0}^{2}\times 50+{0.2}^{2}\times 67+{0.762}^{2}\times 82+{0.273}^{2}\times 95}{{0.2}^{2}+{0.762}^{2}+{0.273}^{2}}\\ =82.551\end{array}$ (22)

where ${a}_{1}=31,{a}_{2}=50,{a}_{3}=67,{a}_{4}=82$ and ${a}_{5}=95$ is the appraisal vector in [38] representing “very poor”, “poor”, “medium”, “good”, “excellent”.

We now calculate the consistency of our result. That is

$AW=\left(\begin{array}{ccc}1& 1/2& 2\\ 2& 1& 3\\ 1/2& 1/3& 1\end{array}\right)\left(\begin{array}{c}0.2972\\ 0.539\\ 0.1641\end{array}\right)=\left(\begin{array}{c}0.8945\\ 1.625\\ 0.4922\end{array}\right).$ (23)

${\lambda}_{\mathrm{max}}={\displaystyle \underset{i=1}{\overset{3}{\sum}}}\frac{{\left(AW\right)}_{i}}{n{w}_{i}}=\frac{0.8945}{3\times 0.2972}+\frac{1.625}{3\times 0.539}+\frac{0.4922}{3\times 0.1641}=3.01$ (24)

$CI=\frac{3.01-3}{3-1}=\mathrm{0.05.}$ (25)

5. Conclusions

This paper designs a pedagogical game of mathematical proof to help non-STEM mathematics educators to produce pedagogically effective active learning exercises within the constructivist paradigm using toys and without tedious calculations supporting anxious students in deriving mathematical proofs playfully. The purpose of this learning activity is to help students improve self-efficacy and reduce mathematics anxiety.

An in-class experiment shows that our pedagogy based on PGMPs satisfies a pedagogical quality criterion. We derive this conclusion from our Pedagogy Quality Assurance Evaluation Method, which we introduce using fuzzy variables. Our model uses Learning Effectiveness, Learning Efficiency, and Learning Satisfaction as inputs to derive a weighted measure of pedagogical quality. The advantage of this measure of learning activity evaluation is that it relies on data from both, students and experts.

The data from the experiment show that students were effective in playfully producing mathematical proofs using a hands-on approach using toys demanding them to physically rotate and flip geometric objects, and write up their observations in a table. The table, together with a set of sequential instructions effectively guided students through the reasoning required to produce a complete proof. The success was not only reflected in high performance attainment but also in the survey responses.

To improve on the robustness of our main result, future work should consider different metric functions such as trapezoids, bell-shaped, and sigmoid functions to specifically take into account other pedagogical variables. An empirical experiment with a large group of mathematics education experts would provide more accurate metrics.

Obtaining a more specific measure of our learning activity of reducing mathematics anxiety would be a valuable result for many educators. A robust measure could be obtained from a field-experiment with a much larger subject pool incorporating established instruments for measuring mathematics anxiety [23] [24] [33] into our model.

Acknowledgement

This work was supported by the Research-Inspired Inquiry-Led (RIIL) projects fund, as part of the Exeter Education Incubator, and by the NUBS School Research Fund.

Appendix

Definition 1. *A symmetry** is any transformation of
$\Delta $ which leaves
$\Delta $ in an equivalent state *(−5 pts)

Theorem 1
${\varphi}_{1}$ :* There are *6* symmetries.*

Task 1. Label the edges of the triangle
$\Delta $ with the capitals *A*, *B*, and *C* in counterclockwise direction. Similarly, label the edges of the corresponding hole
$\Delta $ in the box
$\square $ so that
$A\Delta $ matches
$A\square $ ,
$B\Delta $ matches
$B\square $ , and
$C\Delta $ matches
$C\square $ . [20 pts]

Task 2. Check that if starting at position
$A\Delta $ matches
$A\square $ (*AA*) a rotation
$\Theta $ moves to the new position
$A\Delta $ matches
$B\square $ (*AB*). We express this compactly as
$r{\left(A\right)}_{AB}$ . Check that if starting at position
$A\Delta $ matches
$A\square $ (*AA*) with fixed (*AA*) a flip
$\Psi $ leaves
$A\Delta $ matches
$A\square $ unchanged, but moves
$B\Delta $ on
$B\square $ (*BB*) to
$B\Delta $ on
$C\square $ (*BC*) and
$C\Delta $ on
$C\square $ (*CC*) to
$C\Delta $ on
$B\square $ (*CB*). We use the notation
${f}_{A}$ to express this property. [15 pts]

Task 3. Construct a table to show all symmetries of $r{\left(A\right)}_{AA}\mathrm{,}r{\left(A\right)}_{AB}$ and $r{\left(A\right)}_{AC}$ only using rotations $\Theta $ and its compositions $\circ $ . [15 pts]

Task 4. Fix the edge (*AA*) and use a flip
$\Psi $ in isolation and in combination with rotations
$\Theta $ to obtain the symmetries of
${f}_{A}$ . Follow this procedure to obtain the symmetries associated with
${f}_{B}$ and
${f}_{C}$ and collect them in a table. [15 pts]

Task 5. Collect all symmetries from above tables to define the set $\mathcal{G}$ . [15pts]

Task 6. Use the results of tasks 1 - 5 in given order to formally write up the proof of theorem 1. [20 pts]

The expected result from task 6 is summarized in the proof bellow.

*Proof.* Label the vertices of
$\Delta $ counterclockwise starting from the top by *A*, *B* and *C*. Similarly on
$\square $ label the initial vertices of
$\Delta $ counterclockwise starting from the top by *A*, *B* and *C*, such that the labeling for
$\Delta $ matches the labeling on the
$\square $ , *i.e.*, *AA*, *BB* and *CC*.

We introduce two operations; a counterclockwise rotation around the center of
$\Delta $ and a flip of
$\Delta $ at a fixed vortex (*A*, *B*, or*C*). Let
$\Theta $ denote the counterclockwise rotation of
$\Delta $ from *i.e.* initial *A* to final *B* such that *A* on
$\Delta $ moves from position *A* to position *B* on the
$\square $ . This is compactly expressed as
$r{\left(A\right)}_{AB}$ . Let
$\Psi $ be a flip with fixed vortex *A* such that initial *B* moves to final *C* and initial *C* moves to final *B* while *A* remains unchanged. We compactly express this as
${f}_{A}$ .

The composition of two operations is denoted by $\circ $ . Then by definition of symmetry we obtain:

$r{\left(A\right)}_{AA}=\Theta \circ \Theta \circ \Theta ={\Theta}^{3}={n}_{0}\mathrm{,}\text{noaction}$

$r{\left(A\right)}_{AB}=\Theta $

$r{\left(A\right)}_{AC}=r{\left(A\right)}_{AB}\circ r{\left(A\right)}_{BC}={\Theta}^{2}.$

These are the symmetries obtained from only applying rotation to $\Delta $ . Notice that there are two ways to achieve $r{\left(A\right)}_{AA}$ ; i) apply rotation three times, or ii) do not take any action ${n}_{0}$ at all. Let’s generate the symmetries associated with flipping.

${f}_{A}=\Psi $

${f}_{B}=r{\left(A\right)}_{AB}\circ r{\left(A\right)}_{BC}\circ {f}_{A}={\Theta}^{2}\circ \Psi $

${f}_{C}=r{\left(A\right)}_{AB}\circ {f}_{A}=\Theta \circ \Psi .$

We collect all symmetries in a set $\mathcal{G}=\left\{{n}_{0}\mathrm{,}\Theta \mathrm{,}{\Theta}^{2}\mathrm{,}\Psi \mathrm{,}{\Theta}^{2}\Psi \mathrm{,}\Theta \Psi \right\}$ .

Definition 2. *A Latin square is an
$n\times n$ array filled with n different symbols*,* each occurring exactly once in each row and exactly once in each column. *(−5 pts)

Definition 3. *A group is a set
$\mathcal{G}$ with a binary operation
$\circ $ *(−2* pts each*),

$\circ \mathrm{:}\mathcal{G}\times \mathcal{G}\to \mathcal{G}\mathrm{.}$

for all $a\mathrm{,}b\mathrm{,}c\in \mathcal{G}$ satisfying:

1) Closure: $a\circ b\in \mathcal{G}$ .

2) Associativity: $a\circ \left(b\circ c\right)=\left(a\circ b\right)\circ c$ .

3) Identity element: There exists and element $e\in \mathcal{G}$ such that $a\circ e=a=e\circ a$ .

4) Inverse: For every $a\in \mathcal{G}$ there exists some $b\in \mathcal{G}$ such that $a\circ b=e$ and $b\circ a=e$ .

Definition 4. *A Cayley table describes the structure of a finite group by arranging all the possible products of all the group’s elements in a square table *(−5* pts*)*.*

Theorem 2.
${\varphi}_{2}$ :*
$\Delta \mathrm{,}\text{\hspace{0.17em}}\square \mathrm{,}\Theta $ *,* and
$\Psi $ *,* form a group
$\left(\mathcal{G}\mathrm{,}\circ \right)$ .*

1) Construct a table of symmetries (Cayley table) and show that it is a Latin square. [20 pts]

2) Using the definition of a group show the closure property (use the Cayley table). [15 pts]

3) Using the definition of a group demonstrate the associativity condition (use the Cayley table). [15 pts]

4) Using the definition of a group find the identity element (use the Cayley table). [15 pts]

5) Using the definition of a group show that each element has an identity (use the Caleyly table). [15 pts]

6) Write up a formal proof using the results of tasks 2 - 5. [20 pts]

*Proof.* To show the closure property we construct a Cayley table for
$\left(\mathcal{G}\mathrm{,}\circ \right)$ . From TableA3 one can also see that it defines a Latin square. The closure property follows from the Latin square property. Associativity follows from the fact that the composition of bijections from a set to itself is associative. By inspection of the Cayley table we have:

${n}_{0}\circ {n}_{0}={n}_{0}={n}_{0}\circ {n}_{0}$

${n}_{0}\circ \Theta =\Theta =\Theta \circ {n}_{0}$

${n}_{0}\circ {\Theta}^{2}={\Theta}^{2}={\Theta}^{2}\circ {n}_{0}$

Table A1. Task success theorem I.

Table A2. Task time evaluation theorem I.

Table A3. Cayley table for the six symmetries of $\Delta $ .

${n}_{0}\circ \Psi =\Psi =\Psi \circ {n}_{0}$

${n}_{0}\circ {\Theta}^{2}\Psi ={\Theta}^{2}\Psi ={\Theta}^{2}\Psi \circ {n}_{0}$

${n}_{0}\circ \Theta \Psi =\Theta \Psi =\Theta \Psi \circ {n}_{0}.$

It follows that ${n}_{0}\in \mathcal{G}$ is the identity element since for any $g\in \mathcal{G}$ ${n}_{0}\circ g=g\circ {n}_{0}=g$ . Each of our symmetries $g$ has an inverse ${g}^{-1}$ such that $g\circ {g}^{-1}={g}^{-1}\circ g={n}_{0}$ . We have

${n}_{0}$ , with inverse ${n}_{0}$ since ${n}_{0}\circ {n}_{0}={n}_{0}$

$\Theta $ , with inverse $\Theta $ since $\Theta \circ {\Theta}^{2}={\Theta}^{2}\circ \Theta ={n}_{0}$

${\Theta}^{2}$ , with inverse ${\Theta}^{2}$ since ${\Theta}^{2}\circ \Theta =\Theta \circ {\Theta}^{2}={n}_{0}$

$\Psi $ , with inverse $\Psi $ since $\Psi \circ \Psi ={n}_{0}$

${\Theta}^{2}\Psi $ , with inverse ${\Theta}^{2}\Psi $ since ${\Theta}^{2}\Psi \circ {\Theta}^{2}\Psi ={n}_{0}$

$\Theta \Psi $ , with inverse $\Theta \Psi $ since $\Theta \Psi \circ \Theta \Psi ={n}_{0}$ . ¨

NOTES

^{1}https://www.cne.psychol.cam.ac.uk/what-is-mathematics-anxiety.

^{2}https://www.ams.org/publications/journals/notices/201702/rnoti-p124.pdf.

^{3}The learning activities correspond to
${\varphi}_{1}$ and
${\varphi}_{2}$ of section three. The other statements can be used as further active learning activities within the PGMP model.

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

[1] |
Bandura, A. (1977) Self-Efficacy: Toward a Unifying Theory of Behavioral Change. Psychological Review, 84, 191-215. https://doi.org/10.1037/0033-295X.84.2.191 |

[2] | Bandura, A. (1986) Social Foundations of Thought and Action: A Social Cognitive Theory. Prentice-Hall, Englewood Cliffs. |

[3] | Bandura, A. (1997) Self-Efficacy: The Exercise of Control. W.H. Freeman, New York. |

[4] |
Pajares, F. (2021) Self-Efficacy Beliefs in Academic Contexts: An Outline. http://des.emory.edu/mfp/efftalk.html |

[5] |
Carey, M.P. and Forsyth, A.D. (2021) Teaching Tip Sheet: Self-Efficacy. https://www.apa.org/pi/aids/resources/education/self-efficacy |

[6] |
Ashcract, M.H. (2002) Math Anxiety: Personal, Educational, and Cognitive Consequences. Current Directions in Psychological Science, 11, 181-185. https://doi.org/10.1111/1467-8721.00196 |

[7] |
Richardson, F.C. and Suinn, R.M. (1972) The Mathematics Anxiety Rating Scale: Psychometric Data. Journal of Counseling Psychology, 19, 551-554. https://doi.org/10.1037/h0033456 |

[8] | Tahamassian, K. and Moghadam, N.J. (2011) Relationship between Self-Efficacy and Symptoms of Anxiety, Depression, Worry and Social Avoidance in a Normal Sample of Students. Iranian Journal of Psychiatry and Behavioral Sciences, 5, 91-98. |

[9] | Unlu, M., Ertekin, E. and Dilmae, B. (2017) Predicting Relationships between Mathematics Anxiety, Mathematics Teaching Anxiety, Self-Efficacy Beliefs towards Mathematics and Mathematics Teaching. International Journal of Research in Education and Science, 3, 636-645. |

[10] |
Hackett, G. and Betz, N.E. (1989) An Exploration of the Mathematics Self-Efficacy/Mathematics Performance Correspondence. Journal for Research in Mathematics Education, 20, 261-273. https://doi.org/10.2307/749515 |

[11] |
Effandi, Z. and Norazahn, M.N. (2008) The Effects of Mathematics Anxiety on Matriculation Students as Related to Motivation and Achievement. Eurasia Journal of Mathematics, Science & Technology Education, 4, 27-30. https://doi.org/10.12973/ejmste/75303 |

[12] | Luo, X., Wang, F. and Luo, Z. (2009) Investigation and Analysis of Mathematics Anxiety in Middle School Students. Journal of Mathematics Education, 2, 12-19. |

[13] |
Lim, E. and Seng, K. (2015) The Influence of Pre-University Student’s Mathematics Test Anxiety and Numerical Anxiety on Mathematics Achievement. International Education Studies, 8, 162-168. https://doi.org/10.5539/ies.v8n11p162 |

[14] |
Alves, M., Rodrigues, C.S., Rocha, A.M. and Coutinho, C. (2016) Self-Efficacy, Mathematics—Anxiety and Perceived Importance: An Empirical Study with Portuguese Engineering Students. European Journal of Engineering Education, 41, 105-121. https://doi.org/10.1080/03043797.2015.1095159 |

[15] |
Ma, X. (1999) A Meta-Analysis of the Relationship between Anxiety toward Mathematics and Achievement in Mathematics. Journal for Research in Mathematics Education, 30, 520-540. https://doi.org/10.2307/749772 |

[16] |
Mavridis, A., Katmada, A. and Tsiatsos, T. (2017) Impact of Online Flexible Games on Students’ Attitude towards Mathematics. Educational Technology Research and Development, 65, 1451-1470. https://doi.org/10.1007/s11423-017-9522-5 |

[17] |
Rounds Jr., J.B. and Hendel, D.D. (2018) Mathematics Anxiety and Attitudes toward Mathematics. Measurement and Evaluation in Guidance, 13, 83-89. https://doi.org/10.1080/00256307.1980.12022221 |

[18] | Ku, O., Chen, S.Y., Wu, D.H., Lao, A.C. and Chan, T.W. (2018) The Effects of Game-Based Learning on Mathematical Confidence and Performance: High Ability vs. Low Ability. Journal of Educational Technology & Society, 17, 65-78. |

[19] |
Shroff, R., Ting, F. and Lam, W. (2019) Development and Validation of an Instrument to Measure Student’s Perceptions of Technology-Enabled Active Learning. Australian Journal of Educational Technology, 35, 109-127. https://doi.org/10.14742/ajet.4472 |

[20] |
Freeman, S., Eddy, S., McDonough, M., Smith, M., Okoroafor, N., Jordt, H. and Wenderoth, M. (2014) Active Learning Increases Student Performance in Science, Engineering, and Mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111, 8410-8415. https://doi.org/10.1073/pnas.1319030111 |

[21] |
Laursen, S.L., Hassi, M.L., Kogan, M.M., and Weston, T.J. (2014) Benefits for Women and Men of Inquiry-Based Learning in College Mathematics: A Multi-Institutional Study. Journal for Research in Mathematics Education, 45, 406-418. https://doi.org/10.5951/jresematheduc.45.4.0406 |

[22] |
Kim, K., Sharma, P., Land, S.M. and Furlong, K.P. (2013) Effects of Active Learning on Enhancing Student Critical Thinking in an Undergraduate General Science Course. Innovative Higher Education, 38, 223-235. https://doi.org/10.1007/s10755-012-9236-x |

[23] |
Marton, F. (2018) Towards a Pedagogical Theory of Learning. In: Matsushita, K., Ed., Deep Active Learning, Springer, Singapore, 59-77. https://doi.org/10.1007/978-981-10-5660-4_4 |

[24] |
Kerrigan, J. (2018) Active Learning Strategies for the Mathematics Classroom. College Teaching, 66, 35-36. https://doi.org/10.1080/87567555.2017.1399335 |

[25] |
Sung, H.Y. and Hwang, G.J. (2018) Facilitating Effective Digital Game-Based Learning Behaviors and Learning Performances of Students Based on a Collaborative Knowledge Construction Strategy. Interactive Learning Environments, 26, 118-134. https://doi.org/10.1080/10494820.2017.1283334 |

[26] |
Hwang, G.J., Wu, P.-H. and Chen, C.C. (2012) An Online Game Approach for Improving Students—Learning Performance in Web-Based Problem-Solving Activities. Computers & Education, 59, 1246-1256. https://doi.org/10.1016/j.compedu.2012.05.009 |

[27] |
Thou Ting, F.S., Lam, W.H. and Shroff, R.H. (2019) Active Learning via Problem-Based Collaborative Games in a Large Mathematics University Course in Hong Kong. Education Sciences, 9, Article No. 172. https://doi.org/10.3390/educsci9030172 |

[28] | Yang, J.C., Chien, K.H. and Liu, T.C. (2012) A Digital Game-Based Learning System for Energy Education: An Energy Conservation Pet. Turkish Online Journal of Educational Technology, 11, 27-37. |

[29] |
Naik, N. (2017) The Use of GBL to Teach Mathematics in Higher Education. Innovations in Education and Teaching International, 54, 238-246. https://doi.org/10.1080/14703297.2015.1108857 |

[30] |
Gil-Doménech, D. and Berbegal-Mirabent, J. (2019) Stimulating Students-Engagement in Mathematics Courses in non-STEM Academic Programmes: A Game-Based Learning. Innovations in Education and Teaching International, 56, 57-65. https://doi.org/10.1080/14703297.2017.1330159 |

[31] |
O-Neil, H.F., Wainess, R. and Baker, E.L. (2005) Classification of Learning Outcomes: Evidence from the Computer Games Literature. Curriculum Journal, 16, 455-474. https://doi.org/10.1080/09585170500384529 |

[32] | Perrotta, C., Featherstone, G., Aston, H. and Houghton, E. (2013) Game-Based Learning: Latest Evidence and Future Directions. NFER Research Programme: Innovation in Education, National Foundation for Educational Research, Slough. |

[33] |
Skilling, K., Bobis, J. and Martin, A.J. (2021) The “Ins and Outs” of Student Engagement in Mathematics: Shifts in Engagement Factors among High and Low Achievers. Mathematics Education Research Journal, 33, 469-493. https://doi.org/10.1007/s13394-020-00313-2 |

[34] |
Mac, C., Fitzmaurice, O., Fhloinn, E.N. and O’Sullivan, C. (2013) Student Non-Engagement with Mathematics Learning Supports. Teaching Mathematics and Its Applications, 32, 191-205. https://doi.org/10.1093/teamat/hrt018 |

[35] | Zimmerman, H.J. (2011) Fuzzy Set Theory and Its Applications. Springer Science & Business Media, Berlin, Heidelberg. |

[36] |
Saaty, T.L. (1977) A Scaling Method for Priorities in Hierarchical Structures. Journal of Mathematical Psychology, 15, 234-281. https://doi.org/10.1016/0022-2496(77)90033-5 |

[37] |
Liu, S., Triantis, K.P. and Sarangi, S. (2010) Representing Qualitative Variables and Their Interactions with Fuzzy Logic in System Dynamics Modeling. Systems Research and Behavioral Science, 28, 245-263. https://doi.org/10.1002/sres.1064 |

[38] | Zhou, J.H., Wang, Z., Wang, Z.K. and Liu, X. (2004) Research on Software Quality Evaluation Based on Fuzzy Method. Systems Engineering and Electronics, 26, 988-991. |

[39] | Ying-Feng, K. and Ling-Show, C. (2002) Using the Fuzzy Synthetic Decision Approach to Assess the Performance of University Teachers in Taiwan. International Journal of Management, 19, 593. |

[40] |
Lewis, J.R. (1995) IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. International Journal of Human-Computer Interaction, 7, 57-58. https://doi.org/10.1080/10447319509526110 |

[41] |
Saaty, T.L. (1990) How to Make a Decision: The Analytic Hierarchy Process. European Journal of Operations Research, 48, 9-26. https://doi.org/10.1016/0377-2217(90)90057-I |

Journals Menu

Contact us

customer@scirp.org | |

+86 18163351462(WhatsApp) | |

1655362766 | |

Paper Publishing WeChat |

Copyright © 2022 by authors and Scientific Research Publishing Inc.

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.