Analyzing the Impact of Age and Gender on COVID-19 Deaths Using Two-Way ANOVA

Azad Abdulhafedh

University of Missouri, Columbia, USA.

DOI: 10.4236/oalib.1109658                

Abstract

Since the start of COVID-19 there have been widely spread notions that men can contract the disease at a higher rate than women and are more likely to die from it. Furthermore, age has been reported as a risk factor in COVID-19 mortality, and a vast majority of COVID-19 deaths have been among elderly people. Therefore, it is interesting to conduct a statistical analysis on COVID-19 deaths to draw conclusions on whether men are more likely to die of COVID-19 than women, and the influence of age groups on the number of reported deaths of COVID-19 and the interaction between these two factors. This paper uses a two-way analysis of variance (two-way ANOVA) as a statistical tool to analyze COVID-19 deaths in the US. Two-way ANOVA can effectively determine whether the age and gender are significant factors in COVID-19 death cases in the US. The dependent variable in the analysis is the number of COVID deaths in the entire US, and the two independent variables are the age groups and gender (sex). The age groups consist of 11 subgroups or levels ranging from babies to elderly people. The sex is either male or female. Results showed that age group is a significant factor in COVID deaths, while gender was found to be insignificant factor in the mortality of COVID.

Keywords

COVID Deaths, Two-Way ANOVA, Post-Hoc ANOVA Test, Tukey HSD Test

1. Introduction

COVID-19 pandemic has impacted all aspects of our daily life including education, culture, employment, communication, sports, shopping, and other areas. The pandemic has changed how we work, learn, and interact as social distancing guidelines have led to a more virtual existence, both personally and professionally. There is a widely spread notion that men are more vulnerable to COVID-19 than women, and more men than women have died of COVID-19. Furthermore, the age is reported as a significant risk factor in COVID-19 mortality, and a vast majority of COVID-19 deaths have been among people older than 70. Therefore, the role of gender in COVID-19 mortality has to be investigated along with the age groups. It is vital to consider gender as a biological variable in the prevention, and care of the virus. Additionally, understanding the effects of the age groups associated with gender and the interaction between them is just as important. Differences in sex are biological. These include differences in reproductive organs and their functions, sexual hormones, and the gene expression of chromosomes [1] [2] [3] [4]. Gender is the performance of socially constructed roles, behaviors, and attributes considered socially acceptable for men and women. In addition to sex differences in immune responses, hormones, and genes, there are also psychological, social, and behavioral components that influence COVID-19 progression. Compared with women, men tend to engage in more high-risk behaviors that generate the potential for contracting COVID-19. There might be a range of biological, psychological, and behavioral factors that can explain why men have higher rates of COVID-19-associated morbidity and mortality than women [5] [6]. Therefore, we recognize that gender and age groups are important and should call attention to the COVID-19 reported deaths. Knowing the impact of age groups and gender on coronavirus deaths would help in making precautions in advance for those people who are more vulnerable to the disease, like prioritizing vaccinations and health care services for those groups.

2. Data

Data are obtained from the “Provisional COVID-19 Deaths by Sex and Age” datasets available for the public at data.gov and cdc.gov websites. The data contains the number of COVID-19 deaths, and other diseases, such as pneumonia, and influenza reported to the National Center for Health Statistics (NCHS) by sex, age group, and jurisdiction of occurrence. Data we used contained COVID-19 deaths recorded from 1/1/2020 to 11/09/2022. COVID death records are available for the entire US, and for each individual state as well. This paper uses the recorded COVID deaths for the entire USA in the analysis. Data was carefully inspected and cleaned. Missing values and outliers were removed. Only COVID-19 deaths were used in the analysis including both independent variables; age groups and sex. The age groups have 11 subgroups (levels) as shown in Figure 1. These subgroups are; (0 - 1), (1 - 4), (5 - 14), (15 - 24), (25 - 34), (35 - 44), (45 - 54), (55 - 64), (65 - 74), (75 - 84), and (85 years and above). The sex variable includes males and females. Data used in the analysis is presented in Table 1.

3. Exploratory Data Analysis

First, a summary statistics for the variables in the analysis is conducted as shown in Table 2.

We can see from Table 2 that the mean of “COVID deaths” in the dataset is 48,524.5, the median is 14,490, and the standard deviation is 58,978.6. The mean of the “age group” is 40.86, the median is 40.0, and the standard deviation is 30.28. The mean of the variable “sex” is 1.5, the median is 1.5, and the standard deviation is 0.52.

Exploratory data analysis is conducted to discover the hidden patterns in the data and the relationships between the variables. The normal distribution curve of the response variable (number of COVID deaths in the US) is generated

Figure 1. Levels of age groups in COVID-19 data.

Table 1. Data used in the analysis.

Table 2. Summary statistics of the variables included in the analysis.

using the R software as shown in Figure 2. We can see from Figure 2 that the top of the curve is centered at approximately (50,000 COVID deaths), and it is skewed to the right.

Figure 3 shows a bar chart presenting the number of COVID deaths by age

Figure 2. Normal distribution curve of COVID-19 deaths.

Figure 3. Number of COVID deaths by age groups for males.

groups for males, and Figure 4 shows a bar chart presenting the number of COVID deaths by age groups for females. Figure 5 and Figure 6 show pie charts showing the percentages of COVID deaths by age groups for males and females, respectively. We can see from both pie charts that babies and children suffer very small percentages of COVID deaths compared to elderly people. For example, the percentages of COVID deaths of age groups (0 - 1, 1 - 4, 5 - 14) for both males and females are (0.04%, 0.02%, 0.04%) respectively, while the percentages of COVID deaths for age groups (65 - 74, 75 - 84) are (24.4%, 26.3%) for males and (20.6%, 25.4%) for females.

Figure 7 shows a line plot of the COVID-19 deaths by the age groups for both males and females together in the US. We can see from Figure 7 that the cases of COVID deaths are relatively low for groups (0 - 1, 1 - 4, 5 - 14, 15 - 24). However, they start to increase from group (25 - 34) until the last age group (85 yr. and

Figure 4. Number of COVID deaths by age groups for females.

Figure 5. Percentages of COVID deaths by age groups for males.

Figure 6. Percentages of COVID deaths by age groups for females.

Figure 7. Line plot of COVID-19 deaths by age groups for both males and females together.

above). The increase in COVID deaths is apparent for groups of elderly (55 - 64, 65 - 74, 75 - 84, 85 and above). This indicates that the age group has impact on the number of deaths. As the age group increases the number of COVID deaths increases as well.

Figure 8 shows the COVID deaths for males and females by age groups separately. We can see from Figure 8 that the total number of deaths is almost identical for both males and females among the first three age groups (0 - 1, 1 - 4, 4 - 15). Then, the number of COVID deaths increases for both males and females until

Figure 8. COVID-19 deaths for males and females by age groups separately.

the last group. However, the number of COVID deaths for males is slightly more than females for all age groups, except for the last group (85 and above), where the number of deaths for females is larger than males as can be clearly seen. This indicates that males had experienced more fatality from COVID than females, however, we have to figure out if this is statistically significant by employing the Two-way ANOVA procedure in the analysis.

Boxplots of COVID-19 deaths by age groups for both males and females together are shown in Figure 9. A boxplot is a useful visualization tool that can show the minimum, 1^st quartile, median, 3^rd quartile, and maximum values. We can notice from Figure 9 that starting from group 5 (25 - 34) the number of COVID deaths increases with a median value of (5909.5). The median for group 6 (35 - 44) is (14,490), the median for group 7 (45 - 54) is (34,571). The median for group 8 (55 - 64) is (76,700). The median for group 9 (65 - 74) is (121,238.5). The median for group 10 (75 - 84) is (138,326.5). The median for group 11 (85 yrs. and above) is (140,610.5). As the age group increases, the number of COVID deaths increases with their median values as well.

Figure 10 shows the COVID-19 deaths by gender (sex) types in the US. We can see from Figure 10 that the median number of COVID deaths for males is (17,966), whereas for females, the median is smaller than males (11,014). The mean (average) of COVID deaths for males is (53,527.18), while the mean for females is only (43,521.82). The third quartile (Q3) for males is (123,830), while for females is only (98,568). The interquartile range (IQR = Q3 − Q1) for males is (123,617), while for females, the IQR is only (98,358). Again, this indicates that males had experienced more fatality from COVID-19 than females, however, we have to determine if this is statistically significant or not through the ANOVA method.

Figure 9. Boxplots of COVID-19 deaths by age groups in the US.

Figure 10. Boxplots of COVID-19 deaths by gender types in the US.

4. Overview on the Analysis of Variance (ANOVA)

ANOVA, which stands for “Analysis of Variance”, is a statistical test used to analyze the differences between the means of more than two groups. A one-way ANOVA uses one independent variable, while a two-way ANOVA uses two independent variables. MANOVA uses multiple independent variables. ANOVA can determine if the dependent variable changes according to the level of the independent variable. The statistic generated by ANOVA is called the F-statistic, which is, the ratio of between-treatments variance to within-treatment variance [7] - [12]. Within-treatment variance refers to the variability within a particular sample. There are two sources that contribute to the variability within a sample: individual differences and experimental error. Both of these sources of variance are considered to be random error variance because they are unintentional and not the result of planning or design. Between-treatments variance refers to the variability between the treatment groups. The same two factors that contributed to within-treatment variance (individual differences and experimental error) also contribute to between-treatments variance. However, there is an additional source that contributes to between-treatments variance: treatment effects. If the independent variable had no influence on the dependent variable, the value of the F-statistic would be approximately 1. Conversely, if there were treatment effects that created large differences between group means, then the treatment variance would be bigger than the within-treatment variance and the value of F would be larger than 1. The within-treatment variance is also referred to as error variance and between-treatments variance is referred to as treatment variance [13] [14] [15] [16]. Thus, another way of stating the F-statistic is:

F = Treatment variance/Error variance.

A typical ANOVA table is usually presented with the terms shown in Table 3:

where:

Between Groups Degrees of Freedom: DF = k − 1, where k is the number of groups;

Within Groups Degrees of Freedom: DF = N − k, where N is the total number of subjects;

Total Degrees of Freedom: DF = N ? 1;

Sum of Squares Between Groups: ${\text{SS}}_{\text{B}}={\displaystyle {\sum }_{i=1}^k{n}_i{\left(x_i-x\right)}^2}$, where n_i is the number of subjects in the i-th group;

Sum of Squares Within Groups: ${\text{SS}}_{\text{W}}={\displaystyle {\sum }_{i=1}^k\left(n_i-1\right)S_i^2}$, where S_i is the standard deviation of the i-th group;

Total Sum of Squares: SS_T = SS_B + SS_W;

Mean Square Between Groups: MS_B = SS_B/(k − 1);

Mean Square Within Groups: MS_W = SS_W/(N − k);

F-Statistic (or F-ratio): F = MS_B/MS_W.

Additionally, in conducting an ANOVA, there should be a hypothesis testing, the null hypothesis predicts that the value of F will be 1.00 because, it assumes

Table 3. Typical ANOVA table.

no difference between means; and that any differences found between means were simply due to chance, or random error. The alternative hypothesis assumes that there will be significant differences between some of the means. F-statistics will not have negative values because they are calculated from ratios of variances, which are squared scores. Thus, F-values will only be positive. The ANOVA alone does not tell us specifically which means are different from one another. To determine that, we would need to follow up with multiple comparisons (or post-hoc) test. ANOVA makes the following assumptions [7] [8] [11] [14] [15]:

1) Normality: Each sample is drawn from a normally distributed population.

2) Equal Variances: The variances of the populations that the samples come from are equal.

3) Independence: The observations in each group are independent of each other and the observations within groups are obtained by a random sample.

5. Methodology Steps and Discussion

We have conducted the following steps in the analysis:

Step 1: Hypothesis testing

We will test three hypothesis in this research as follows:

1^st null hypothesis: H₀₁: There is no difference in COVID deaths for any age group.

1^st alternative hypothesis: There is a difference in COVID deaths by age group.

2^nd null hypothesis: H₀₂: There is no difference in COVID deaths at either sex level (males or females).

2^nd alternative hypothesis: There is a difference in COVID deaths by sex level.

3^rd null hypothesis: H₀₃: The effect of one independent variable on COVID deaths does not depend on the effect of the other independent variable (i.e., no interaction effect).

3^rd alternative hypothesis: There is an interaction effect between age group and sex on COVID deaths.

Step 2: Fitting a two-way ANOVA with interaction term

We run a two-way ANOVA using the R software with interaction term between the age groups and sex. The outcome is shown in Table 4.

・ We can see from Table 4 that the p-value for the independent variable “age

Table 4. Outcome of two-way ANOVA with interaction term.

***significant at 0.001.

group” is very small and significant at 0.001 level. So, the age group is statistically significant.

・ The p-value for the independent variable “sex” is 0.689 > 0.05. So, the sex variable is statistically insignificant at the significance level of 0.05.

・ The p-value for the interaction term between age group and sex is 0.748 > 0.05. So, the interaction term is statistically insignificant at the alpha level of 0.05.

・ These results indicate that the “age group” is the only risk factor that has a statistically significant effect on COVID deaths. The sex and interaction term have no significant effects on COVID deaths.

・ Based on these results, we reject the 1^st null hypothesis and accept the 1^st alternative hypothesis and conclude that there is a difference in COVID deaths by age groups.

・ Also, we cannot reject the 2^nd and 3^rd null hypotheses, and conclude that there is no difference in COVID deaths at either sex level (males or females), and the interaction between age group and sex has no significant effect on COVID deaths as well.

Step 3: Checking the assumptions of the two-way ANOVA

Two-way ANOVA makes some assumptions that should be checked in order to be sure that the procedure was well fitted to our data. We can check these assumptions as follows:

1) Homogeneity of variance (or homoscedasticity): We can use the residuals versus fitted plot to check the homogeneity of variances. The plot is generated in R as shown in Figure 11. We can see from Figure 11 that the red line of the

Figure 11. Plot of residuals vs fitted, normal Q-Q plot, and residuals vs leverages.

residuals vs fitted values is almost straight, so, there are no evident relationships between residuals and fitted values. This indicates that the variation around the mean for each group of residuals being compared is almost similar among all groups. So, we can assume that the homogeneity of variances has been met. To further support our assumption, we conducted Levene’s test to check the homogeneity of variances in R, and the results are shown in Table 5.

We can notice from Table 5 that the p-value (0.148 > 0.05), which means that there is no evidence to suggest that the variance across groups is significantly different. Therefore, we can assume that the homogeneity of variances in the different age groups has been met.

2) In order to check the normality assumption, we can see from Figure 7 that the normal quartile-quartile (Q-Q) plot approximately follows an inclined straight line, and all the points fall approximately along this reference line, so, we can assume normality distribution. To support our assumption, we conducted the Shapiro-Wilk normality test on the ANOVA residuals in R, and the test results are shown in Table 6.

We can see from Table 6 that the p-value of the test is (0.6927 > 0.05), which indicates that the normality of the residuals has been met.

3. We also assume that the observations in each group are independent of each other when data were collected and the observations within groups were obtained by a random sample, so that the ANOVA assumption of independence is met as well.

Step 4: Effect Size

We can use the “Partial eta squared” to measure the effect size of different variables in two-way ANOVA models. It measures the proportion of variance explained by a given variable of the total variance remaining after accounting for variance explained by other variables in the model. When there is only one predictor variable in the model (i.e., a one-way ANOVA), then the value for eta squared and partial eta squared will be equal [17] - [25]. Partial eta squared is calculated as follows:

Partial eta squared = SS_effect/(SS_effect + SS_error)

where:

Table 5. The outcome of Levene’s test for homogeneity of variances.

Table 6. The outcome of Shapiro-Wilk normality test.

・ SS_effect: The sum of squares of an effect for one variable.

・ SS_error: The sum of squares error in the ANOVA model.

The value for Partial eta squared ranges from 0 to 1, where values closer to 1 indicate a higher proportion of variance that can be explained by a given variable in the model after accounting for variance explained by other variables in the model.

The following rules of thumb are used to interpret values for Partial eta squared [18]:

0.01 or smaller: small effect size.

0.06: medium effect size.

0.14 or higher: large effect size.

Table 7 shows the values of the partial eta squared for variables, age groups and sex.

We can see from Table 7 that the age group variable has a very large effect size, but the sex variable has a very small effect size.

Step 5: Post-hoc Test

ANOVA outcome can tell which variable is significant, but not which levels of the variable are different from one another. To determine this, we can use a post-hoc test. The Tukey’s Honestly-Significant-Difference (Tukey HSD) test can effectively identify which groups are significantly different from one another. Tukey’s HSD focuses on the difference between the groups with the largest and smallest means. If the difference is less than or equals a margin of error for the difference in the means, then the confidence intervals for that difference in the means will contain zero value. If the confidence interval contains zero, then the group pairs do not differ significantly. If the confidence interval does not cover zero, then the group pairs significantly differ. A Tukey HSD test is conducted in R. The outcome shows the pairwise differences between the 11 levels of age groups with the average difference, the lower and upper bounds of the 95% confidence interval and the p-value of the difference. There were 55 pairwise groups. In order to graphically illustrate the Tukey HSD outcome, we generated the Tukey HSD plot of 95% family-wise confidence level for the variable age groups in R as shown in Figure 12. The significant groupwise differences are anywhere the 95% confidence interval does not include zero. This is another way of saying that the p-value for these pairwise differences is <0.05.

From the outcome of the Tukey HSD test and the 95% family-wise plot, we can determine the following significant levels of the variable “age groups” as shown in Table 8.

We can realize from Table 8 that only 26 pairwise levels of the “age group”

Table 7. Effect size of variables in two-way ANOVA.

Figure 12. The 95% family-wise confidence level of pairwise differences for age groups.

Table 8. The significant pairwise levels of the age groups.

variable is statistically significant out of a total of 55 pairwise levels. The significant pairwise age groups mainly included old people, babies, children, and young adults, such as 8-1, 8-2, 8-3, 9-1, 9-2, 9-3, 10-1, 10-2, 10-3, 11-1, 11-2, and 11-3. Possible reasons behind these significant pairwise groups are that older people (i.e., groups 8, 9, 10, 11) are more likely than younger people (i.e., groups 1, 2, 3, 4) to have underlying health problems, such as dementia, cardiovascular diseases, and diabetes. Children and young adults are not as prone to severe forms of COVID-19 compared to old people. So, COVID deaths are more likely to increase among older people, hence making the mean differences in these groups statistically significant as was shown by Tukey test and the graph of 95% family-wise confidence level. Compared to people under 45 years old, the chances of dying from COVID-19 are much higher among those aged 75 and beyond, therefore the obtained results were evident in our analysis.

6. Conclusion

COVID-19 pandemic has affected all aspects of our daily life. It has changed how we work, learn, and interact. Since the start of the pandemic, there have been many guesses that men are contracting COVID-19 at a higher rate than women and are more likely to die from the disease. In order to investigate the role of gender and age groups in COVID mortality in the US, a two-way ANOVA was used in this paper to test whether men are more likely to die of COVID-19 than women, and whether the age matters for COVID-19 deaths in the US. The response variable was number of COVID deaths in the US, and the independent variables were, age groups and sex. The two-way ANOVA was run with the interaction between age groups and sex. The results revealed that the variable “age groups” was statistically significant, as its p-value was very small and significant at 0.001 level. However, the variable “sex” and the interaction term were found to be insignificant, as their p-values were greater than alpha (0.05). These results indicate that the age group is the only factor that has a statistically significant effect on COVID deaths. The sex and interaction term has no significant effects on COVID deaths. The ANOVA assumptions of homogeneity of variance, normality, and independence of observations were tested and found to be met. A post-hoc Tukey HSD test was conducted on levels of the significant variable “age groups” to identify the significant pairwise groups that can affect COVID deaths. The outcome from Tukey test showed that there were 26 significant pairwise age groups in the data. Also, we have shown by conducting the ANOVA test that the gender was insignificant factor on COVID deaths, which contradicts the widely spread notion that men are more susceptible to COVID deaths than women. Although men and women differ in their genetic makeup, immune responses, and hormones, however, they both can suffer from COVID-19 consequences. There might be other social factors contributing to the sex differences in COVID deaths, such as job types, behavioral patterns, and underlying health issues. Another reason might be that women tend to have stronger immune systems than men. Men also tend to engage in more risky behaviors and can ignore physical distancing, and they might not take COVID symptoms as seriously. Behaviors that impact lung health, such as smoking, also may play a role in the disease’s deadly impact on men.

Conflicts of Interest

The author declares no conflicts of interest.

References

[1]	Griffith, D.M., Sharma, G., Holliday, C.S., Enyia, O.K., Valliere, M., Semlow, A.R., et al. (2020) Men and COVID-19: A Biopsychosocial Approach to Understanding Sex Differences in Mortality and Recommendations for Practice and Policy Interventions. Preventing Chronic Disease, 17, Article ID: 200247. https://doi.org/10.5888/pcd17.200247
[2]	Danielsen, A.C., Boulicault, M., Gompers, A., Rushovich, T., Lee, K.M.N. and Richardson, S.S. (2022) How Cumulative Statistics Can Mislead: The Temporal Dynamism of Sex Disparities in COVID-19 Mortality in New York State. International Journal of Environmental Research and Public Health, 19, Article No. 14066. https://doi.org/10.3390/ijerph192114066
[3]	Wenham, C., Smith, J. and Morgan, R. (2020) COVID-19: The Gendered Impacts of the Outbreak. Lancet, 395, 846-848. https://doi.org/10.1016/S0140-6736(20)30526-2
[4]	Barber, S.J. and Kim, H. (2020) COVID-19 Worries and Behavior Changes in Older and Younger Men and Women. Innovation in Aging, 4, 939-940. https://doi.org/10.1093/geroni/igaa057.3441
[5]	Sharma, G., Volgman, A.S. and Michos, E.D. (2020) Sex Differences in Mortality from COVID-19 Pandemic: Are Men Vulnerable and Women Protected? JACC: Case Reports, 2, 1407-1410. https://doi.org/10.1016/j.jaccas.2020.04.027
[6]	Alkhouli, M., Nanjundappa, A., Annie, F., Bates, M.C. and Bhatt, D.L. (2020) Sex Differences in COVID-19 Case Fatality Rate: Insights from a Multinational Registry. Mayo Clinic Proceedings, 95, 1613-1620. https://doi.org/10.1016/j.mayocp.2020.05.014
[7]	Gelman, A. (2005) Analysis of Variance—Why It Is More Important than Ever. Annals of Statistics, 33, 1-53. https://doi.org/10.1214/009053604000001048
[8]	Hinkelmann, K. and Kempthorne, O. (2008) Design and Analysis of Experiments. Vol. I and II. Wiley, Hoboken.
[9]	Howell, D.C. (2002) Statistical Methods for Psychology. 5th Edition, Wadsworth, Belmont, CA.
[10]	Hollander, M., Wolfe, D. and Chicken, E. (2013) Nonparametric Statistical Methods. Wiley Series in Probability and Statistics. Wiley, Hoboken.
[11]	Willard, C.A. (2020) Statistical Methods: An Introduction to Basic Statistical Concepts and Analysis. 2nd Edition, Routledge, New York. https://doi.org/10.4324/9780429261039
[12]	Moore, D.S. and McCabe, G.P. (2003) Introduction to the Practice of Statistics. 4th Edition, W. H. Freeman and Company, New York.
[13]	Rosenbaum, P.R. (2002) Observational Studies. 2nd Edition, Springer-Verlag, New York.
[14]	Cox, D.R. (2006) Principles of Statistical Inference. Cambridge University Press Cambridge.
[15]	Chambers, J. (2008) Software for Data Analysis: Programming with R. Statistics and Computing, Springer, NY. https://doi.org/10.1007/978-0-387-75936-4
[16]	Abdulhafedh, A. (2022) Comparison between Common Statistical Modeling Techniques Used in Research, Including: Discriminant Analysis vs Logistic Regression, Ridge Regression vs LASSO, and Decision Tree vs Random Forest. Open Access Library Journal, 9, e8414. https://doi.org/10.4236/oalib.1108414
[17]	Chen, W.W.S. (2004) Statistical Methods in Computer Security. In: Statistics: A Series of Textbooks and Monographs, CRC Press, Boca Raton.
[18]	Freedman, D.A. (2005) Statistical Models: Theory and Practice. Cambridge University Press, Cambridge.
[19]	Armstrong, R.A., Eperjesi, F. and Gilmartin, B. (2002) The Application of Analysis of Variance (ANOVA) to Different Experimental Designs in Optometry. Ophthalmic and Physiological Optics, 22, 248-256. https://doi.org/10.1046/j.1475-1313.2002.00020.x
[20]	Shin, J.H. (2009) Application of Repeated-Measures Analysis of Variance and Hierarchical Linear Model in Nursing Research. Nursing Research, 58, 211-217. https://doi.org/10.1097/NNR.0b013e318199b5ae
[21]	Mendes, M. and Yigit, S. (2013) Comparison of ANOVA-F and ANOM Tests with Regard to Type I Error Rate and Test Power. Journal of Statistical Computation and Simulation, 83, 2093-2104. https://doi.org/10.1080/00949655.2012.679942
[22]	Cohen, J. (1973) Eta-Squared and Partial Eta-Squared in Fixed Factor Anova Designs. Educational and Psychological Measurement, 33, 107-112. https://doi.org/10.1177/001316447303300111
[23]	Cohen, J. (1988) Statistical Power Analysis for the Behavioral Sciences. Routledge, New York.
[24]	Olejnik, S. and Algina, J. (2000) Measures of Effect Size for Comparative Studies: Applications, Interpretations, and Limitations. Contemporary Educational Psychology, 25, 241-286. https://doi.org/10.1006/ceps.2000.1040
[25]	Okada, K. (2013) Is Omega Squared Less Biased? A Comparison of Three Major Effect Size Indices in One-Way Anova. Behaviormetrika, 40, 129-147. https://doi.org/10.2333/bhmk.40.129