An Exploratory Study of Elementary School Students’ Reading Performance Scores before and after COVID-19 ()
1. Introduction
The onset of the COVID-19 pandemic in the spring of 2020 disrupted educational institutions as they abruptly transitioned from traditional face-to-face (FTF) instruction to online delivery of instruction. Although online instruction existed as an option for students at all levels of education before the global emergency, virtual learning became mandatory in many school districts across the United States during school closures in March 2020 (Sato, 2020) . According to research from The Department of Education’s Office of Civil Rights (2021) , most educators felt unprepared to provide high-quality instruction in online formats, especially when the transition demanded rapid deployment. Additionally, many families lacked the technical resources to support their children in online environments.
The research design of this study was a nonexperimental, exploratory study of archival data to examine one research question: What were the reading performance scores of third-grade students before and after the COVID-19 pandemic? The research sample included a single cohort of grade 3 students as they progressed through the fourth grade and the beginning of fifth grade in one school district in central Florida. Reading performance was measured by the students’ scale scores on i-Ready diagnostic tests (Curriculum Associates, 2019) , which were used by the school district to track reading development among all elementary students in grades 3 - 5 in the district.
2. Brief Overview of Literature
Researchers have conducted previous studies on the relationships between unscheduled and prolonged interruptions in educational programming and student achievement. For example, Cooper et al. (1996) conducted a meta-analytic review of 39 studies on the relationships between summer vacation and math and reading achievement scores from 1906 to 1995. The researchers reported that students from low-income homes demonstrated lower reading skill scores after the summer break; in contrast, students from middle-income families experienced little or no loss in reading skills over the summer. The researchers suggested that this disparity might be attributed to the availability of opportunities and resources for students to practice reading skills during the summer.
In a more recent study, Marcotte and Hemelt (2008) conducted a quantitative study of third-, fifth-, and eighth-grade students to determine whether unscheduled school closures influenced student performance on standardized reading assessments. These researchers found that unscheduled closures before February demonstrated a significant negative effect for third grade reading and math assessments.
Sandberg Patton and Reschly (2013) conducted a quantitative study of students’ oral reading fluency (ORF) in grades two through five to determine the influence of summer break on reading growth. The researchers suggested that second- and third-grade students required more help with reading practice when learning to read, while students in later stages of reading development could read independently.
Immediately after school closures due to COVID-19 in spring 2020, Dorn et al. (2020) developed predictive models to predict the differential influences of school closures on learning for minority students and those from low-income homes. These predictions were especially pertinent to the target school district in this study, which is a Title I district. The results of the Dorn et al.’s modeling suggested that high-school students who participated in remote instruction from March 2020 to January 2021 were at-risk for 6 to 7 months of learning loss in mathematics. Minority students and low-income students’ risks for learning loss in mathematics were even greater. The authors suggested that the level of learning loss during remote instruction in the past was related primarily to student disengagement, lack of technology, and lack of internet access.
Dorn et al. (2020) also predicted that high-school dropout rates would dramatically increase after school closures based on previous studies of interrupted educational programming. For example, after Hurricane Maria in 2017, 14 to 20% of high-school students never returned to high school (Declining Enrollment, Shuttered Schools: Puerto Rico’s Education System in Numbers, 2019) . If Dorn et al.’s (2020) predictive modeling is found to be accurate in future studies, the influence on the American economy and society could be enormous. The authors offered recommendations to jump-start educational responses to alleviate the deleterious influence of school closures: tutoring programs, summer school, parent education, improvements in online curriculum design and implementation, and professional development for teachers on best practices in remote teaching and learning.
Each year, Curriculum Associates publishes an annual report entitled “State of Student Learning”. Each annual report describes progress in math and reading of large numbers of students in grades 1 through 8 as measured by i-Ready diagnostic reading and math assessments. The 2022 annual report was of particular interest since it compared reading progress longitudinally and served to assist educators and researchers as they evaluated the influence of the pandemic, school lockdowns, and remote instruction. The results of the longitudinal comparisons in reading performance, typically conducted in May or June of each school year, are depicted in Figure 1 and Figure 2.
As indicated in Figure 1 and Figure 2, the percentages of students who were reading on grade level in Grades 1 and 2 appeared to drop dramatically after the introduction of online instruction in March 2020 and continued after students returned to classroom-based instruction through fall 2022. By third grade, the students’ scores approached historical performance levels by the end of 2021 and 2022, indicating that the students were closing the learning gaps. However, overall percentages of students who were on-level in reading remained relatively consistent to historical scores. These results point directly to reading development theory.
![]()
Figure 1. i-Ready diagnostic reading assessment results, Fall 2020 and Fall 2021. Adapted from student growth during COVID-19: grade-level readiness matters, by M. Dawson, 2022 , Curriculum Associates: research and efficacy, p. 9.
![]()
Figure 2. i-Ready diagnostic reading assessment results from Spring 2021 and Spring 2022. Note. Adapted from “The State of Student Learning in 2022,” by Curriculum Associates, 2022 , in Curriculum Associates State of Student Learning Annual Report, p. 10 (https://www.curriculumassociates.com/-/media/mainsite/files/corporate/state-of-student-learning-2022.pdf).
3. Reading Theory and Research
According to Chall’s (1995) and Chall & Jacobs (2003) seminal research on reading development, students develop critical reading concepts, skills, and dispositions in an orderly, hierarchical fashion. Skillsets developed in early childhood and the primary grades (kindergarten through second grade) provide the foundation for all future reading development. When the development of the basic skills of reading is interrupted or compromised, especially in early elementary grades, learning gaps can manifest in higher grade levels. Comprehensive research studies substantiate that students who do not learn to read on grade level by grade 3 are likely to continue to need reading support throughout their lives (e.g., Aber et al., 2013; Gersten et al., 2020) . According to Chall (1995) , critical periods exist during which reading skill development is optimal and maximized; these critical periods build the foundation for continued reading development into adulthood. Theoretically, the possibility of learning loss or reduction in reading performance would be greater during the pandemic for primary-grade students (kindergarten through grade two) who were still learning the basic concepts and skills of reading and who might have difficulty following instructions and responding appropriately in online formats due to undeveloped reading skills and unfamiliarity with keyboards. On the other hand, third-grade readers presumably have already covered and mastered the basics of phonics, decoding, and other elements of reading taught in the primary grades and could be considered independent readers, which would enable them to fully engage in remote instruction.
This theoretical assumption was confirmed in the 2022 annual report published by Curriculum Associates, which pointed to the challenges and difficulties of teaching the basic skills of reading to young children during remote instruction. The results of extensive analyses of i-Ready data in the 2022 annual publication revealed that the percentages of first-, second-, and third-grade students who were below grade level in phonics increased over time (Figure 3) during remote instruction due to the pandemic.
![]()
Figure 3. Percentage of students below grade level in phonics skills on i-Ready diagnostic assessments. Note. Adapted from “The State of Student Learning in 2022,” by Curriculum Associates, 2022 , in Curriculum Associates State of Student Learning Annual Report, p. 13 https://www.curriculumassociates.com/-/media/mainsite/files/corporate/state-of-student-learning-2022.pdf.
Whether the increases in percentages of below-level students in phonics were directly related to remote instruction is debatable and deserves further investigation. In any case, Chall’s (1995) work on reading development over time provides an important foundation for understanding the COVID-19 research and promoting young children’s overall reading development as well as reducing the number of older students who struggle to read.
Taken as a whole, the preponderance of evidence in the literature substantiates the differential influence of school closures and remote instruction on young students, below-level students, students of color, ELL (English Language Learners) and ESE (Exceptional Student Education) students, homeless students, and students from low-income homes. For these students, research-based interventions must be implemented to reduce the learning gaps found in the current literature. The research on student achievement during the pandemic is growing and should continue for many years to assist future researchers in their efforts to identify and promote the lessons learned during the pandemic.
4. Research Methods and Analyses
The researchers submitted an official request to the target school district to obtain individual student-level and district-level reading performance data from the district’s database. The resulting dataset included i-Ready composite reading scale scores for each third-grade student in the district for five administrations of i-Ready’s diagnostic assessments before and after school lockdowns and the introduction of online instruction for all students.
The researchers also requested students’ demographic data to include ESE and ELL designations, and information on whether the students had been retained at any point during their education. For comparison purposes, the 2018-2019 national norms were obtained from the developers of i-Ready at Curriculum Associates.
4.1. Research Population and Sample
In the target school district, approximately 41,000 students were enrolled in kindergarten through Grade 12 during the 2020-2021 school year (Florida Department of Education, 2021a, 2021b) . The race and ethnic origins of the target district were identified as 48% White, 20% Black or African American, 25% Hispanic, 3% Asian or Pacific Islander, and 5% other. The percentage of households with a computer was 88%, and 81% of the households had broadband internet access. The entire school district was designated as a Title I district. The research population in this study consisted of all third-grade students in the target district during the 2019-2020 academic year. Third-grade students in the target district were chosen for this study since reading performance scale scores were available for all students in grades three through five.
Demographic information provided by the school district was disaggregated according to the initial cohort, students in Exceptional Student Education (ESE), and students designated as English Language Learners (ELL). Students who had been retained at any point during their education were removed from the entire third-grade sample. In addition, students with missing data were removed from the research sample. The final sample size included 2,006 students in the initial cohort, 580 ESE students, and 169 ELL students. Reading scale scores of each individual student in the study were tracked and compiled as students moved through the fourth grade and the fall of the fifth grade. In other words, the data were composed of individual students’ scaled scores over five different assessment periods.
4.2. Instrumentation
Every elementary student in grades one through five in the target district participated in the i-Ready reading program published by Curriculum Associates and was administered the reading diagnostic tests at the beginning, middle, and end of each academic year. The i-Ready diagnostic tests are computer-based and adaptable to individual student progress; for example, as students progress through instruction in phonics, the online assessment of that skillset decreases in frequency, and comprehension-based assessments increase in frequency.
Each i-Ready diagnostic assessment provides an overall scale score that represents a comparison of student performance to grade-level norms and grade-level designations for each of the measured stages of reading (Curriculum Associates, 2019) . For example, a second-grade student might earn a scale score of 432 in the first assessment period, which represents reading at a first-grade level. The same student might test out of the phonological awareness test, perform at first-grade level for phonics, second-grade level for high-frequency words, and first-grade level for vocabulary, literature comprehension, and comprehension of informational text. These assessment results were designed to inform reading instruction by teachers.
4.3. Analyses
After disaggregating the research cohorts’ i-Ready reading datasets based on demographic variables and removing students with missing data, the researchers conducted descriptive and inferential analyses of the i-Ready reading scale scores of the third-grade student subgroups’ scores from five different assessment periods: January 2020 (Grade 3 mid-year assessment before COVID-19 school closures); August 2020 (Grade 4 beginning-of-the-year assessment after schools in the district reopened in August); January 2021 (Grade 4 mid-year assessment period); April 2021 (Grade 4 end-of-the-year assessment period, and August 2021 (the Grade 5 beginning-of-the-year assessment). The inferential analyses included 5 one-sample t-tests of reading scale scores over time to compare to national i-Ready norm scale scores.
5. Results
5.1. Descriptive Results
Preliminary analyses of the research data revealed that the data were normally distributed and that there were no missing data. Cronbach alpha analysis of the student groups revealed a high level of internal consistency in the datasets (α = 0.95). Descriptive statistics were computed to describe the Grade 3 cohorts’ reading performance before and after the school closures (see Tables 1-4 below). The students’ mean scale scores after the January 2020 assessment revealed small increments of reading growth over time after school reopened in August 2020.
![]()
Table 1. Descriptive statistics summary table: initial cohort (No ESE or ELL) over time.
Note. N = 2,006. M = mean; SD = standard deviation; SEM = standard error of measurement; Min = minimum; Max = maximum. aThese data were the baseline scores for the Grade 3 cohort. Students did not take the May 2020 i-Ready assessment due to school closures, and all the students were promoted to Grade 4 at the end of the 2019-2020 academic year.
![]()
Table 2. Descriptive statistics summary table: ESE only over time.
Note. N = 580 M = mean; SD = standard deviation; SEM = standard error of measurement; Min = minimum; Max = maximum. aThese data were the baseline scores for the ESE students from the grade 3 cohort. Students did not take the May 2020 i-Ready assessment, and all were promoted to Grade 4.
![]()
Table 3. Descriptive statistics summary table: ELL only over time.
Note. N = 169. M = mean; SD = standard deviation; SEM = standard error of measurement; Min = minimum; Max = maximum.
![]()
Table 4. Comparison of means and standard deviations of three subgroups of students over time.
Note. SD = Standard Deviation; Initial Cohort N = 2,006; ESE N = 580; ELL N = 169.
5.2. Inferential Results
The researchers compared the research cohorts’ mean reading scale scores to i-Ready’s mean scale scores of the most recent i-Ready national norm group (academic year 2018-2019) for the five assessment periods under study. The results of the t-test comparisons are presented in Tables 5-7 that follow.
![]()
Table 5. Results of one-sample t-test comparisons of initial cohort’s mean scale scores to i-Ready national mean scale score norms by assessment period.
Note. N = 2,006. M = mean; SD = standard deviation; t = t-test; p = probability.
![]()
Table 6. Results of one-sample t-test comparisons of ESE mean scale scores to i-Ready national mean scale score norms by assessment period.
Note. N = 580. M = mean; SD = standard deviation; t = t-test; p = probability.
![]()
Table 7. Results of one-sample t-Test comparisons of ELL mean scale scores to i-Ready national mean scale score norms by assessment period.
Note. N = 169. M = mean; SD = standard deviation; t = t-test; p = probability.
The students in each of the subgroups in this study performed significantly lower in reading performance when compared to the 2018-2019 pre-COVID national norm group, which included all three types of students. The differences were observed prior to the pandemic in the January 2020 assessment period, just before the introduction of remote teaching and learning due to COVID-19. In addition, the mean reading scale scores of the subgroups were consistently lower than the 2018-2019 national norm group in each of the five assessment periods. Although the students in this study demonstrated progress over time, the rate of progress was not commensurate with the progress of the 2018-2019 national norm group.
6. Discussion
Many educational stakeholders were deeply concerned about learning loss when students returned to classrooms in the fall of 2020. In a meta-analysis of the relationships between summer vacation and student achievement in Grades 1 through 8, Cooper et al. (1996) found that, on average, readers experienced a loss of 1.5 months in reading skills. However, the third-grade students in the current research study were able to maintain their reading skills during remote instruction and the summer months; in fact, the initial cohort’s mean reading scale score went up slightly from January 2020 to August 2020. Apparently, the three-month period of mandated remote instruction was effective in continuing to develop and maintain reading progress among this group of students. Whether the maintenance of reading skills was related to remote instruction, parental involvement, student maturation, or other factors cannot be known without further study.
After most students in Florida returned to classroom-based instruction in August 2020, the elementary students in the initial cohort made very small levels of progress in reading performance over time. These results align with the results published in a research report for Curriculum Associates (Dawson, 2022) , which described small increments of growth in reading from fall 2020 to fall 2021 as measured by percentages of students who were reading on grade level.
6.1. National Norm Comparisons
The differences between reading performance scores among the initial cohort students (no ESE or ELL students) were most apparent when their scores were compared to 2018-2019 (pre-COVID) national i-Ready norms. An examination of the comparisons’ results revealed that the two groups were significantly different across all assessment periods, with medium effect sizes in January 2020 and August 2020, large effect sizes in January 2021, and very large effect sizes in May and August 2021. In other words, as the cohort students progressed through fourth grade, the differences between 2018-2019 pre-COVID national norms grew larger and larger. The scale score differences between the initial cohort’s scores and the 2018-2019 national norms are easily observed in Table 8.
![]()
Table 8. i-Ready scale score differences between means of initial research cohort (no ESE or ELL) and 2018-2019 national norms.
*These values represent scale score differences.
Perhaps these scores would be different if the parents or the school district had opted to create opportunities for tutoring or reading camps during the summer of 2020. Unfortunately, many parents were still concerned about the spread of COVID-19 during that summer and might not have participated if the resources had been offered. In any case, the results of the national comparisons point to the need for further investigation into children’s reading development after unexpected and extended changes related to school interruptions coupled with the introduction of exclusively remote instruction, or other factors. Was a critical reading skillset overlooked or missed during a critical period of reading development during remote instruction that influenced a measurable, cumulative decline in reading performance later among cohort students?
6.2. Mitigating Factors
In a qualitative study of middle-school science teachers who delivered instruction remotely during the 2020 school lockdowns, Ward (2022) identified mitigating factors that may have influenced the academic performance of students: the initial chaos related to the rapid deployment of remote instruction, the teachers’ adjustment to teaching with technology, and inconsistent access to online curricula and technology tools in impoverished and mountainous areas (pp. 80-81). The teachers in the study reported that absenteeism and inconsistent internet access for both teachers and students had an enormous influence on student performance (Ward, 2022: pp. 81-82) . One teacher in Ward’s (2022) study stated, “Only one-third of the students attended class on a daily basis” (p. 112). In this relatively small district, school and district administrators and the local community valiantly attempted to provide both technical and curricular support to the teachers and students during the unprecedented changes related to the pandemic. However, all the teachers in the study agreed that they were unprepared to transition rapidly to remote instruction and that student engagement with the curricula and with age peers was diminished during remote instruction in the spring of 2020.
The problem of absenteeism and disengagement in classroom-based instruction has been extensively studied in the past (e.g., Marcotte & Hemelt, 2008 ). Educators know that time on task, attention, and focus is critically important to students’ academic success and overall development. Online and hybrid instruction cannot be effective if students do not participate. How can educators ensure that students remain engaged with the material, the instructor, and their peers, just as they would in classroom-based instruction? How can parents ensure that their children are present and engaged during online instruction? Educational stakeholders, including district and school administrators, teachers, students, and parents, could collaborate to conduct a review of recent research studies, technological tools, and instructional strategies related to the problems of absenteeism and student disengagement during remote instruction. The result of the collaboration might lead to the development of consistent school-wide guidelines and expectations for participation in remote instruction. School and district administrators can also assist by providing technological tools, technical support, flexible staffing, and different instructional models.
As teachers continue to use remote instructional delivery methods, whether via hybrid or exclusively online delivery options, students must be accountable for being present and actively engaged during direct instruction, small group interactions, and student projects. Teachers can acquire and polish their skills and dispositions to make online instruction developmentally appropriate, engaging, and challenging. In addition, teachers can create and share effective online material and ways to assess learning. Schools and districts can make accommodations to teachers’ schedules to promote and provide outstanding and easily accessed professional development to teachers as they continue to develop technical skills and curricular expertise to provide high-quality teaching and learning.
7. Conclusion
School closures during the Covid-19 pandemic created new opportunities to study the unexpected disruptions in educational programming and virtual instruction. To date, the results of COVID-19 research studies in education tend to vacillate between slight learning losses and catastrophic learning losses. Studies by Kuhfeld et al. (2020) , Kogan and Lavertu (2021) , and Pier et al. (2021) provided evidence of differential levels of learning loss in math and reading during the pandemic based on socioeconomic factors and access to technology. However, studies by Domingue et al. (2021) , Gore et al. (2021) , Hammerstein et al. (2021) , and Dawson (2022) provided evidence of either little or no learning loss in reading and math during the pandemic. In a recent Harvard study of 2.1 million students in grades 3 to 8 across 49 states, Goldhaber et al. (2022) concluded that “remote instruction was a primary driver of widening achievement gaps”, especially in high-poverty schools and districts.
Although most students in Florida and in the target school district returned to classroom-based instruction in August 2020 (Marshall & Bradley-Dorsey, 2020) , the subgroups’ reading progress in this study continued to increase in very small increments, yet commensurate with the district norms from the previous school years. These results may relate to those of Dawson (2022) , who found that mean reading improvement scores on i-Ready assessments were not dramatically different from mean scores prior to COVID-19 among second- through fourth-grade students who were reading on grade level, regardless of instructional delivery method (in school, mostly in school, mostly remote, or remote only). However, Dawson reported that reading improvement scores declined dramatically among students who were reading two or more grade levels below norms, regardless of instructional delivery methods. Dawson’s results were especially pertinent to the target district because the entire district is designated as a Title I district.
The results of this study of a single cohort of students who were in third grade in spring 2020 and whose reading scores were followed over time yielded interesting results that may serve to inform and guide educational practice and future research. In many classrooms, the predictions of disastrous learning loss in reading and math after school lockdowns were realized, especially among young children (Dawson, 2022) , minority students (Dorn et al., 2020) , and students from low-income homes (Gore et al., 2021) . Clearly, much work is needed to ensure that all children and young people overcome any losses during the pandemic. The lessons learned during the pandemic can assist educators as they move away from the chaos and uncertainty of the pandemic and move forward into more orderly, planned change and innovation in educational delivery. In addition, future research studies can continue to inform and assist educators as they make the transitions to remote, hybrid, or face-to-face instruction. This study adds to the body of knowledge related to the influence of school closures and remote instruction during school lockdowns and remote instruction.