COVID-19: How Do Engineering Students Assess its Impact on Their Learning? ()
1. Introduction
COVID-19 outbreak created unprecedented transitions in all aspects of life around the globe. Universities had to comply with the directives from the national and local health authorities to switch from traditional face-to-face instruction to remote learning. San Diego State University and its College of Engineering had to make a quick and as-smooth-as-possible transition into e-learning in mid-March of 2020, almost exactly in the middle of its Spring semester.
The literature on COVID’s impact on education results is still very limited because of the short observation period since the early months of 2020. Most of those publications use online venues to offer faster access to them as compared with the print journals. Although the online journals have enjoyed increasing popularity before the COVID pandemic, its outbreak appears to accelerate this process.
Concerns about the potentially negative impact of the pandemic on student learning are reflected in several publications. Some authors focus on policies introduced centrally for their countries. P. K. Jena (2020) lists highlights of measures taken by the government of India to provide seamless education in the country during the pandemic realizing also the fact that some corners of the country are not fully equipped with the digital platforms. The author sees the benefits of online education and suggests that it should be continued after the COVID lockdown. K. Naji et al. (2020) report their experience with the COVID-based transition for engineering students in Qatar, listing effective communication, proper pedagogical mode (e.g., student group projects), interactive mode of instruction, and availability of early assistance for students who struggle as the critical factors to overcome the problems with the online course transitions in response to COVID-19.
Pradeep Sahu (2020) and P. Hiremath et al. (2020) address the impact of COVID on the mental health and economic conditions of the students. They list regular and timely information about the outbreak, availability of the proper counseling services, food and accommodation assistance for international students, and a helpful attitude of instructors as important factors in reducing the pandemic-related stress among students. Marcus L. George (2020) presents his experience with effective teaching and examination under the pandemic at the University of West Indies. He lists an effective e-mail consultation and practices mock quizzes as ways to prepare students for an effective transition to a new learning model. The application of those strategies resulted in the student performance that was not worse than what was experienced in the past five years. However, he is not sure whether extensive student collaboration on the projects was also a factor.
M. Park et al. (2020) discuss their experience with engineering education under COVID focusing on disruptions affecting primarily courses with lab activities and related stress that may affect students’ enrollment in those courses. J. Qadir and A. Al-Fuqaha (2020) offer a primer for students who need to take courses under the pandemic times that are: “violate, uncertain, complex and ambiguous”, offering adherence to seven steps students can take to make remote learning as effective as the traditional face-to-face instruction.
S. Asgari et al. (2020) present results of an extensive qualitative and qualitative study performed at SDSU’s sister campus at the California State University at Long Beach, based on a survey of 110 faculty and 627 students in various Engineering fields. They discovered serious negative impacts of COVID-19 on issues related to learning and teaching as well as proper assessment methods affecting primarily vulnerable, disadvantaged, and underrepresented groups. They addressed numerous technical challenges facing students and instructors and proposed strategies to cope with those issues. They found out that the top challenge facing students taking remote courses was the difficulty in maintaining focus. There have also been more opportunities for students to cheat on exams. Seventy percent of instructors opted for the open-book/open-notes format of their tests.
Earlier, in June 2020 nationwide reports based on a nationwide survey study among the faculty and students representing the STEM fields found that female faculty and students reported more challenges to adopt to remote learning than their male counterparts G. K. Saw et al. (August 2020) and that a substantial fraction of STEM graduate students would face a delay of their graduation because of the pandemic G. K. Saw et al. (June 2020). E. Aucajo et al. (2020) also reported students’ anxieties about COVID-related delayed graduation, lost job or internship, and prospects of reduced salaries. C. Son et al. (2020) reported 71% increase in stress and anxiety among the college students attributed to COVID-19.
Some authors offer recommendations how to improve online teaching and learning. A. Ozadowicz (2020) advocates a blended learning method as a solution for effective laboratory classes. T. Tang et al. (2020) recommend a combined model of online teaching with the flipped learning. The generally negative student perceptions about their online learning were improved because of applying such a model.
2. The SDSU Study
The SDSU study reported in this paper focused on Civil Engineering undergraduate students who took courses during the Spring semester 2020. Selection of this group of students was logical as the first author of this paper as Department Chair was able to administer an additional, mid-semester survey for the Civil Engineering students to mirror similar surveys the students answer routinely at the end of each semester. The intended study of the COVID impact on Civil Engineering students at San Diego State University had the following starting objectives: 1) to be related to the ABET accreditation procedures the program needs to comply with; 2) be limited in scope yet based on a large database; 3) as a pure before/after study, be unlikely to be biased by unaccounted factors, and 4) be suitable to statistical tests of significance. All those conditions appeared feasible when a sudden COVID-19 transition was to be implemented.
Each respectable Engineering program aspires to be accredited by the Accreditation Board of Engineering and Technology (ABET). ABET accreditation is valued highly as it is demanding and rigorous. It examines the fulfillment of nine major evaluation criteria by the program at hand. The key component of the ABET evaluation is the expectation that each program has procedures aimed at Continuous Improvement (CI). CI documentation needs to deliver evidence that the education process in any program relies on assessment and evaluation of student learning data to continuously improve the program. Evidence of improvement is typically quantitative and should be performed at 1) Individual course level; 2) Student Outcome (SO) level; 3) Program Educational Objective (PEO) level; and 4) Department Mission level.
Assessment of SOs is the heart of the ABET accreditation. There are two main kinds of assessment: direct and indirect. Direct assessment relies on instructors’ own grading results of assignments designated to any given Student Outcome. The indirect assessment uses students’ own perceptions about their learning. Although not as reliable as the direct assessment results, indirect evidence is still a useful tool in the continuous improvement paradigm.
One of such ABET-inspired indirect assessment procedures performed every semester at the Department of Civil, Construction, and Environmental Engineering at SDSU for each course is a survey that asks each student to assess the degree to which Student Outcomes assigned for that course were delivered as planned. The newest version of ABET rules specifically requires that all Engineering courses have to cover and evaluate student learning in seven well-defined Student Outcomes through graded assignments, tests, and/or projects. In addition, for various Engineering disciplines, additional Student Outcomes are defined by the respective professional engineering societies. Although those additional Student Outcomes are not formally required by ABET, they are enforced by ABET. In the case of Civil Engineering programs, the American Society of Civil Engineers is the organization defining those additional Student Outcomes for ABET.
The Civil Engineering program at SDSU currently requires compliance with 16 well-defined Student Outcomes. The first seven of them are specifically mandated by ABET for all Engineering programs; they were adopted without any modification. The remaining nine were produced by the constituencies relevant to this process: faculty, Industrial Advisory Board members, and student representatives. The list of those SOs is presented in Appendix.
Coverage of those SOs was assigned to individual courses in a logical way. For example, SO 6 requiring performing experiments was assigned to courses with lab experimentation such as CivE 463, Geotechnical Engineering Lab. In order to have a robust set of data, it was decided that each course would need to cover at least three SOs, and each SO needs to be tested in at least two courses.
The data collecting procedure during the Spring semester 2020 was done in a consistent way: students were evaluating each course from the point of view of the coverage level of all SOs designated for that course on a scale from 1 to 5. On March 16, 2020, the Chancellor of the CSU system issued an executive order to switch all face-to-face classes to the remote instruction mode. Because of the incoming Spring Break, instructors and students were given just 10 days to adapt to the new teaching paradigm. It was a truly unprecedented and very challenging task for professors and students alike. There were adaptation issues to be sure. However, the CCEE Department with a vast majority of technology-savvy professors handled that unique situation remarkably well.
At the time of that dramatic COVID-19-based transition, students were approximately halfway through the semester, and all courses have already had an opportunity to cover the respective portions of the SOs assigned to them. That created an opportunity to perform the indirect student assessment before the new remote mode was about to start. An identical second survey was performed online toward the end of the semester to capture potential differences in student perceptions about their SO-related learning.
3. Analysis and Results
The intention of the study was to capture the potential impact COVID-19 pandemic had on students’ learning in the context of ABET-mandated Student Outcomes. The following null hypotheses were formulated:
Ho #1: Coverage of SOi was not affected by the pandemic-related course mode transition.
Ho #2: SO-related instructor performance in course j was not affected by mode transition
In addition, it is of interest to investigate whether courses that require hands-on lab experience were affected by the pandemic restrictions the same way as the lecture-based courses.
To analyze the statistical significance of the observed changes, the two-sided t-test was applied in order not to prejudge the direction (decrease or increase) of the changes in student scores.
Combined student score averages for sixteen SOs received in relevant courses before and after the COVID-transition are presented in Table 1. In addition, a number of students’ responses in each case is presented. Clearly, the results are
![]()
Table 1. Average student scores of student outcome coverage in civil engineering at SDSU: Pre-COVID period versus COVID period results.
based on robust data sets totaling 2473 responses for the pre-COVID survey and 2333 total responses for the COVID survey at the end of the Spring 2020 semester.
All COVID scores are lower than the pre-COVID ones. However, only four out of 16 (25%) score differences are statistically significant at the 5% significance level.
Table 2 and Table 3 show the results of the COVID versus pre-COVID score comparisons for all contributing courses separately. Differences in student scores that were statistically significant at the 5% level (tcalc > 1.96) are labeled y; those not significant are labeled n; empty space indicates that a given course was not designated to deliver that specific SO. Letters y or n in italics indicate that the COVID period scores are actually higher rather than lower than the pre-COVID scores.
Generally, Hypothesis 1 cannot be rejected at the 5% level of significance for 12 out of 16 Student Outcomes analyzed. Student scores lowering under the COVID period was observed for all 16 student Outcomes but was significant in only four of them.
Also, Hypothesis 2 cannot be rejected at the 5% level of significance in 11 out of 18 Civil Engineering courses, showing again a rather limited impact of COVID on the students’ assessment of SO coverage in individual courses.
![]()
Table 2. T-statistic significance summary of scores of student outcome coverage in civil engineering courses A through I at SDSU: Pre-COVID versus COVID periods.
![]()
Table 3. T-statistic significance summary of scores of student outcome coverage in civil engineering courses J through S at SDSU: Pre-COVID versus COVID periods.
Examination of Tables 1-3 leads to the following, more detailed comments:
1) The COVID-19-mandated sudden transition of the teaching mode from face-to-face (F2F) to online (OL) in mid-March 2020 did affect perceptions of Civil Engineering students at SDSU about the effectiveness of the coverage of the ABET-related Student Outcomes in courses offered to them during the Spring semester 2020. However, only 17 of the 80 changes in Student Outcome scores analyzed (21.3%) were statistically significant at the 5% level. This would suggest that the overall impact of COVID-19 on student’s opinions about their learning experience during Spring 2020 was somewhat limited as the COVID scores for the vast majority courses of were high by a standard adopted by the CCEE department: they were typically higher than the 4.0/5.0 which is well above the 3.5/5.0 acceptance level. In fact, while the lowest overall score for any SO in the pre-COVID survey was 4.38/5.00, the lowest score for the COVID period survey was 4.25/5.00.
2) Out of the 80 cases analyzed, 62 (77.5%) of the COVID scores for Student Outcomes coverage were lower on average than the comparable pre-COVID scores for the same Student Outcomes. Three cases (3.8%) reported no change in scores (t = 0), and 15 cases (18.7%) reported the COVID scores to be actually higher than the symmetrical pre-COVID scores. Thus, the COVID impact on students’ perception of their learning was not uniformly negative. None of the positive changes was statistically significant at the 5% level.
3) Twelve out of those 15 positive changes were found in CIVE 495 Senior Design course. That course is responsible for covering 13 out of 16 adopted Student Outcomes. Contrary to some other Engineering disciplines, Civil Engineering students do not produce a fabricated model or similar. It appears that the ZOOM format for their group interactions created a disciplined, working model for their designs.
4) In four out 16 Student Outcomes (25.0%), student scores drops were statistically significant. As expected, most negatively affected by COVID were SO 6 and SO 11 that expect students to “develop and conduct experiments”. The remote versions of courses that normally offer a “hands-on” experience were unable to deliver those Outcomes fully.
5) Student Outcome 5 “an ability to function effectively on a team…” appears to be also negatively affected by the COVID restrictions for ad hoc team assignments. The Senior Design is a noted exception here as teams are created even before the projects start and are formed in such a way as to have all the necessary expertise for the chosen project. Interactions among team members are indispensable and frequent. It appears that these interactions when done remotely via pre-scheduled ZOOM meetings were actually more efficient than face-to-face contacts that are more affected by outside factors such as traffic jams or similar.
6) Statistically significant scores drops were also found in SO 10 “ability to analyze and solve problems in at least four technical areas...”. The reasons for that result are not known.
7) COVID impacts on individual courses vary. Out of 18 Civil Engineering courses offered at SDSU in Spring 2020, seven (38.9%) courses had statistically significant drops in the students’ scores. While the Course Q; Senior Design course was not negatively affected by the COVID-related restrictions and precautions, courses with significant lab components (Course A: Computer Graphics; Course B; Surveying; Course D: Solid Mechanics Lab; Course K: Geotechnical Engineering Lab; and Course N: Highway Design) experienced statistically significant scores reductions overall.
8) Other courses that reported a statistically significant drop in the student evaluation scores were the ones that emphasize an interactive style of instruction (Course E: Structural Analysis I) or rely heavily on presentations in discussions (Course F: Civil Engineering Seminar).
4. General Conclusions
According to students’ scores, main accreditation expectations in the form of the mandated Student Outcomes were met satisfactorily not only before the COVID-19 restrictions but also under the COVID-19 restrictions. In both cases, the average student scores for all SOs were above 4.0/5.0. Only in 21% of cases, the COVID-caused transition led to scoring lowering that was statistically significant at the 5% level.
Student Outcomes that emphasize lab experience and teamwork were the ones most negatively affected by the transition from the face-to-face to remote instruction mode. It is not clear whether the drop in the student scores represented their perception that that new mode of instruction was ultimately inferior or because the transition to the new mode of instruction was so dramatic and abrupt.
COVID’s impact on the performance of individual instructors in delivering the mandated Student Outcomes varied. A majority of instructors effectively used the Spring Break week to adjust their course instruction to the new mode of instruction and was able to continue quality teaching. The most difficult and demanding adjustments were in the courses with significant lab components where the drops in students’ scores were statistically significant at the 5% level.
Judging by the SDSU case, COVID-19 will likely lead to changes in the way Engineering courses are to be taught in the future as several instructors originally “forced” to teach in the remote mode indicate now that they want to continue their online or hybrid teaching even after the COVID pandemic is finally over. New models for teaching some critical laboratory classes such as blending or flipped learning that appear to work satisfactorily at some universities are likely to emerge.
In summary, the course assessments made by the Civil Engineering students at San Diego State University suggest that the negative impact of COVID-19 restrictions on student learning was noticeable but lower than expected and in most cases not significant statistically. Both instructors and students adapted remarkably well to the rapid transition of all courses to the remote mode. Despite the unavoidable challenges and problems related to this transition, students’ own evaluation of their learning remained high.
Appendix
Student Outcome Definitions, Civil Engineering Program at SDSU, Spring 2020
Student Outcomes are established by ABET Criterion 3.a-k, relevant Program Criteria, and our program constituencies. They are formulated as follows:
By the time of graduation, the Civil Engineering graduates will have:
1) An ability to identify, formulate, and solve complex engineering problems by applying principles of engineering, science, and mathematics.
2) An ability to apply engineering design to produce solutions that meet specified needs with consideration of public health, safety, and welfare, as well as global, cultural, social, environmental, and economic factors.
3) An ability to communicate effectively with a range of audiences.
4) An ability to recognize ethical and professional responsibilities in engineering situations and make informed judgments, which must consider the impact of engineering solutions in global, economic, environmental, and societal contexts.
5) An ability to function effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives.
6) An ability to develop and conduct appropriate experimentation, analyze and interpret data, and use engineering judgment to draw conclusions.
7) An ability to acquire and apply new knowledge as needed, using appropriate learning strategies.
8) An ability to apply knowledge of mathematics through differential equations, calculus-based physics, chemistry, and at least one additional area of basic science.
9) An ability to apply probability and statistics to address uncertainty.
10) An ability to analyze and solve problems in at least four technical areas appropriate to civil engineering.
11) An ability to conduct experiments in at least two technical areas of civil engineering, analyze, and interpret the resulting data.
12) An ability to design a system, component, or process in at least two civil engineering contexts.
13) An ability to include principles of sustainability in design.
14) An ability to explain basic concepts in project management, business, public policy, and leadership.
15) An ability to analyze issues in professional ethics.
16) An ability to explain the importance of professional licensure.