Abstract
This research study investigates the impact of the COVID-19 pandemic, course delivery modality, and student engagement on student performance in data collected from business statistics courses at a medium-sized southeastern public university. Our data analysis suggests student engagement and disruptions caused by the COVID-19 pandemic are the strongest predictors of overall student performance. While course delivery modality did not seem to have a significant impact on student performance in and of itself, statistical analysis suggests there is a significant impact on the performance of students enrolled in online courses during the Spring 2020 semester despite the fact there were no emergency changes to the delivery modality of their class. An important takeaway for distance learning administrators is “other factors” outside of the time spent engaged in completing course requirements seem to have had a significant, negative impact on student outcomes in the Spring 2020 semester. Further research is needed to better understand the interaction between general societal disruptions caused by COVID-19 restrictions and student outcomes for the online course delivery modality.
Introduction
The COVID-19 pandemic restrictions forced many universities to transition traditional face-to-face (F2F) classes to a virtual, online delivery modality beginning in the Spring semester of 2020 (Gallagher & Palmer, 2020). However, it is important to note that prior to the COVID-19 restrictions, online courses had already gained in popularity through existing distance learning programs. The National Center for Education Statistics’ Integrated Postsecondary Education Data System (de Brey, Snyder, Zhang, & Dillow, 2021) found that in the Fall semester of 2018 more than 6.9 million students (35.3% of students in the United States) were enrolled in online college classes. The asynchronous online instructional modality successfully employed by distance learning programs have offered students the flexibility to work remotely at their own pace for decades prior to the COVID-19 pandemic. These distance learning programs have provided a great alternative to traditional F2F instruction for non-traditional students who are managing families, jobs, athletics, and other extracurricular activities. The asynchronous mode of instruction removes the constraints of time and place from traditional F2F on-campus programs (Swart & MacLeod, 2020). Furthermore, technology savvy professors had been using publisher online learning platforms prior to the COVID-19 pandemic in traditional F2F, hybrid (part F2F / part virtual), and distance learning courses as a way to provide flexibility for students to learn outside the F2F physical classroom (Johnson, 2016).
This paper compares the performance of 308 students enrolled in business statistics courses at a medium-sized university located in the Southeastern United States. These students were enrolled in two undergraduate business statistics courses all taught by a single instructor over a period of four semesters with course materials delivered through an online learning platform provided by Cengage Publishing called WebAssign and the university’s internal Learning Management System (LMS) commonly known as Blackboard. Students are required to pass both of these two sequential business statistics courses to earn an undergraduate business degree. The textbook adopted for both business statistics courses is titled “Statistics for Business & Economics (14th edition)” authored by Anderson, Sweeney, Williams, Camm, and Cochran (2019). Historically, these undergraduate business statistics courses have a lower level of student success as measured by the DFW rate (D, F or Withdrawal from the course) and these outcomes are comparable with core business courses at other peer institutions of higher education (Mitra & Le, 2019).
The paper is organized as follows: Section I provides a literature review; Section II describes methods including with the data collected, sample descriptive information, and our research questions; Section III presents the results of our analysis; Section IV provides a summary and discussion; Section V describes the limitations of our paper; and, Section VI provides suggestions for future research ideas based on our findings and the limitations of this research study.
I. Literature Review
Mitra and Le (2019) stated the level of success students achieve in their years at college has far-reaching implications for students’ personal and professional lives, persistence in elected majors, perseverance in higher education, and their success in their future career. In universities nationwide, dissatisfaction with low performance in college classes is a serious problem (L. Horn, Peter, & Rooney, 2002; L. J. Horn & Premo, 1995). Research has shown a student’s overall academic success, success in their future professional career, and even success in their personal life can be significantly, and negatively, impacted when the student’s performance is low in courses designed to develop essential skills and introduce students to disciplinary studies (Seymour & Hewitt, 1997; Tobias, 1990). These courses, such as the two required sequential business statistics courses in our current study, often have high failure rates reducing overall academic success and causing “bottlenecks” in the graduation timeline. The problem is especially evident in required courses representing integral, core components of an undergraduate curriculum (Mitra & Le, 2019). Understanding a student must successfully complete the first business statistics course, business statistics one (BusStats1), as a prerequisite to enrolling in the second business statistics course in the sequence, business statistics two (BusStats2), we hypothesize:
Hypothesis 1: Overall student performance in BusStat2, regardless of course delivery modality, will be lower than student performance in BusStat1.
Student performance in traditional F2F versus online courses. Providing students with the option to self-select their preferred course delivery method, specifically F2F versus online, seems to contribute to overall student satisfaction and success (Beetham, McGill, & Littlejohn, 2009). Consequently, most universities offer some form of online instructional modality, and many of these universities have established formal distance learning administration initiatives. However, the education research literature suggests online delivery modality could negatively impact students’ academic performance when compared to the performance of students in traditional F2F classes (Debashis Bir, 2019). While many students struggle with online learning, research also seems to suggest students have the opportunity to succeed in courses delivered in either a traditional F2F or online asynchronous format under the right conditions (Bergeler & Read, 2021; Cornish, Jameson, & Records, 2020).
Many of the recommendations of Bishop-Monroe (2020) for improving student performance were already in place prior to the initial planning and data collection stages of the current research study. More specifically, prior to the COVID-19 pandemic, the business statistics courses included in our study already had (a) a strong technical function; (b) were hosting many classes asynchronously for both online and F2F courses; (c) were using additional instructional videos within each course; and, (d) were providing a generous amount of contact time with students via email, Blackboard, Zoom, and F2F office hours. Well in advance of the COVID-19 crisis, the university’s commitment to quality distance learning initiatives was demonstrated by establishing a goal of acquiring Quality Matters (QM) certification. This goal had already led to the development and implementation of many faculty-driven policies, procedures, best practices, and continuous improvement initiatives in existing distance learning courses.
The research literature has identified technical challenges commonly experienced by distance learning students which specifically includes; (a) weak internet connection, (b) poor technical proficiency, (c) poorly trained faculty, (d) slow institutional technical support, and (e) incompatible devices (Qureshi, Khawaja, & Zia, 2020). Proper administration of distance learning initiative requires a university to build an effective academic technology function, with resources and support for faculty and students (Rose & Moore, 2019) to improve both student and teacher success in distance learning courses. This led the university in the current study to develop an excellent Information Technology Services department that provides technical support to students and faculty which includes: (a) providing free laptops to new students; (b) maintaining excellent learning labs; (c) managing all the technology on campus; (d) providing robust technical support to students, staff, and faculty; and, (e) as previously noted an overall goal of acquiring QM certification for online instruction. This university has also implemented a Faculty Development Institute (FDI) to train and support online, hybrid, and F2F course development using the Blackboard LMS. The FDI also provides training and support to students related to the university’s LMS and all other technology students may encounter in any class across campus.
In the data set, as described in (Wakeling, Doral, Robertson, & Patrono, 2018), all online and F2F sections follow the same instructor-regulated teaching format. Under this format, the instructors developed, implemented, and communicated the course requirements employed to measure student performance (e.g., assignments, quizzes, and exams). In an effort to promote student retention and success, prerecorded instructional videos were available to all students in every class section of both business statistics courses within this study (S.S. Jaggars, Edgecombe, & Stacey, 2013; S. S. Jaggars, Folk, & Mullins, 2018). These videos provided training for (a) students who were unfamiliar with learning platforms (i.e., WebAssign and Blackboard), and (b) students who needed additional clarification regarding technology requirements and course requirements (S. S. Jaggars et al., 2018). Custom videos of the instructor’s lectures and sample problem demonstrations for each chapter were available to all students in all classes through the LMS course site. These recorded instructor videos give students the opportunity to review the lectures and sample problems repeatedly at their convenience regardless of course delivery modality. This research study addresses two main research questions related to the impact of (a) student engagement and (b) COVID-19 restrictions on overall student performance.
Research Question 1. Does the amount of time a student spends engaged in course requirements have a significant impact on student performance? If so, to what extent?
For the purposes of this empirical study we specifically define student engagement as the duration of time a student spends actively working in the WebAssign system as a measure of student engagement in the course materials. Again, all practice assignments, quizzes, and exams were delivered via Cengage’s WebAssign platform. Therefore, the time a student invests working towards successfully completing all course requirements (i.e., graded practice assignments, quizzes, and exams) is used to operationalize a student’s overall level of engagement in the course materials. The grades students earn on their practice assignments, quizzes, and exams are used to calculate their overall grade in the course. Consequently, in our data collection and analysis we operationalized the concept of overall student performance using students’ final course grade measured on a continuous scale from zero to one hundred.
Much research has been focused on identifying factors that affect student performance and success in “bottleneck courses,” like business statistics which include: (a) demographic variables such as gender, ethnicity, and age (Brower & Ketterhagen, 2004; Herndon & Moore, 2002); and, (b) academic variables such as GPA, prior academic history, and class attendance (Devadoss & Foltz, 1996; Romer, 1993). This study focuses on procrastination which is defined as a voluntary, irrational delay of behavior. Research has provided evidence suggesting procrastination impairs individual academic and work group performance (Jiao, DaRos-Voseles, Collins, & Onwuegbuzie, 2011; Lakshminarayan, Potdar, & Reddy, 2013). This research literature further suggests students who procrastinate the most on completing academic work, have the lowest levels of achievement and performance. Procrastination is widely prevalent among college students worldwide and is an important factor to consider for student success. This study measures the duration of time students spend engaged in the WebAssign learning platform as an indicator of procrastination versus engagement as students work to complete all course requirements. Therefore, we hypothesize:
Hypothesis 2: The time a student spends actively engaged in completing the course requirements will have a significant, positive impact on overall student performance.
Research Question 2: Did the emergency transition to an online class delivery modality in response to COVID-19 pandemic restrictions impact overall student performance?
A literature review conducted by Lack (2013) identified approximately thirty relevant and acceptably rigorous studies leading them to conclude “the literature ...yields little, if any, evidence to suggest that online or hybrid learning, on average, is more or less effective than face-to-face learning” (p. 10). As noted by Lack (2013), few studies control for preexisting effects when measuring the impact of online courses. Lam (2009) used regression analysis to assess student performance in traditional and online delivery methods for an undergraduate computer programming course and found evidence suggesting delivery method had no significant effect on student performance. Ary, Hickingbotham, and Brune (2011) compared learning outcomes in traditional and online formats in a personal finance course, and their data analysis suggests delivery method did not significantly influence course averages. However, they did note the percentage change in scores between pre- and post-tests was significantly higher for the traditional format (Ary et al., 2011). The articles by Lam (2009) and Ary et al. (2011) included controls for a number of background characteristics and/or other predictor variables. The current study investigates the potential difference in student performance in F2F versus online delivery of business statistics courses using the WebAssign. Therefore, we hypothesize:
Hypothesis 3: Online course delivery modality will have a significant, negative impact on student performance when compared to student performance in a traditional face-to-face course delivery modality.
Student Performance and COVID-19 Restrictions. Although online courses and online publisher platforms have become increasingly popular, not all students thrive in this learning environment. Asynchronous online learning requires self-discipline and time management skills to successfully complete coursework within the imposed deadlines. Some students may not have the motivation, organizational skills, and work habits needed to properly manage their level engagement in online classes and platforms. This deficiency in student engagement can adversely impact their academic success. Oh and Reeves (2014) determined that the students’ lack of self-regulation significantly influences academic outcomes. This lack of self-regulation may have a more significant impact on students who (a) solely enroll in F2F classes, (b) only occasionally enroll an online class, or (c) have the online delivery modality forced upon them by the emergency COVID-19 pandemic restrictions mandated during the Spring semester of 2020. As previously noted, providing students with the option to self-select their preferred course delivery modality seems to contribute to overall student satisfaction and success (Beetham et al., 2009). Therefore, we hypothesize:
Hypothesis 4: Online course delivery modality will have a significant, negative impact on student performance when students’ autonomy to self-select course delivery method is removed due to emergency COVID-19 pandemic restrictions.
II. Method
The original data set contained 367 undergraduate business statistics students across four semesters [(Pre-Pandemic - Fall 2019), (Emergency COVID-19 transition to online delivery modality for all Spring 2020 courses), (Summer 2020 – online delivery modality for all courses), (Fall 2020 – online delivery modality for all courses)] and eleven sections of undergraduate business statistics taught by a single faculty member. Over these four semesters the course requirements and course materials remained unchanged across both business statistics courses, for all sections, for all course delivery modalities. Therefore, the data collected controls for variation potentially caused by different instructors, changes to course requirements, and changes to course materials. The university requires all business majors to complete a sequence of two consecutive business statistics courses. The first business statistics course in the sequence, Business Statistics 1 (BusStat_1), focuses on descriptive statistics, basic probability, and interval estimation. The second business statistics course in the sequence, Business Statistics 2 (BusStat_2), builds on BusStat1 and is concerned primarily with statistical inference and regression. The data set consists of three sections of the first course (BusStat_1) and eight sections of the second course (BusStat_2). Six of the sections were taught in a traditional F2F lecture format, while the other five were delivered completely online. All eleven sections were comprised of the same course requirements which included four exams and eleven quizzes. These exams and quizzes were delivered to students via Cengage Publishing’s WebAssign platform. Within a given course, all of the graded components were identical for the study period. For each student, the final course average and total time duration, measured in minutes, spent engaged in WebAssign was collected. Students who either withdrew from the course or failed to submit any assignments in WebAssign were excluded, leaving a usable sample size of n=308 which represents all business students enrolled in their required business statistic courses over a period of four semesters (one year and four months).
Kelani, Doral, and Post (2021) used Final Course Completion Grades (FCCG) as a proxy to assess students’ academic performance in both F2F and online sections, and this paper uses time duration of student engagement within the learning platform to predict FCCG. In the current research study, ordinary Least Squares (OLS) regression modeling and descriptive statistics were used to analyze the data. Microsoft Excel was utilized for data manipulation and analysis. FCCG’s are based on letter grades A, B, C, D and F (See Table 2).
III. Data Analysis
Table 3 shows the results of the time spent in the WebAssign platform versus final grade in the course using a standard zero to one hundred grading scale (A = 90 to 100; B = 80 to 89; C = 70 to 79; D = 60 to 69; F = 0 to 59).
In the first course BusStat_1, students who failed the course with a final grade of less than 60 spent an average duration of 752.3 minutes engaged in the course material within the WebAssign system which is an average of (43.88%) less time than students who earned a grade of 60 or better. By comparison, the passing students who earned a final course grade of 60 or higher had an average duration of 1714.1 minutes spent engaged in the course material within the WebAssign system during the semester. What was surprising is that students who earned a final grade of 60-70 spent an average duration of 218.3% more time than their failing counterparts. It is also interesting to note students who earned a final course grade of B (80-90) spent the highest duration of time engaged in the course materials within WebAssign.
In the second business statistics course BusStat_2, students who failed the course with a final grade less than 60 spent an average duration of 814.3 minutes engaged in the course materials within WebAssign which is an average duration of 49.62% less time than students who earned a grade of 60 or better. By comparison, the students who passed the course with a final course grade of 60 or better spent an average duration of 1641.1 minutes engaged within the WebAssign system during the semester. Again, students who earned a final course grade of 60-70 spent substantially (187.6%) more time engaged in WebAssign than their failing counterparts. It is interesting to note students who earned a final course grade of C (70-80) had the highest average duration of time (1746.0 minutes) engaged in WebAssign.
A significant difference was found in the final grades for online vs. traditional courses, but this only applies to the second (BusStat_2) course in the sequence. Traditional course grades average 7.69 (±5.487) points higher than online courses. For the second course, traditional grades average 6.94 (±6.440) points higher than online courses. This difference is only marginally significant (p-value=.0348).
Spring 2020 seems to dominate all other differences in the data set, with the possible exception of student engagement with the WebAssign online platform (See Table 4). The COVID-19 transition semester in Spring 2020 exhibits a much larger effect than either the difference between courses (BusStat1 versus BusStat2) or presentation modality (online versus traditional F2F). Across all courses and modes, Spring 2020 grades averaged 18.07 (±6.310) points lower than grades in all of the other semesters in the data set combined (see Table 3). It appears that, at least for this data set, COVID-19 represents a one-time disruption in grade performance, as Fall 2020 grades were nominally, but insignificantly, higher than Fall 2019 grades. It is surprising to note, student performance in the online sections appear to have been more negatively impacted than the traditional F2F sections in the Spring 2020 semester, despite the fact the online sections did not experience a sudden, unexpected change in delivery modality due to COVID-19. During the week of March 8, 2020, the administration of the university announced all courses would be converting to a fully online delivery modality following Spring Break. This was approximately the halfway point in the semester, with respect to the remaining course grading components.
IV. Summary and Discussion of Results
After controlling for the variables course, course delivery modality, and the Spring 2020 semester, the coefficient for student engagement was found to be profoundly and positively significant (p-value = 1.85x10-21). The magnitude of the coefficient for student engagement suggests an approximate eighty-minute increase in student engagement is associated with an expected one-point increase in student performance. Therefore, hypothesis 2 was strongly supported.
After controlling for student engagement, the Spring 2020 semester, and course delivery modality, we found statistical evidence suggesting the second business statistics course (p-value = 0.019221) had a statistically significant, negative impact on student performance when compared to the first business statistics course (BusStat1). The coefficient suggests student performance in BusStat2 is predicted to be 6.60 points lower than student performance in BusStat1. Therefore, hypothesis 1 was supported by our research model.
After controlling for course, the Spring 2020 semester, and student engagement, we found no statistical evidence (p-value = 0.533401) to suggest student performance is significantly lower for online course delivery modality versus traditional F2F delivery modality. Consequently, hypothesis 2 was not supported, which is consistent with much of the research literature focused on investigating differences found in student performance across courses with an online versus traditional F2F delivery modality (Ary et al., 2011; Lack, 2013; Lam, 2009).
The variable for the Spring 2020 (p-value = 5.01x10-10) semester, which was heavily disrupted by COVID-19 restrictions, was found to have a very significant, negative impact on student performance as it led to an estimated 21-point grade reduction in student performance in contrast to other semesters in which course delivery was not transitioned to an online modality. Therefore, hypothesis 4 was supported.
We also included a variable in our model for the interaction between the Spring 2020 semester and the online course delivery modality with surprising results. The interaction term Spring2020*Online (p-value = 0.017083) was found to have a significant, negative impact on student performance. That is, students enrolled in courses with an online delivery modality during the Spring 2020 semester are predicted to have an additional 11.12 point reduction in their overall student performance. This finding is quite perplexing understanding that students who were originally enrolled in courses with an online delivery modality did not experience any unexpected changes in the delivery modality of their course when the administration of the university abruptly announced all courses would transition to an online delivery modality due to COVID-19 pandemic. It is well documented that interpreting the interaction terms within regression models can be quite challenging, and often practical explanations are not reliable (Hair, Black, Babin, & Anderson, 2009; Kutner, Neter, Nachtshein, & Wasserman, 2013).
Finally, it seems interesting to note the variables included in our research model might be able to describe more than 40% (Adjusted R Square = 0.398178) of the observed variation in student performance without including variables that measure student ability (e.g., ACT scores, etc.), prior preparation, or performance (e.g., cumulative GPA, etc.).
Spring 2020 seems to dominate all other differences in the data set, with the possible exception of student engagement. Across all courses and modes, Spring 2020 grades averaged 18.07 (±6.310) points lower than grades in all the other semesters in the data set combined. It appears COVID-19 represents a one-time disruption in grade performance, as Fall 2020 grades were not significantly higher than Fall 2019 grades. Surprisingly, grades in the online section appear to have suffered worse than the traditional courses during Spring 2020, despite no delivery change at all due to COVID-19.
V. Limitations / Implications
The most significant takeaway for distance learning administrators is that overall student performance among students enrolled in distance learning courses was found to be significantly lower than students originally enrolled in F2F courses in the Spring 2020 semester. That is, students enrolled in business statistics online courses which did not experience an emergency transition in delivery modality did not perform as well as students who were enrolled in courses that were forced to make an emergency transition in course delivery modality from F2F to online.
Every research project has limitations, and the current research is no exception. All of the data collected came from courses in one subject area (i.e., business statistics) taught at only one university in a single geographical area. Consequently, the data analysis and specific results is not highly generalizable to all subject areas at all universities in all geographical areas.
However, similar to the results of Haughton and Kelly (2015), the current study provides results that are broadly applicable in the sense that business statistics courses are a component of most undergraduate business degree programs, the data sample represents all undergraduate students enrolled in the required business statistics courses at a university, these students represent a wide range of ability, the relatively large sample size, and the ability to control for many potential influences on performance including instructor, textbook, course requirements, and course delivery methods.
Also, the authors defined and operationalized “student engagement” for the purposes of this empirical research study as “the time a student spent in the WebAssign system engaged in completing the course requirements (assignments, quizzes, exams, etc.).” The authors realize that student engagement consists of many additional factors, including student-to-student and student-to-instructor interactions, that have the potential to significantly impact overall student performance.
The distance learning administration research literature has identified a distinction between distance learning courses that were well-developed, well-planned, and well-executed versus courses that were forced to make an emergency transition from a F2F delivery modality to an online modality during the Spring 2020 semester due to COVID-19 restrictions. This transition is often described as a haphazard, emergency online delivery modality that is assumed to be poorly-developed, poorly planned, and poorly executed. However, the business statistics courses included in the current research study, regardless of course delivery modality, were all strategically developed to use the same online WebAssign system. Consequently, the emergency transition from F2F delivery modality to an online modality was an efficient turn-key transition which was well-developed, well-planned, and well-executed.
VI. Future Research
Future research could improve on this study in numerous ways. First, collecting data from a number of universities located in various geographical locations would greatly increase the generalizability of the results of our current study. Second, future research could include data on both (a) student demographics (age, gender, etc.) and (b) general student aptitude (GPA, ACT/SAT score, college credits completed, number of math classes taken, etc.) in an effort to further control for the variability in student performance. Third, a survey instrument could be used to measure students’ self-efficacy as it relates to their ability to successfully complete and master the subject matter in math-based courses. Finally, future research could collect self-reported data related to “how” students spend their time (quantity versus quality) engaged in online platforms such as Cengage Publishing’s WebAssign. As previously noted, the group of students who earned a final letter grade of “A” in our study, did not have the highest average for student engagement. Perhaps the balance of quantity versus quality of student engagement is moderated by the alignment between an individual student’s learning style and the manner in which course materials are presented (i.e., watching instructional videos, reading the chapters, working the sample problems, etc.).
Finally, one of the most significant opportunities for future research as it relates to distance learning administration is to investigate the impact of overall societal disruptions directly linked to the COVID-19 pandemic that had a significant, negative impact on student performance during the Spring 2020 semester. For example, could the unexpected lower student overall performance among students who were enrolled in courses with an online delivery modality for the entire Spring 2020 semester (with no emergency transitions in modality) be explained by the mental health issues of society in general which were directly related to uncertainty, fear, and panic in the general population at large during the COVID-19 pandemic?
References
Anderson, D. R., Sweeney, D. J., Williams, T. A., Camm, J. D., & Cochran, J. J. (2019). Statistic for Business & Economics (14 ed.): Cengage Learning.
Ary, D. J., Hickingbotham, F. D., & Brune, C. W. (2011). A comparison of student learning outcomes in traditional and online personal finance courses. Journal of Online Learning and Teaching, 7(4), 465-474.
Beetham, H., McGill, L., & Littlejohn, A. (2009). Thriving in the 21st century: Learning Literacies for the Digital Age (LLiDA Project).
Bergeler, E., & Read, M. F. (2021). Comparing Learning Outcomes and Satisfaction of an Online Algebra-Based Physics Course with a Face-to-Face Course. Journal of Science Education and Technology, 30(1), 97.
Bishop-Monroe, R. (2020). Reflections, Challenges, and Strategies for Online Academic Instruction: A Faculty Perspective on the Rapid Transition from Face-to-face to Online Instruction During the COVID-19 Crisis. Online Journal of Distance Learning Administration, 23(4), 1-12.
Brower, A. M., & Ketterhagen, A. (2004). Is There an Inherent Mismatch Between How Black and White Students Expect to Succeed in College and What Their Colleges Expect from Them? Journal of Social Issues, 60(1), 95-116.
Cornish, S., Jameson, M., & Records, K. (2020). Student Success with Asynchronous Online Courses. Teaching, Learning & Assessment: University of Northern Colorado.
de Brey, C., Snyder, T. D., Zhang, A., & Dillow, S. A. (2021). Digest of Education Statistics, 2019. (NCES2021009). 2/25/2021: National Center for Education Statistics at IES Retrieved from https://nces.ed.gov/pubs2021/2021009.pdf
Debashis Bir, D. (2019). Comparison of Academic Performance of Student in Online VS Traditional Engineering Course. European Journal of Open, Distance & E-Learning, 22(1), 1-13.
Devadoss, S., & Foltz, J. (1996). Evaluation of Factors Influencing Student Class Attendance and Performance. American Journal of Agricultural Economics, 78(3), 499-507.
Gallagher, S., & Palmer, J. (2020). The Pandemic Pushed Universities Online. The Change Was Long Overdue [Article]. Harvard Business Review Digital Articles, 2-6. Retrieved from https://hbr.org/2020/09/the-pandemic-pushed-universities-online-the-change-was-long-overdue
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009). Multivariate Data Analysis (7th ed.).
Haughton, J., & Kelly, A. (2015). Student Performance in an Introductory Business Statistics Course: Does Delivery Mode Matter ? Journal of Education for Business, 90(1), 31-43. Retrieved from <Go to ISI>://WOS:000210812300005
Herndon, M. K., & Moore, J. L. (2002). Practice: African American Factors for Student Success: Implications for Families and Counselors. FAMILY JOURNAL -ALEXANDRIA VA-, 10(3), 322-327.
Horn, L., Peter, K., & Rooney, K. (2002). Profile of Undergraduates in U.S. Postsecondary Institutions: 1999-2000. Statistical Analysis Report. National Postsecondary Student Aid Study. (NCES2002-168). National Center for Education Statistics, Washington D. C. Retrieved from https://files.eric.ed.gov/fulltext/ED468124.pdf
Horn, L. J., & Premo, M. D. (1995). Profile of Undergraduates in U.S. Postsecondary Education Institutions: 1992-93. With an Essay on Undergraduates at Risk. Statistical Analysis Report. (NCES 96-237). National Center for Education Statistics, Washington D. C. Retrieved from https://files.eric.ed.gov/fulltext/ED468124.pdf
Jaggars, S. S., Edgecombe, N., & Stacey, G. W. (2013). Creating an Effective Online Environment. 8. Retrieved from https://login.proxy006.nclive.org/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED542153&site=eds-live&scope=site
Jaggars, S. S., Folk, A. L., & Mullins, D. (2018). Understanding students’ satisfaction with OERs as course materials. Performance Measurement and Metrics, 19(1), 66-74.
Jiao, Q. G., DaRos-Voseles, D. A., Collins, K. M. T., & Onwuegbuzie, A. J. (2011). Academic Procrastination and the Performance of Graduate-Level Cooperative Groups in Research Methods Courses. Journal of the Scholarship of Teaching and Learning, 11(1), 119-138.
Johnson, S. (2016). Digital Textbooks: Working with publisher-provided online platforms. Vanderbilt University Center for Teaching. Retrieved from https://cft.vanderbilt.edu/guides-sub-pages/digital-textbooks-working-with-publisher-provided-online-platforms/
Kelani, Z., Doral, M., & Post, Y. R. (2021). Academic Performance of Face-to-Face and Online Students in an Introductory Economics Course and Determinants of Final Course Grades. Online Journal of Distance Learning Administration, 24(2), 1-13.
Kutner, M. H., Neter, J., Nachtshein, C. J., & Wasserman, W. (2013). Applied Linear Statistical Models (5th ed.): McGraw-Hill Companies.
Lack, K. A. (2013). Current status of research on online learning in postsecondary education: Ithaka S+R.
Lakshminarayan, N., Potdar, S., & Reddy, S. G. (2013). Relationship Between Procrastination and Academic Performance Among a Group of Undergraduate Dental Students in India. Journal of Dental Education, 77(4), 524-528.
Lam, M. (2009). Effectiveness of Web-Based Courses on Technical Learning. Journal of Education for Business, 84(6), 323-331.
Mitra, S., & Le, K. (2019). The effect of cognitive and behavioral factors on student success in a bottleneck business statistics course via deeper analytics. Communications in Statistics-Simulation and Computation, 1-30.
Oh, E., & Reeves, T. C. (2014). Generational Differences and the Integration of Technology in Learning, Instruction, and Performance(pp. 819).
Qureshi, F., Khawaja, S., & Zia, T. (2020). Mature Undergraduate Students’ Satisfaction with Onlne Teaching During COVID-19. European Journal of Education Studies; Vol 7, No 12 (2020) ; 25011111, 7(12), 456-475.
Romer, D. (1993). Do Students Go to Class? Should They? The Journal of Economic Perspectives, 7(3), 167-174.
Rose, M., & Moore, A. (2019). Student Retention in Online Courses: University Role. Online Journal of Distance Learning Administration, 22(3), 1-14.
Seymour, E., & Hewitt, N. M. (1997). Talking about leaving: why undergraduates leave the sciences. United States of America: Westview Press.
Swart, W. W., & MacLeod, K. R. (2020). Flipping Online Analytics Classes: Achieving Parity with Their Face‐To‐Face Counterparts. Decision Sciences Journal of Innovative Education, 18(1), 119-137.
Tobias, S. (1990). They're not dumb, they're different : stalking the second tier. Tucson, Ariz. (6840 E. Broadway Blvd., Tucson 85710-2815): Research Corp.
Wakeling, V., Doral, M., Robertson, P. R., & Patrono, M. (2018). Perceptions of Undergraduate Students of Student-Regulated Online Courses. Online Journal of Distance Learning Administration, 21(3), 1-11.