ICYMI | How Online Learning Compares to the Traditional Classroom

Measuring Learner Characteristics

A total of 238 students responded to a survey in either an online or in-class section of several upper-level accounting courses, including junior-level cost accounting, Intermediate Accounting I, Intermediate Accounting II, and senior-level auditing. The survey asked students a number of questions aimed at measuring personal characteristics, such as study approaches, satisfaction with the course, learning effectiveness, and some demographic questions. Cognitive effort was measured based on the summed responses to 12 items, each of which described a study approach that ranged from memorization to approaches involving more synthesis. Self-regulation was measured similarly, but with 10 items that dealt mostly with how hard a student works in his studies. Both of these measures were adapted from Paul R. Pintrich and Elisabeth V. De Groot’s “Motivational and Self-Regulated Learning Components of Classroom Academic Performance” (Journal of Educational Psychology, January 1990, http://bit.ly/2v3pRMw). All of these items used a 7-point scale (7=very true of me; 1=not true of me); therefore, the total scores can range from 12 to 84 for cognitive effort and from 10 to 70 for self-regulation. Perceived learning effectiveness and satisfaction each consisted of four items measured on a seven-point scale (7=strongly agree; 1=strongly disagree), with totals ranging from 4 to 28. Both perceived learning effectiveness and satisfaction were both adapted from Jillian C. Sweeney and Deborah Ingram’s “A Comparison of Traditional and Web-based Tutorials in Marketing Education” (Journal of Marketing Education, April 2001, http://bit.ly/2vu3lPh). Course performance is the student’s expected grade (A–F) as of the last week of the course, with A coded as “4,” B coded as “3,” and so on. The associations of self-regulation, cognitive effort, and learning mode (online versus in-class) were modeled with perceived learning effectiveness, student course satisfaction, and expected grade.

Results

As shown in Exhibit 1, there is generally an advantage to higher self-regulation and cognitive effort in terms of students’ perceptions of the learning experience in both online and traditional classes. It is not surprising that the best perceptions occur when students are both high in self-regulation and use more cognitive effort (means of 24.2 and 24.5 for traditional and online, respectively). Statistical tests showed a meaningful effect for both self-regulation and cognitive effort, but no effect for delivery method. The most visible difference for cognitive effort occurs when students taking an online class have higher self-regulation. Students in this category, perhaps partially due to the fact that they are in an online class, found that they must exert more effort to experience more effective learning, which is consistent with prior research (Robert Irizarry, “Self-Efficacy and Motivation Effects on Online Psychology Student Retention,” USDLA Journal, 2002).

Another important course outcome is the student’s level of satisfaction. As shown in Exhibit 2, the results for students’ satisfaction form a pattern similar to that of perceived learning above, except that cognitive effort appears to have a bit more of an effect than self-regulation. Statistical tests bear this out, with self-regulation failing to make a meaningful difference and cognitive effort stealing the show. In fact, the most visible advantage for cognitive effort occurs for online students who are relatively low in self-regulation. Those exerting greater effort were considerably more satisfied with the course than their counterparts (means of 22.8 vs. 18.5). As a practical matter, satisfaction level is likely to be a strong ultimate driver of a student’s more formal evaluation of the course.

As shown in Exhibit 3, the pattern is different when looking at course performance. School regulations prevent associating actual grades with individual students, so the performance measure is the expected grade at a late point in the course, which has been shown as a capable substitute for actual course performance in previous studies (R. Eric Landrum and Ronna J. Dillinger, “Student Perceptions of Grading Practices: Does ‘Average’ Class Performance Equal a ‘C’ Grade?” Journal of Research and Development in Education, Fall 2000, http://bit.ly/2u58MR2; Stephen D. Phipps, Robert S. Kidd, and David Al. Latif, “Relationships among Student Evaluations, Instructor Effectiveness, and Academic Performance,” Pharmacy Education, December 2006, http://bit.ly/2v1ajL9). For online students with low self-regulation, greater cognitive effort is associated with better course performance (mean of 3.6, vs. 3.2). There is, however, an opposite pattern for traditional-delivery students with low self-regulation and online students with high self-regulation. Statistical tests show that delivery mode, self-regulation, and cognitive effort interact; that is, the association of each with the expected grade depends on the other two.

An examination of the survey items making up cognitive effort may partially explain the counterintuitive results in Exhibit 3. Several of the survey items are related strictly to remembering material, without any emphasis upon deeper learning, while others relate to stepping beyond memorization to higher levels of understanding and synthesis. Students could score high in cognitive effort, while not necessarily going deeper in their thought processes. To the extent that a course lends itself to performing well with mere memorization, such an approach can work; however, upper-level accounting courses do not often fall into this category.

Implications

The results indicate that a student’s ability to regulate her commitment and effort can be associated with a more positive course experience, which may ultimately translate into a better evaluation of the course. This seems true whether the course is delivered in a traditional or online format. As such, differences among students seemed to have more of an effect than course delivery method; online students do not necessarily appear to be at a performance disadvantage. As noted earlier, some research suggests that online students are more motivated because they know that they cannot rely on class attendance.

This explanation breaks down somewhat when examining course performance, at least with regard to cognitive effort, which did not necessarily translate into better performance. Certainly, it is not uncommon for a student to be disappointed upon receiving his score from the first exam. In his mind, the long hours spent studying the instructor-provided lecture notes and homework solutions (which the student may not have worked out on his own) should have translated into better performance. A twist in the way the information is presented on the exam, however, may reveal a student’s insufficient depth of understanding.

Bloom and his coauthors would suggest that it is not necessarily the amount of cognitive effort that matters, but rather the type of effort. In courses that require synthesis and application of concepts to perform well on exams, students can spend many hours thinking they are preparing themselves adequately, only to perform poorly. Many would do better if they worked the “recommended” problems on their own and used class time and solutions for feedback as to their progress; this approach may force them to move to deeper cognitive levels as they struggle to complete the problems. At many schools, however, students in this category may well constitute only the top students in most courses.

Both the student and the instructor may be better served when homework and other assignments require students to use deeper thinking skills to build upon and apply their basic knowledge. This might include case discussions or even short in-class questions that are equal in depth to exam questions and require students to proceed logically. One example is that most cost/managerial accounting textbooks will include problems that provide variance amounts and certain other numbers, and then require students to “back into” inputs that might have been given in previous problems. Without a good working understanding of variances, such problems will be challenging because of the analytical thinking required.

While such calculations as incremental analysis, operating leverage, and breakeven points are often quite simple as textbook exercises, real-world problems are far less straightforward or uniform. They may call upon the highest levels of thinking and require careful analysis, often by multiple people. Classroom materials that require students to incorporate the concepts of revenue/cost relevance, such as analyzing what is relevant at a global versus business unit–level, can force students to synthesize and reason at a higher level when thinking about such seemingly simple concepts as breakeven points. Students can then understand that the most difficult part is not the final calculation, but deciding which factors and details go into that calculation.

The task of the accounting and business educator is generally to develop competencies in students that will prepare them to succeed in their careers. More and more, employers are requiring higher levels of cognition and critical thinking in new employees. Employers don’t want memorizers; they want those with analytical and problem-solving skills. Thus, student characteristics such as level of self-regulation and cognition are possibly more important than the method in which courses are delivered.

The challenge for accounting firms and other employers is to identify and develop new professionals who have the potential to use higher levels of reasoning to solve problems and contribute to the success of the organization. Identifying students who can analyze, synthesize, and solve problems is a difficult task. It cannot be accomplished by simply reviewing transcripts to determine whether the courses completed were or were not taken online, especially given the importance of individual characteristics. It is likely that course grades in upper level accounting courses, where higher-level reasoning is often more integrated into the subject matter, are greater differentiators than introductory courses.

Beyond transcripts, the interviewing process is crucial to assess whether the candidate has the ability to think critically and at higher levels to solve problems. Asking candidates probing questions about complex situations that they have faced, challenges they have overcome, and even their responses to failures that they have encountered are as important as the candidates’ grades. Similarly, the quality of the questions that the candidate asks is equally important. Once the hiring decision is made, it is incumbent upon the employer to expose the new hire to situations that require higher levels of thinking, such as analysis, synthesis, and exercise of judgment while providing the support and feedback to learn from these challenging situations. This commitment to the development of critical and high levels of thinking will pay dividends in the long run.