Graduate Instruction Combining
Online, On-Site, and Face-to-Face: A Study

Priscilla Norton, EdD
Graduate School of Education
George Mason University
Fairfax, VA USA 22030
pnorton@gmu.edu

William Warrick, Instructor and Doctoral Candidate
Graduate School of Education
George Mason University
Fairfax, VA USA 22030
wwarrick@gmu.edu

Abstract: The efficacy of combining face-to-face and online instruction remains unclear in the literature. Nevertheless, for the mature learner, some combination of delivery models might offer a desirable learning option. Thus, in the Fall Semester, 2001, twenty students enrolled in a graduate level course focusing on learning and technology. Students were dispersed over a broad geographical area. A course was designed for these students that combined three patterns of instruction or learning structures: 1.) online discussion boards, 2.) face-to-face classes, and 3.) on-site small group, student run meetings. Students' ratings concerning their learning experience and the quality of their interactions with the instructor and peers in each of the three learning structures are presented and compared. In addition, students' rankings of the three learning structures related to learning and quality of interactions were collected at the end of the course and are presented.

Introduction

Much enthusiasm exists in the higher education community about the potentials of online delivery systems for coursework. Professional development programs and courses are being offered in a variety of ways, utilizing technology to differing degrees. Some simply use technology to improve the presentations, while others use technology to offer courses entirely online. The U.S. Department of Education's National Center for Educational Statistics (NCES), reported that the number of distance education programs increased by 72 percent from 1994-95 to 1997-98 (Quality On the Line: Benchmarks for Success in Internet-Based Distance Education, 2000). "However since the quality of these new programs can vary widely, education experts are telling teachers to shop around and ask questions first. Not all online programs offer teachers the right mix of collaboration and feedback (Weiner, 2001).

The efficacy of online versus face-to-face instruction or combining face-to-face and online instruction remains unclear in the literature. For instance, Hall, Watkins, & Ercal (2000) state that web-based instruction can be as effective as face-to-face instruction and that factors that lead to improved performance in face-to-face classes can also increase performance and positive attitude in web-based courses. Conversely, Thornam & Phillips (2001) suggest that those taking an online section perceive less interactivity than those taking a face-to-face section. Additionally, Johnson, et. al. (1999) found slightly more positive perceptions about the instructor and overall course quality in a face-to-face version of a course, although there was no difference between the two course (face-to-face and online) formats in learning outcomes. Cereijo, Young, and Wilheim (2001) examined students' comments while taking a CD/Web-based course. Their research revealed that students reported the advantages of CD/Web-based learning to be its convenience, flexibility, and opportunity for enhanced learning. Conversely, students reported concerns about isolation, the learning environment, and technological problems to be significant disadvantages.

Despite concerns about web-based learning, for the mature learner hampered by geographical and time considerations, online learning remains a demanded and potentially viable learning option. It is likely that for many learners today some combination of delivery models offers a desirable and "doable" learning option. Thus, in the Fall Semester, 2001, twenty students requested an online course. These students were geographically distributed throughout northwestern Virginia. Because all students were practicing classroom teachers with full day responsibilities, it was not feasible for them to commute to campus for evening classes. Additionally, because of distance considerations, it was not feasible for faculty to commute to weekly classes in students' varied communities. We asked ourselves if it might be possible to use some combination of online and face-to-face patterns of delivery while continuing to maximize student interactions and learning. A course was designed that combined three learning structures or patterns of instruction: 1.) online discussion, 2.) face-to-face classes held off campus, and 3.) on-site, student-run, small group meetings. Given existing concerns about online learning and the experimental nature of this combination, we asked four questions: (1) How will students rate their learning experiences and the quality of their interactions in each of the three learning structures? (2) Will there be a difference between students' ratings concerning their learning experiences and the quality of their interactions depending on type of learning structure? (3) Which learning structure will students find most useful and least useful for their learning? and (4) Which learning structure will students report as affording the highest and lowest quality of interactions with their instructor and their peers?

Methodology

Twenty practicing educators enrolled in the course. All of these educators were candidates in a Master's of Education program emphasizing the integration of technology with classroom practice. All had previously complete 18 graduate credit hours. Four of the educators were elementary technology resource teachers. One was a middle school administrator; another was a district level technology coordinator. The remaining fourteen students were classroom teachers, spanning the K-12 curriculum.

The fifteen-week course focused student attention on learning and technology, placing attention on constructivist learning theory, the role of symbolic competence in learning, and the connection between theories of learning and the selection and use of a range of technologies to support teaching and learning. The course was conceptually divided into three four-week segments with a remaining three-week period for production of collaborative student projects. In each of the three four-week segments, students participated in each of the three learning structures selected to facilitate the course activities. Thus, students attended one face-to-face, instructor led class meeting during each four-week cycle. The face-to-face classes met in a middle school media center with internet-linked computers. During these sessions students shared the results of their on-site activities, discussed readings, and participated in instructor led activities. In addition, students posted weekly to a Blackboard discussion board. Discussions were prompted by the instructor as well as monitored and shaped by the instructor. Finally, during each four-week period, students met in small groups of three to five (organized around geographical proximity). These on-site groups engaged in activities assigned and described by the instructor using Blackboard's class delivery system. Care was taken to insure that these activities were well designed, comprehensively described, and concluded with the production of a concrete product. In addition, guidance was provided concerning processes the groups might use to successfully complete the assignment. On-site small group activities engaged students in completely a product designed to encourage students to discuss, interpret, and apply insights and understandings from the readings. Products that resulted from the on-site group activities were either shared electronically or brought to face-to-face sessions to be shared. Email was used throughout the course to clarify, remind, and facilitate the work of the course.

At the culmination of each four-week period, students were emailed the link to an electronic survey. The researchers used the Zoomerang (http://www.zoomerang.com) survey service. Students were asked to respond to six questions. Students were asked to rate the usefulness (3 - Useful (U), 2 - Somewhat Useful (S), and 1 - Not Useful (N)) of each learning structure as a learning experience. Students were then asked to rate the quality of their interactions during the four-week period with the instructor, with their on-site peers, and with fellow discussion board participants. Students rated the quality of their interactions as positive (P), neutral (N), or unhelpful (U). Survey results were tabulated anonymously and summarized by Zoomerang. At the conclusion of the course, a final Zoomerang survey was emailed to students asking them to rank the three learning structures and their combination for their importance in students' overall learning experience. In addition, students were asked to rank the three learning structures and their combination for the quality of their overall course interactions with the instructor and peers. Students were asked to assign 1 to the most important or highest quality through 4 as the least important or lowest quality. Using the Zoomerang survey software facilitated the collection of data. It also insured students' anonymity. However, because of the anonymity of responses, it was not possible to pair student responses across surveys, and thus, statistical analysis of data became problematic. As a result, data analysis was limited to descriptive and summative analysis strategies.

Results

The first question posed was: How will students rate their learning experiences and the quality of their interactions in each of the three learning structures? In order to answer this question, student responses to each of the three surveys were tallied and percents computed. Results of the analysis are presented in Table 1. Examination of the data suggests that more than half of the students viewed all three learning activities as useful. The data also suggest a slight decline for all three learning structures over time. The same basic patterns occur related to quality of interactions. That is, 60% or more of students rated the quality of their interactions to be positive regardless of learning structure as well as revealing a slight decline in ratings over time. Finally, in all instances, the online discussion board structure is rated slightly lower than the face-to-face whole group class and the on-site small group activity.

__________________________________________________________________________________________
Table 1. Summary of Ratings of Learning Experience Using Three Different Learning Structures

 

The second question posed was: Will there be a difference between students' ratings concerning their learning experiences and the quality of their interactions depending on type of learning structure? In order to answer this question, means for student ratings were computed for each survey period and for the research period as a whole. Results of the analysis are presented in Table 2. Examination of the means for student ratings of their learning experience related to the three learning experiences further support the conclusion that students found the face-to-face whole group learning structure to be the most useful followed by the on-site small group structure with the online discussion board being the least useful. Students' rating of the quality of their interactions with the instructor cannot be compared with the on-sire and online structures since the instructor played a role during all three learning structures. It is possible, however, to note that students rated the quality of their interactions during on-site activities more positive than the quality of their interactions during online discussions.

__________________________________________________________________________________________
Table 2. Overall Mean Ratings for Learning Experience and Quality of Interactions

 

The third question of the study asked: Which learning structure will students find most useful and least useful for their learning? In order to answer this question, students were asked to rank the learning structures, using 1 for the most useful structure through 4 for the least useful structures. Percentages for rankings were computed and reported in Table 3. Examination of the data suggests that the majority of students (nearly half) ranked on-site group work as the most useful learning structure with face-to-face whole group receiving the next highest or useful ranking. Discussion board activities received the highest percent of "least useful" rankings, suggesting that this learning structure was not highly regarded as a valuable learning experience.

___________________________________________________________________________
Table 3. Overall Ranking of Learning Structures as a Learning Experience (Most Useful to Least Useful)

 

The fourth question of the study asked: Which learning structure will students report as affording the highest and lowest quality of interactions with their instructor and their peers? In order to answer this question, students were asked to rank the learning structures for the quality of interactions with peers and the instructor ­ 1 for highest quality through 4 for lowest quality. Results for the two questions are presented in Table 4. Examination of the data shows that the highest ranked learning structure related to quality of interactions with the instructor was the face-to-face whole group structure while the on-site group structure was ranked lowest. This is not surprising since the instructor's role in the on-site group structure was minimal while the instructor had a substantial role in the face-to-face whole group structure. Conversely, nearly half of the students ranked the quality of their interactions with peers highest in the on-site group structure. The face-to-face whole group learning structure was ranked second highest for quality of interactions with peers.

___________________________________________________________________________
Table 4. Overall Ranking of Learning Structures for Quality of Interactions with Instructor and Peers

 

Discussion

While all learning structures were rated as useful learning experiences and interactions were generally rated positively, the discussion board activities were consistently rated lowest. It seems clear that students found the face-to-face and on-site learning activities more important for their learning. Given that much of online learning centers around individual readings and research and group postings on discussion boards, these results are important. It is possible that the more common learning structure of postings to a discussion may not be the most effective manner in which to organize and conduct online learning. Perhaps Sonwalkar (2001) captures the limitations of discussion boards as extensions of the classroom discussion method of teaching when he writes: "In attempting to harness the capabilities of digital interfaces, the mistake is often made of recreating a classroom-teaching model within an online learning environment. Online technology designed to mimic the classroom becomes a restriction and a barrier to the teacher's ability to impart knowledge. (p. 2)."

Not only do the results suggest stepping away from conventional classroom-teaching models in online learning, the results of this study suggest a manner in which online learning might progress- small groups of students studying with the guidance of a virtual instructor. In all instances, students in this study rated their learning experiences and the quality of their interactions with peers to be highest when using the on-site learning structure. It might have been possible that on-site activities resulted merely in social gatherings or trivial commentary. Instead, students reported that these activities were the most useful for their learning. This not only reinforces the social constructivist perspective of learning ­ that understanding is achieved through dynamic, negotiated group construction of knowledge ­ but it also suggests that the most important role of the instructor in an online learning environment may be as designer. When the instruction in this course was carefully designed around a problem and a product ­ a performance of understanding, students were able to meet, interact with ideas, concepts specified in readings, and each other to build bridges between knowledge and understanding. Once students had solved a problem and constructed a product, they were able to share, elaborate, critique, and expand upon their understanding through both online and classroom presentation. It seems it is possible to have a virtual instructor ­ one who designs robust learning experiences and coaches and provokes from a distance. On the other hand, it seems that having virtual classmates ­ posting comments in response to instructor prompts or peer comments ­ does not facilitate learning in the same way as face-to-face peer encounters and collaboration.

---

References

Cereijo, M., Young, J., & Wilheim, R. (2001). Factors facilitating student participation in asynchronous web-based courses. Journal of Computing in Teacher Education. 18(1), p. 32-39.

Hall, R., Watkins, S., & Ercal, F. The horse and the cart in web-based instruction: Prevalence and efficacy. Paper presented at the Annual Meeting of the American Educational Research Association (AERA) (New Orleans, LA, April 24-28, 2000). ERIC No. ED443425.

Johnson, S., Aragon, S, Shaik, N., & Palma-Rivas, N. (1999). Comparative analysis of online vs. face-to-face instruction. Paper presented at the WebNet 99 World Conference on the WWW and Internet Proceedings (Honolulu, HA, October 24-30, 1999). ERIC No. ED448722.

Quality On the Line: Benchmarks for Success in Internet-Based Distance Education (2000). Washington, D.C.: The Institute for Higher Education Policy, U.S. Department of Education's National Center for Educational Statistics (NCES).

Sonwalkar, N. (2001). Changing the interface of education with revolutionary learning technologies. Syllabus Magazine. http://www.syllabus.com/syllabusmagazine/article.asp?id+5663. [Accessed December 10, 2001.

Thornam, C. & Phillips, S. (2001). Interactivity in online and face-to-face sections of a graduate nursing course. TechTrends. (45)1, p. 34.

Weiner, R. (2001). Online courses to improve teacher technology skills. The New York Times. http://www.nytimes.com/2001/04/25/technology/25EDUCATION.html. [Accessed April 25, 2001].