The Learner And The Expert Mentor, Learners And A Facilitator, Peer Facilitated Learning: A Comparison Of Three Online Learning Designs

William Warrick
Graduate School of Education
College of Education and Human Development
George Mason University, USA
wwarrick@gmu.edu

Abstract: This study addressed the question: Which online learning design - constructing knowledge one-on-one with an expert mentor, constructing knowledge with peers and a facilitator, or constructing knowledge with peers alone - most positively impacts learners’ attitudes, beliefs, and knowledge outcomes? Participants were 46 graduate students enrolled in Technology and Leadership, a three-credit course. During the 10-week course, participants were randomly assigned to one of three groups. In Group 1, learners worked individually with an expert mentor. In Group 2, groups of learners worked with peers and an online facilitator. Group 3 was comprised of groups of learners and an instructor. Groups shared common course materials and assignments. Surveys were administered pre and post treatment to compare the impact of these designs on students’ knowledge and beliefs about leadership and their satisfaction with their online learning experience.

Introduction

The face of higher education is changing. New designs for instruction and new theories about learning are supplanting time-honored structures on campuses across the country. Perhaps the greatest innovations have been brought about by new technologies and their integration with redesigned instructional practices. Campus networks, the changing population of graduate students, and the Internet have the potential to revolutionize higher education. The impact of these factors is best illustrated by the number of online courses now available. According to US News and World Report, enrollment in online courses has increased by almost 20 percent in 2004 over the previous year; 11 percent of post secondary students will take at least one course online. Additionally, over 90 percent of public colleges offer at least one online course (Boser, 2004). It is estimated that by 2005, the E-learning market will top $4 billion (Boser, 2004).

There are a number of advantages to teaching and learning online. Higher education institutions are seeing the economies of scale of distance education offerings as attractive in light of budget realities. The costs of providing online courses are being reduced yearly making all institutions able to enter the online world (Carr-Chellman & Duchastel, 2000). Because the web is easy to use and capable of presenting multimedia, it has expanded the range of disciplines that can be offered online (Harasim, 2000). Courses can be offered to a larger number of students since distance education affords the opportunity to reach students who are too distant to travel to campus. In addition, including online opportunities in course offerings is an attractive option for instructors. Weekly or semiweekly meetings of graduate courses limit the amount of instructional time available. By integrating an online component, instructors are able to offload classroom discussion to an online environment. Students, then, can access discussions from home. This allows students time to reflect on information and activities from the face-to-face meetings and profit from peer-to-peer interactions. Finally, for the student, online course offerings provide greater flexibility in graduate study. For many, the requirements of job and family or physical distance from a college or university prevent travel to campuses. Online courses provide them the option to study at home according to a schedule that is most convenient to them.

There are, however, issues arising from the proliferation of online courses that must be addressed. Statistics show that the number of students failing to finish courses in which they are enrolled is much higher for students taking online courses than for those enrolled in traditional face-to-face classes. Such students are known as 'stop-outs' as opposed to 'drop outs'. Stop outs simply cease communicating with online instructors.

Given the rise in popularity of online courses and the advances in the technology of telecommunication tools, it appears evident that online learning is well on its way to becoming a permanent part of teaching and learning. Online learning is no longer peripheral or supplementary; it has become an integral part of mainstream society (Harasim, 2000). Instructors, then, are left to consider the variety of tools and designs for online learning and to decide how to maximize the potentials of those tools in their courses. Emerging educational theories about teaching and learning suggest new trends for the design of online learning. Efforts to ensure that the online course is robust and meaningful for students has led to the design and testing of various designs for online learning. This study compares the impact of three of those on students’ knowledge about being a leader and their beliefs about themselves as leaders.

Methodology

The participants for this study were 46 practicing teachers enrolled in the final semester of the Integrating Technology in Schools (ITS) program. The two cohorts met at different locations: one on campus and the other at a local high school in another county. In addition to this weekly meeting, subjects completed practicum assignments individually in their own classrooms. Finally, the two cohorts were merged for the online Technology and Leadership course. This course was designed around five modules. Each module was to be completed within a two-week period, resulting in a ten-week course. Each module included readings from the course text,, additional selected online articles, activities to synthesize course readings, a leadership case study, and individual completion of a segment of subjects Personal Leadership Plan

For the purposes of this study, the merged 46 participants were randomly assigned to one of the three groups. This resulted in 16 subjects assigned to Group One; 15 subjects assigned to Group Two; and 15 subjects assigned to Group Three. Demographic information related to the sample was collected. The data collected showed that the participants are all employed in public education settings ranging from kindergarten through high school.

Participants discussed assignments in a format determined by assignment to research group. The groups are described below.

Group One - Novice-Expert Mentor

The instructional design for this group was based upon the mentor-protégé model. The sixteen subjects in this group were individually assigned to one of four expert mentors as part of the random assignment process. Weekly assignments and PLP prompts were provided to the participants via web pages designed to support the course and to which the participants had access. The participants read assigned materials, completed activities, and wrote designated sections of their PLP. An email dialogue was initiated and sustained between novice and expert mentor and focused on, but was not limited to, discussion of the participant's reaction to the readings, assignments, and the PLP prompt. Through this interaction, the participants were encouraged by the expert mentor to refine and expand on all assignments.

Group Two - Novices-Facilitator

In Group Two, the fifteen participants were divided into three groups of five each as part of the random assignment process. This group of novices was given access to a course structured by the Blackboard course management system. Each group of five was assigned an online facilitator. Facilitators participated in and guided discussions and progress on assigned activities. The facilitator answered questions and assisted the group in their work. Like the first group, the participants in this group completed the common course assignments. However, these participants, supported by a facilitator, discussed the readings and activities among themselves using threaded discussion forums created for the course. They also individually prepared selected sections of the PLP. As they worked on their PLP, they were encouraged to discuss ideas and challenges with their peers and the facilitator. At the end of each module, each of the novices emailed selected sections of the PLP to the facilitator for comment and revisions. All products were posted in the discussion forum for comment and revisions by the facilitator as well as by peers. The facilitator in this group managed and guided the discussions and, when gaps in understanding were evident, assisted the group towards understanding.

Group Three – Novices – Instructor

The final group of fifteen participants was divided into subgroups of five each as part of the random assignment process. Each of the groups of five was assigned an instructor. Course materials were posted to Blackboard and available to participants. Group discussions and the completion of assignments were facilitated by students in the group on a rotating basis. For each of the 5 modules, a different student acted as facilitator. The student facilitator’s role was to guide discussions, to facilitate work on assignments, and to submit products to the instructor on behalf of the group. The instructor's role was strictly managerial. The instructor made sure that the groups of five had access to a working version of the course in Blackboard. The instructor received submissions from student facilitators and provided assessment feedback and comments to the group by posting comments on the discussion board. However, the instructor’s comments were summative rather than formative – that is, comments were provided only at the culmination of each group’s work not during the work-in-progress phase.

The study examined the three online instructional designs, asking: Which of the three online learning designs - constructing knowledge with peers and a facilitator, constructing knowledge one-on-one with an expert mentor, or constructing knowledge with peers alone - most positively impacted learner's attitudes, beliefs, and knowledge outcomes?

In order to answer this question, the study’s design, data collection, and analysis were framed by the following questions:

1. Is there a difference in learner attitudes concerning the efficacy of their online experience between the three groups - constructing knowledge with peers and a facilitator, constructing knowledge one-on-one with an expert mentor, and constructing knowledge with peers alone as measured - as measured by WEBLEI?
2. Is there a difference in the changes in learner perceptions of the roles and responsibilities of a leader between the three groups - constructing knowledge with peers and a facilitator, constructing knowledge one-on-one with an expert mentor, or constructing knowledge with peers alone - as measured by the LPI?
3. Is there a difference in overall course satisfaction between the three groups -constructing knowledge with peers and a facilitator, constructing knowledge one-on-one with an expert mentor, or constructing knowledge with peers alone – as measured by course evaluations?

In order to answer the first two questions of the study, two quantitative instruments were used. The Leadership Practices Inventory (LPI) was developed by Pozner and Kouzes (2001) and was designed to measure the participant’s evaluation of their own leadership practices. A comparison of the responses to the survey’s questions prior to and after treatment indicate the effect of the course activities on the participant’s beliefs about those practices. The LPI measures five practices demonstrated by leaders. Those practices include: Challenging the Process – looking for innovative ways to improve the organization; Inspiring a Shared Vision – enlisting others in their image of what the organization can become; Enabling Others to Act – fostering collaboration and team building; Modeling the Way – setting an example for others to follow; and Encouraging the Heart – keeping hope and determination alive (Pozner & Kouzes, 2001). The instrument itself consists of thirty items with a 10-point Likert scale ranging from "Almost Never" to "Almost Always".

In order to assess students’ attitudes and beliefs concerning their learning and experiences during an online course, the Web-Based Learning Environment Inventory (WebLEI) (Chang & Fisher, 2001) was given pre and post treatment. This instrument was developed and used to assess students’ perceptions of online learning. The WebLEI consists of 30 questions with five point Likert scale ranging from “Never” to “Always”. The survey is divided into four subsections: Access (students’ access, convenience of materials), Interaction (students’ participation and enjoyment), Response (level of activity and interactivity between student to student and student to lecturer) and Results (scope, layout, presentation, and links of the web-based learning materials) (Chang & Fisher, 2001).

The results of the LPI and WEBLEI were analyzed using statistical techniques to discover differences, if any, between the scores on the two instruments.

Results
Efficacy of Online Learning Experience

The first question of the study was: Is there a difference in learner attitudes between the three groups concerning the efficacy of their online experience? In order to answer the question, the Web Based Learning Environment Instrument (WEBLEI) was given before the treatment and at the conclusion of the treatment. An ANOVA comparison of the mean totals for the WEBLEI pretest showed there was no significant difference between the three groups. The means of the posttest were analyzed to determine the differences between the three groups after the treatment. Total scores for each of the four subsections were calculated using SPSS software. Finally, a one-way ANOVA was performed on the mean total scores for each subsection.

Analysis of the results of the ANOVA procedure showed significant differences between the three groups in three of the four subsections. However, a review of the WEBLEI survey questions showed a number of questions which were not applicable to the Mentored treatment group. Participants in the mentor group answered “Not Applicable” for these questions, and the value of their answers was 0. This caused the total scores for the mentor group to be lower. The questions that were not applicable for the mentored group were:

9.	I communicate with other students in this subject electronically (email, bulletin boards, chat line). (INTERACTION)
12.	I have the autonomy to ask other students what I do not understand. (INTERACTION)
13.	Other students respond promptly to my queries. (INTERACTION)
15.	I was supported by positive attitude from my peers. (INTERACTION)
16.	This mode of learning enables me to interact with other students and the tutor asynchronously. (RESPONSE)
20.	It is easy to organize a group for a project. (RESPONSE)
21.	It is easy to work collaboratively with other students involved in a group project. (RESPONSE)

Each of these questions relates to students communicating with other students. By design, the participants in the mentored group communicated only with their online mentor and did not collaborate with peers. In light of this, the responses to these questions were omitted in calculating the means for the four sections of the WEBLEI. The ANOVA procedure was repeated on the new totals. The results of this procedure are shown in table 1

Table 1. Analysis of Variance – WEBLEI Post test

		Sum of Squares	df	Mean Square	F	Sig.
ACCESS	Between Groups	36.46	2	18.23	.73	.487
	Within Groups	1021.43	41	24.91
	Total	1057.89	43
INTERACTION	Between Groups	2.74	2	1.37	.38	.685
	Within Groups	154.07	43	3.58
	Total	156.80	45
RESPONSE	Between Groups	53.01	2	26.51	2.69	.080
	Within Groups	424.47	43	9.87
	Total	477.48	45
RESULTS	Between Groups	216.31	2	108.16	6.49	.003*
	Within Groups	699.47	42	16.65
	Total	915.78	44

* significant at the 0.05 level

Significant differences existed between the three groups on the Results subsection of the WEBLEI as indicated by the asterisk. Among the participants in the three treatment groups, there is a significant difference in the mean scores on the Results subsection of the WEBLEI, F(2.42) = 6.49, p = .003. This subsection is designed to measure students’ satisfaction with the scope, layout, presentation, and links of the web-based learning materials. A Bonferroni post hoc test revealed that the mentored group scored significantly higher (M = 32.87) than did the facilitated group (M = 28.00) or the instructor led group (M = 28.47).

Changes in Perception of the Roles and Responsibilities of a Leader

The second question of the study was: Is there a difference in the changes in learner perceptions of the roles and responsibilities of a leader between the three as measured by the LPI? In order to answer this question, participants were asked to complete the Leadership Practices Inventory (LPI). This survey asked participants to respond to statements that caused them to reflect on their own leadership practices. Participants were given the survey before the treatment and again after the treatment. The surveys were scored, and total scores for the five subsections were determined using instructions provided in the LPI manual. To determine if significant change in perceptions of leadership of each of the three groups occurred, a paired samples T-Test was performed on the means. The results of this procedure for each group are shown in table 2.

Table 2. Paired Samples t test – Leadership Practices Inventory

Overall Course Satisfaction

The third question of the study was: Is there a difference in overall course satisfaction between the three groups as measured by course evaluations? In order to answer this question, all participants were given the university’s Course Evaluation survey. The responses to each question were entered into SPSS software, and mean responses to each question were calculated. An ANOVA procedure was performed on the means to determine the difference between the groups. The results of the ANOVA showed that there were no significant differences between the three groups on any of the course evaluation questions.

Discussion

This research study was undertaken to examine the effects of three different online learning designs - constructing knowledge with peers and a facilitator, constructing knowledge one-on-one with an expert mentor, and constructing knowledge with peers alone. Surveys were given to each participant in an effort to assess the students’ attitudes towards the online learning structure as measured by the Web Based Learning Environment Instrument; their learning, as measured by the Leadership Practices Inventory; and overall course satisfaction as measured by the Course Evaluation survey.

The results of the WebLEI comparisons indicate that the differences between the groups’ attitudes concerning the efficacy of their online experiences were generally similar. Participants in each of the three groups viewed Access, Interaction, and Response similarly. There was a difference shown in the Results subsection. The scores for the mentored group indicated that they responded to the questions in this part of the survey more positively than the other two groups. The Results subsection contained questions designed to elicit responses about the perceptions of how the web based learning materials were structured and organized, and whether the materials presented follow accepted instructional design standards, such as stating its purpose, describing its scope, incorporating interactivity, and providing a variety of formats to meet different learning styles. Included in this section are relevance and scope of content, validity of content, accuracy and balance of content, navigation, and aesthetic and affective aspects (Chang & Fisher, 2001). It appears likely that one of the strengths of the Mentor design is the way in which the course materials were organized and the interactivity provided by the mentor. In light of the fact that the presentation and organization of course materials was different for the mentored group, this would indicate a positive reaction to the web-based course material as opposed to that of the Blackboard Course Management System and working together collaboratively in Blackboard.

The differences in pre and post treatment scores on the Leadership Practices Inventory showed significant changes for the Mentored group. The LPI was used to measure the participants’ perceptions of the roles and responsibilities of a leader. In each of the five subsections, the mean scores for the Mentored group improved significantly. Given the fact that the Mentored group alone showed significant change, this would indicate that the online learning design utilizing one-on-one communication with a mentor provided the participants with the opportunity to reflect and discuss the roles and responsibilities of a leader to a greater degree than did the other participants working in small groups.

The analysis of the scores on the Course Evaluation survey given at the conclusion of the treatment indicated that there were no significant differences in the ratings between the participants in the three groups. One reason for this could be that the form is inadequate to be used as an assessment tool for this purpose. While it is possible that all participants rated the course in the same way, it is more likely that the evaluation tool is flawed given the differences in scores reflected in the LPI and WebLEI.

This study leaves certain questions unanswered. One recommendation for further study would be the inclusion of qualitative interviews to elicit students’ attitudes and perceptions of their online learning experience. These interviews would provide a richer understanding of the varied aspects of the three design. It is possible that there are affordances in each which can be combined to provide the optimum design for online learning.

References

Carr-Chellman, A., & Duchastel, P. (2000). The ideal online course. British Journal of Educational Technology, 31, 229-241.

Chang, V., & Fisher, D. (2001). The validation and application of a new learning environment instrument to evaluate online learning in higher education. Paper presented at the Australian Association for Research in Education Conference, Fremantle.

Harasim, L. (2000). Shift happens: Online education as a new paradigm in learning. Internet and Higher Education, 3, 41-61.

Kouzes, J. M., & Posner, B. Z. (2001). Leadership Practices Inventory - Individual Contributor. San Francisco: Jossey-Bass/Pfeiffer.