|
|
Table of Contents
- Abstract
- Introduction
- Web Courses in Higher Education: Pedagogy or Profit?
- How Much Multimedia is Enough?
- The Case for Web Telecourses
- Educational Effectiveness and Student Atttitudes
- Hypotheses
- Method
- Results
- Discussion
- Footnotes
- References
- Acknowledgments
- About the Author
Abstract
Prior research on instructional media effects suggested that an audiographic approach to World Wide Web based courses would optimize educational effectiveness along with cost effectiveness, although with a possible loss of teacher immediacy that could adversely affect student attitudes. An introductory telecommunication course was converted to an audiographic Web telecourse in which students listened to pre-recorded audio classroom interactions while viewing a detailed course outline and illustrative sites over the World Wide Web. Forty-nine subjects were recruited from a live lecture class and randomly assigned to either the experimental (Web course) group or a control group that took the class in a traditional lecture section. Analysis of covariance (ANCOVA) showed that the experimental group had test scores and student attitude and teacher immediacy ratings equal to those of the control group after controlling for student gender, class level, grade point average and attendance. Open-ended interviews were also conducted to assess qualitative dimensions of student satisfaction. The results supported the audiographic telecourse model as a potentially cost-effective approach to distributing courses over the Web. New directions in research on instructional media effects and teacher immediacy were formulated from an analysis of the unique characteristics of the World Wide Web as an instructional medium.
In 1998 over 2500 college courses from some 100 universities were available over the Internet (Cape Software, 1998; University of Texas, 1998), marking the World Wide Web as the latest wave of instructional technology to invade the college campus. But with each new wave of educational technology the question of the educational effectiveness of mediated learning arises anew. So it was with film (Hovland et al, 1965), radio (Schramm, 1977), television ( Whittington, 1987, Wetzel, Radtke & Stern, 1994), computer assisted instruction (Clark, 1985), computer mediated conferencing (Hiltz, 1986; Althaus, 1997), interactive video (Storck & Sproull, 1995) and multimedia (Yaverbaum and Nadarajan, 1996) and, for that matter, the printing press and the chalkboard. So it is now with the Internet.
Web Courses in Higher Education: Pedagogy or Profit?
Educators seem naturally drawn to the seemingly limitless pedagogical possibilities of interactive computer multimedia instruction. The potential benefits include personalized instruction, active learning, instant feedback, real world simulations, faster and more effective teaching and empowerment of students (Hiltz, 1986, Ragsdale & Kassam, 1994; McComb, 1994; Woolsey, 1994, Webster & Hackley, 1997). The World Wide Web application of the Internet is poised to fulfill these visions as it expands to encompass interactive audio and video as well as the text and graphics that are its hallmarks today. Hopefully, Web-based courses will equal conventional instruction -- if not surpass it -- in educational effectiveness, while bringing higher education directly to desktops in schools, homes and offices everywhere (Shein, 1997).However, much of the interest in Web courses in higher education may spring not so much from improving the quality and accessibility of education as from making it more profitable. Web courses may improve institutional financial performance if they attract new, untapped pools of students by being "open" all hours of the day and night, making it possible to respond to the time restrictions experienced by the fast-growing segment of "nontraditional" (i.e., over age 25) students (Gubernick & Ebeling, 1997).
The concept of higher education as a competitive business is also gaining currency and with it the notion that Web courses could be a competitive advantage. As Peter McPherson, President of Michigan State University said in reference to Web courses, "Market pressure is going to force educators to think about things unconventionally" (Gubernick & Ebeling, p. 85). Indeed, one experiment with a multimedia classroom was received so favorably by students that it was suggested as a recruiting technique (Pipes & Wilson, 1996) and at least one university has already gambled its future with on-line courses to bolster flagging enrollments (DeLoughry, 1996). Meanwhile, universities now vie to make the top ten on the "most wired" list (ZDNet, 1998) just as they once competed to have the largest libraries.
One way to increase both profitability and competitiveness might be to slash costs by substituting Web courses for classroom instructors. As one student in the present study remarked, "We know why the university is pushing this. It’s not to be cutting-edge, it’s to save money. We’re paying for an instructor but if they can get us to use a computer or a TV instead, it saves them money." Voss (1995) presented the case for improving instructional productivity in higher education through multimedia. She found conventional methods of instruction to be top-heavy with personnel costs and ripe for innovation with multimedia technology. As college costs soar, students shop for more economical alternatives, introducing competitive pressure in higher education and forcing unprofitable colleges to close their doors (Gubernick & Ebeling, 1997).
How Much Multimedia Is Enough?
Thus, the evolution of Web courses in higher education may well come down to finding the cheapest possible mode of instructional delivery that maintains educational effectiveness. In other words, just how much of the multimedia "coolness" of the World Wide Web is really necessary to produce a college course that students can learn from?If history is any guide, current efforts to improve education through computer technology may well founder on economic realities. A previous boom in computer-assisted instruction fizzled when it was realized that programmed instruction, although effective, was less costly when published in the form of a printed text, sparing the expense of the computers and the computer networks. And the students were more likely to complete the classroom courses (Clark, 1983). Time spent on task emerged as a competing explanation for any relative advantage of computer-based instruction (Clark, 1985) and when computer-based instruction was superior to the classroom, it may have been more due to differences in instructional methods than to the presentation medium (Johnston, 1987).
Moreover, the sources cited in the first paragraph of this article suggest that the medium of instruction generally has no impact on educational effectiveness. All forms of mediated instruction are about equally effective to live instruction, and to each other. One scholar dismissed the entire question of media channel effects in interactive video media as an irrelevant issue unworthy of further study (Reeves, 1986). Another went so far as to state, flatly, that "Media will never influence learning" (Clark, 1994). He argued that content of the instruction, not the medium of presentation, is what influences student achievement. The seemingly unique instructional effects of certain media (e.g., the ability to emphasize a detail by zooming in with a camera lens) can always be replaced by other means in other media (e.g., by using italics).
Not all agree that instructional technology media effects are a closed issue. Kozma (1994) argued that media effects might still be discovered through consideration of their interactions with cognitive processes. Indeed new research shows that computer multimedia formats combining text with either audio or video were more effective than still graphics and text alone (Quealy & Langan-Fox, 1998), which the researchers attributed to dual coding of text with audio or video in short term memory. Schutte (1997) found that his virtual classroom-style Web course was more effective than comparable in-class instruction. However, interactions among the students may have accounted for the differential effects on performance. Also, his model still required the attentions of a live instructor and had to be consumed by students at a set time according to a pre-arranged schedule of classes, limiting both its ability to reduce staff costs and to attract new "nontraditional" students.
So, if the question becomes, "how much multimedia technology is enough?" the answer would seem to be "not very much." If the "talking head" on TV (or on the Internet) is equal to the walking-head in the classroom, is equal to the walking-talking-morphing artificially intelligent virtual reality puppet inside the computer, then the talking head will probably prove to be the most cost effective. But there literally does need to be some talking going on, or at least some form of multimedia presentation that will invoke the dual coding process. Text and graphics alone would seem to yield inferior results (Quealy & Langan-Fox, 1998).
This means that the most cost effective approach to Web courses could well be an old stand-by, the telecourse. That is, merely reproducing what goes on in the classroom and putting it on the Web should be as educationally effective as any other approach, but far less costly and far more accessible (Harris and DiPaolo, 1996). This will be especially true if recordings are made and replayed multiple times "on demand" for nontraditional learners at odd hours of the day. And the Web should further improve on the cost effectiveness of the conventional telecourse model by using production technologies that are increasingly found on the desktop of most college instructors and by using a ubiquitous, multipurpose, low-bandwidth network (i.e., the Internet) instead of closed, single-purpose broadband networks.In this spirit, the present effort set out to "push the bottom of the envelope" in online learning. Our approach was to create an audiographic telecourse which used audio captured in a live classroom to augment text-based lecture outlines and graphics published on the Web. Since in-class materials were merely transposed to the Internet, the incremental production costs were minimal, arguing strongly for cost effectiveness. Research on instructional radio (Schramm, 1977) and instructional telephony (Parker, 1976) offered preliminary assurance that an audio-based course could be effective.
On its face, the Web telecourse would appear to be superior to other common varieties of Web courses:
The virtual classroom (first described by Hiltz,1986) uses email, newsgroups, listservs, bulletin boards and chat rooms in place of live classroom interactions (e.g., Schutte, 1997; Althaus, 1997). But virtual classrooms generate huge volumes of transactions that the instructor must monitor and in some implementations require the simultaneous presence (albeit in mediated form) of learners, instructors and technicians so that even more personnel may have to be assigned than to a conventional classroom.
The constructionist models favor learner-centered approaches in which instructors act as facilitators to self-directed learners. Web courseses embodying this approach seek a synthesis between computer games, Multi User Dungeons (MUDs) and instruction in which learners adopt roles and interact with each other and their instructor in artificially constructed settings other than the classroom (Moshell & Hughes, 1996). Here, multimedia production costs as well as time demands on instructors/facilitators would seem to negate cost effectiveness.
Computer managed instruction orchestrates conventional instructional media including graphics and audiovisuals -- but mostly text (Yaverbaum & Nadarajan, 1996; Benyon, Stone & Woodroffe, 1997). The lack of a consistent multimedia approach and an over-reliance on text would seem to make it less effective than the Web telecourse model.
But if the telecourse model is so promising, why not a video Web telecourse? At the time this research was conducted, Internet video was just becoming a reality but was beyond the capabilities of most personal computers common at the time and so was not an option. It is feasible now and within the reach of more users and indeed examples of video web courses may be found (e.g., University of Minnesota, 1998). However, the extra costs of recording, editing, distributing and receiving video over the Web -- and modifying it later --would seem to make it inherently less cost effective compared to the audiographic approach proposed here.
Educational Effectiveness and Student Attitudes
The prospect of world-wide competition for enrollments in Web classrooms introduces a new element into the educational effectiveness equation. In this emerging competitive environment, students will no longer be a captive audience. Institutions of higher education will have to struggle to satisfy students so that they can garner repeat business and generate new customers through the "buzz" generated by past customers.This new fact of life suggests that a third criterion should be added to cost and educational effectiveness when evaluating the merits of Web courses; namely, their ability to satisfy their learners/customers and keep them coming back to the same educational provider for more. There is limited evidence that learner attitudes are correlated to willingness to continue programs of instruction that are computer-managed (Richards & Ridley, 1987). Pipes and Wilson (1996) found that the availability of highly appealing multimedia courses might favorably affect the ability of colleges to attract new students, while conventional telecourses alienated them.
However, Web courses may lack an important quality that fosters positive student attitudes toward instruction: teacher immediacy, or behaviors that enhance physical and psychological closeness (Mehrabian, 1981). Verbal behaviors such as praising students, addressing them by name and using humor in class may evoke immediacy (Gorham, 1988). But most of the existing research focuses on nonverbal immediacy resulting from teacher behaviors such as adopting a relaxed body position, varying one's vocal expression, moving around the classroom, smiling and looking at the class (Richmond, Gorham & McCroskey, 1987).
In previous research teacher immediacy (operationally defined to include both verbal and non-verbal behaviors) was positively correlated to student satisfaction with both the course and the instructor (Hackman & Walker, 1990). Immediacy was also related to Student Instructional Report (SIR) scores, notably including measures that might address the likelihood of taking future courses such as overall perceived value, the quality of instruction and interest in the subject area (Moore, et. al., 1996). Moreover, immediacy can be conveyed through mediated instruction, including real time instructional television presentations (Hackman & Zane, 1990) and video tape (Guerrero & Miller, 1998). Nonverbal immediacy was also related to cognitive learning in a number of studies (e.g., Christensen & Menzel, 1998), although in teacher immediacy studies student perceptions of learning rather than achievement test scores were the criterion.
Web courses are clearly at a disadvantage in terms of their immediacy. Web courses of the "computer managed" variety -- especially if primarily text based -- may lack both verbal and nonverbal cues entirely. "Constructivist" approaches that fall short of fully immersive virtual reality simulations -- still an impossibility with the limited (e.g. VRML) Web technologies available today -- are also likely to be lacking in immediacy cues, particularly nonverbal ones. Video-based Web courses of the "telecourse" or "virtual classroom" types would seem to be the most immediate, provided that the instructors themselves demonstrate immediacy behaviors. However, audiographic courses would presumably be limited to purely verbal immediacy cues, which are regarded as less effective than nonverbal ones. And all asynchronous classes, including telecourses recorded for later replay over the Web, also presumably miss important immediacy cues, such as the ability to address class members by name or to smile at individual students.
The question was whether a minimalist audiographic approach could still be as educationally effective as classroom instruction and whether it would appeal to learners sufficiently to sustain attendance to online instruction. Although Quealy and Langan-Fox (1998) found that audio-plus-text-and-still graphics was an effective format, their research was done under laboratory conditions with brief lessons on first aid that did not reflect real-world higher education environments. While teacher immediacy may be sustained with recorded video (Guerrero & Miller, 1998), it was not clear that the same would be true of recorded audio. Thus, we hypothesized that our approach would be inferior to classroom instruction:
H1: Student achievement from Web-based audiographic instruction will be inferior to live classroom instruction.
However, the audiographic telecourse approach should register low levels of teacher immediacy with students, which in turn would negatively affect student attitudes towards the course (Hackman & Walker, 1990).
H2a: Teacher immediacy will be lower for Web-based instruction than for live instruction.
And, consequently,
H2b: Student attitudes toward Web-based instruction will be lower (i.e. less positive) than for live classroom instruction.
Research on teacher immediacy has found that immediacy is related to student perceptions of achievement but not to actual measures of achievement (e.g., exam scores) such as the ones used here (Richmond, Gorham & McCroskey, 1987). In the present research perceived achievement (i.e., perceived improvement in competence due to the course) was analyzed as a component of student attitudes (see operational measures, below). Therefore, student achievement and immediacy were treated as if they were independent factors, while immediacy and attitudes were presumed to be related.
Stimulus Materials
The experimental Web site was prepared from a widely used introductory telecommunications textbook (Straubhaar & LaRose, 1997a) and the associated instructor’s manual (Straubhaar & LaRose, 1997b). Web pages were created from lecture outlines in the instructor’s manual. Links to Web sites that provided appropriate visual aids were added. The Web pages and the links were presented live in the on-campus lecture course during fall term, 1996. The class was taught in a large lecture hall to 230 students, but small groups of 6-12 students sat at the front of each class and interacted with the instructor.
The classroom audio was recorded and converted to Real Audio files for replay over the Internet. The external links were automatically launched at the appropriate points in the audio files. The Web site also contained review quizzes that students could submit for feedback, and a chatroom and bulletin board system for student interaction.
Procedure
The research took place the semester after the in-class lectures were originally recorded. All subjects were recruited at the initial meetings of two live lecture sections of an introductory telecommunications course taught at a large mid-western university. They were randomly assigned to treatment and control groups. The "virtual students" in the treatment condition viewed the Web pages and listened to the lectures using Web browser software on computers in their residences or in public computer labs. Students not selected to participate in the virtual course were the control group and continued in their on-campus lecture sections, where the same web pages were used by the live instructors on campus. In-class students were unable to view the Web pages except during in-class lectures. Both groups were provided with paper copies of lecture outlines that appeared on the Web pages so that they could take notes on the lectures as they listened.
Operational Measures
Student achievement was assessed using individual student scores on three, 75-item four-choice multiple choice exams. They were administered to both groups on printed paper forms. The experimental group entered their answers on a Web form using computers located in on-campus computer labs, after first recording them on paper. Control students used conventional mark-sensing sheets in their classrooms. The test items were drawn from the instructor’s manual (Straubhaar & LaRose, 1997b) or were constructed from open-ended review quizzes in the text so that they would be independent of content specific to either the live or mediated classes. One point was awarded for each correct answer and the three scores were summed to yield a total test score. There were two versions of each exam, differing by question order only, for each class section on each exam. The individual exam forms had Kuder-Richardson (KR-20) reliability coefficients between .75 and .80. Among the subjects in the study, the mean total score on the three exams was 157.43 (standard deviation = 22.91) and the three test scores had an alpha of .87.
Student ratings were obtained using standard Student Instructional Rating System (SIRS) forms used by the university at which the research was conducted (but which differed substantially from the Student Instructional Report used by Moore, et. al., 1996). The form had 21 items, each rated on a five-point scale with the categories of Superior (coded as 5), Above Average (4), Average (3), Below Average (2) and Inferior (1). It has been found to consist of five factors (Olson, 1978). The student interest factor was used as the basis for the measure of student attitudes. It consisted of four factor-specific items plus one general item that loads on all five factors. Scores on these five items 1 were summed to yield the overall student attitude score (mean = 16.92, s.d.= 3.34, alpha=.87). Two of the other factors in the SIRS instrument contained items that were relevant on their face to the issue of teacher immediacy and their items were combined into an eight-item additive scale 2(mean = 30.11, s.d.= 4.81, alpha = .86). The remaining two factors found by Olson (1978), course demands and course organization, had no apparent relevance to the issues under study and so were not included in the analysis.
Gender, grade point average and class level were obtained from student background items on the student rating form. In cases where rating forms were not completed, missing information was supplied from student records. Gender was coded 1 for female, 2 for male while class level ranged from 1 for freshmen to 4 for seniors. Grade Point Average was collapsed into the categories found on the student rating form, 1.9 or less (coded 1), 2.0-2.2, 2.3-2.7, 2.8-3.3, 3.4-4.0 (coded 5).
Attendance was assessed from information supplied on a follow-up questionnaire administered at the time of the final exam. In-class (control group) students were asked how many of the thirty in-class lectures, including the first class and exams, they attended. Virtual (treatment group) students were asked how many of the on-line lecture modules they had listened to on the web, of which there were 140 in all. The attendance of virtual students who did not supply this information on the exit survey was estimated from computer-generated information from the class web site. 3 Attendance was then computed as a proportion of the total classes (30 for classroom students, 140 for virtual students) that each attended.
Respondents
A total of 49 students recruited from the live classroom sections of the introductory telecommunications course completed the evaluation, including 24 in the control group and 25 in the treatment group. Thirty-one percent of the subjects were female and the rest were male. Forty-one percent were freshmen and the remainder were upperclassmen. Chi-square analysis revealed no significant differences in the gender and class level distributions between treatment and control groups.
Student rating forms were normally administered without any identifying student information to maintain confidentiality. Special permission was obtained from the university human subjects committee to permit voluntary self-identification by the respondents on their rating forms. However, several students chose to submit their rating forms anonymously while others refused to submit one at all and these cases were excluded from the analysis of the student rating data, but were included in the analysis of student achievement. This reduced the sample size to 34 when student attitudes were included and to 35 when teacher immediacy was one of the variables.
Data Analysis
Analysis of Covariance was used in which student achievement and the student attitude and teacher immediacy scales were dependent variables and treatment condition was the independent variable. Gender, grade point average and class level were used as covariates in all analyses using the simple factorial model in SPSS for Windows 7.5 (SPSS, 1997). Covariates were added in hierarchical order, before the main effect. Separate runs using classroom attendance, teacher immediacy and student attitudes as covariates were performed to minimize the impact of missing data.
Hypothesis 1 was not confirmed (see Figure 1). There was no significant main effect for treatment condition on total test scores (F1,44 = .07, p = .792). 4 The overall ANCOVA had a significant F4,44 = 8.59, p <.001 owing to significant covariance effects for gender (F1,44 = 10.38, p <.01) and grade point average (F1,44 = 23.93, p <.001). The beta coeffiecents indicated that males scored higher than females (B = 17.91) and learners with higher GPAs also scored higher on the exams (B = .30). Adding attendance as a covariate did not affect the results and there was no significant effect for the attendance covariate itself (F1,44 = 3.26, p = .079, results not shown in figure). Adding teacher immediacy and attitudes towards the course also had no effect and neither of these covariates was significant.
Hypothesis 2a was not confirmed. There was no significant main effect for treatment condition on teacher immediacy (F1,29 = .03, p = .873). The overall ANCOVA was not significant F6,29 = .855, p = .539, with gender, GPA, class level, attendance and exam scores as covariates.
Hypothesis 2b was also not confirmed. There was no significant main effect for treatment condition on student attitudes (F1,28 = 1.18, p = .287). The overall ANCOVA had a significant F6,28 = 6.12, p <.001 with significant covariance effects for gender (F1,28 = 5.68, p <.05, B = -2.12), attendance (F1,28 = 13.54, p <.001, B = .06) and immediacy (F1,28 = 14.60, p <.001, B = .344).
A relatively modest audiographic approach to Web courses based on the familiar telecourse model proved to be as educationally effective, immediate and enjoyable to learners as live instruction. Since the audiographic telecourse model built on materials produced for live classes that can be re-used in subsequent courses, both live and on the Web, other approaches that require original production or the continuing attentions of a full-time instructor are unlikely to be as cost effective. Returning to the previous taxonomy of Web courses, the virtual classroom and constructionist models are unlikely to be as cost effective as telecourses and computer managed instruction approaches for these reasons. And, while potentially cost effective, the computer managed approach is likely to be neither educationally effective nor to evoke teacher immediacy in view of the apparent inferiority of (primarily) text based instruction. Thus, we believe that the audiographic telecourse represents an optimal balance of educational effectiveness and cost effectiveness for Web courses.This tentative conclusion would break down if, unlike all previous forms of instructional technology, other types of Web courses consistently prove to be more effective than the classroom. Schutte’s (1997) and Althaus' (1997) virtual classrooms were reportedly more effective than their conventional classrooms, for example.
Our audiographic telecourse approach might have been more effective than the classroom under more favorable circumstances, too. Access to the audio lectures through on-campus computer labs was an important problem. Students complained in follow-up interviews that these facilities were not "as advertised" at the outset of the experiment, with the necessary Real Audio players generally unavailable or nonfunctional. In the one lab that system administrators "guaranteed" student access, there was a "simple" 13-step procedure for hearing the Web lectures. The 13th step, which was usually necessary, was "if that doesn’t work, go back to step 1" ! Novice Web users also had difficulty navigating the site, especially when links to other locations were launched automatically from within the Real Audio files, causing many students to lose their way, but no substantive modifications could be made per administrative fiat. So, formative evaluation and better technical support might easily have improved our results.
Moreover, before we could accept the possibility that there is a media effect for Web courses, more research needs to be done that can rule out competing explanations such as compensatory effort on the part of the Web students (as in Schutte, 1997). Moreover, a theoretical model must be advanced that might explain such a radical departure from the "conventional wisdom" that there are no media effects from instructional technology (see below).
Unexpectedly, teacher immediacy and student attitudes in the virtual course were equal to the live classrooms. To understand this, consider the possibility of "vicarious immediacy," or the evocation of feelings of closeness by witnessing interactions between one's teacher and other students. This mechanism is presumably present in any live lecture classroom in which the instructor is unable to interact with each and every individual student present. But the learners who are left out of the interactions may still perceive teacher immediacy. The two prior teacher immediacy studies that used mediated instruction as the stimulus materials did not isolate this element. In one, (Hackman and Zane, 1990) the students were able to interact in real time with the instructor via return audio links, while in the other (Guerrero & Miller, 1998) the videotaped instructors apparently "taught to the camera" instead of to students.
In the present research, the audiotaped lectures featured a variety of cues that might have evoked immediacy perceptions. Each lecture contained dozens of interactions with students in which students responded to open-ended questions and made comments about points on the course outline or provided relevant anecdotes from their own lives. Nor were the cues limited to purely verbal ones. The instructor frequently changed expression as he switched back and forth from lecturing, to personal anecdotes, to questioning, to elaborating on student responses -- and to bemoaning the frequent failures of instructional technology in his classroom. 5 Thus, students listening to the tapes via the Web may have vicariously experienced the immediacy of a live classroom. Indeed, the measures of immediacy used in the present research (e.g., "the instructor’s use of examples," "the student’s opportunity to ask questions.") could have been rated favorably on the basis of the recorded classroom interactions. In contrast, the control students were in large lecture sections where student interaction was not encouraged as much as in the Web version.
Still, the audiographic telecourse approach used in the present research limited teacher immediacy in at least two important ways. Based on the follow-up interviews, the most noticeable immediacy deficit was lack of eye contact with the instructor, which students said made it difficult to concentrate on the lectures. Virtual students also complained about their inability to ask questions spontaneously and to receive immediate feedback on their replies to structured review questions that were interspersed with the lecture material. Note that these limitations might be overcome by technical means, such as by posting an animated graphic of the instructor's gaze on each page, 6 by directing student questions to a discussion or chat area reserved for the class 7 or by automating the responses to review questions.
Limitations
An important limitation on the internal validity of the present research was lack of equivalence between the content in the virtual (treatment) and classroom (control) versions of the course. The classroom sections were taught by graduate teaching assistants who, although they used the same course outlines and accessed the same web pages in class as the virtual students did on-line, did not adopt the same teaching style as the Web course instructor. Despite the common outlines, they were free to vary the emphasis and depth of coverage on different topics.
However, the external validity of the experiment was arguably enhanced by this procedure. Real world learners on the Web do not have the option of taking the course from the same instructor in the classroom. Their choices are to take it on the Web or to take it from their local higher education provider—where they are likely to encounter local graduate teaching assistants. Likewise, the adoption decision at the institutional level also hinges on this choice. Is it more cost effective to teach an on-campus section with a graduate assistant or to use a Web course?
We also cannot generalize from the on-campus students who were the subjects of the present study to non-traditional adult learners who are apparently the true target of Web course efforts. There was a separate section for adult students who were recruited from ads in newspapers, who thus were self -selected into the virtual class and could not be included in the experimental design. Their test scores were generally comparable to the classroom sections and comparable to the experimental section recruited on campus. Of course, self-selection bias may account for the difference, since this group of virtual students may have had superior levels of motivation, academic ability, computer skills or entry-level knowledge.
The small sample size of the present study is also a concern although we believe its power to be adequate to rule out a "false negative" (Type II error) given the range of effects sizes found in prior studies of computer mediated learning (see footnote 3, above). However, replication with a larger sample size would certainly be in order.
As in prior research on computer mediated instruction, the issue of time on task is also a competing explanation for our results. That is, it may have been that the Web course materials themselves were less effective than classroom instruction but that the virtual students compensated by spending more time on the class. We suspect this was in fact true, especially for students who were forced to rely on public computer labs, where access was not as promised. However, the extra time probably went more into non-productive struggles with computers rather than extra learning time. The class attendance measure (the proportion of classes attended) provided a rough indication of time on (productive) learning tasks and it showed no significant difference between the experimental and control groups (F1,43=2.786, p=.102). In fact, the in-class students tended to attend a higher average proportion of lectures (63%) than the virtual students (48%). Still, we cannot rule out the possibility that the virtual students compensated for this by studying harder for their exams (but also see our comments about time-on-task, below).
Suggestions for Further Research
There is a need for a new variety of criterion measurement with which to evaluate instructional products that must succeed in a competitive environment, one that will address the issues of student retention and choice behavior. Measures used in standard instructional ratings forms (e.g. overall satisfaction, perceived value, interest in the subject matter) have some face validity for this purpose, but need further development to make them into reliable and valid measures for these purposes. Behavioral intentions to take future Web courses might also be considered. There is also a need for a valid criterion variable that could be used to compare courses in a developmental, formative evaluation context as opposed to the summative evaluation model which dominates both instructional technology and media effects research. Conjoint analysis is a possibility for assessing the competitive appeal of Web courses under development. There is also a need to expand assessment beyond the course material itself to other salient aspects of the virtual course experience. These include the convenience of taking courses in the home at the learner's own pace but also the cost and effort of establishing and maintaining Web access.
On a theoretical plane, the notion of vicarious teacher immediacy deserves further exploration. We wonder to what extent the immediacy effects found in prior research are the result of observing the interactions of others, rather than of direct experience. Vicarious immediacy also opens up the question of relationships between students. Social learning theory (Bandura, 1986) suggests that not all students will act as good models for all other students and that it should be possible to identify the characteristics of successful models. If vicarious immediacy is effective, how might we best enhance it by using Web features that could evoke it, such as pictures of the teacher's gaze, computer-synthesized vocal expressions, or animated "instructor avatars?"
But perhaps it is time to re-open the issue of immediacy to separate it from the conventional classroom model so that we can begin to think not just about how to make Web courses just as good as the classroom, but how to make them better than the classroom. Take the issue of interactions between students, for example. In the conventional classroom they are a threat to decorum but they are a common feature in Web courses of all stripes and can be exploited for pedagogical purposes there in ways that are perhaps not feasible in the conventional setting. Congenial interactions with other students might have a profound effect on affective learning, the proposed mechanism for understanding the impact of teacher immediacy (Rodriguez, Plax & Kearney, 1996)). In this vein, the well known ability of the Internet to support anonymous interactions could have a disinhibiting effect on both student-student and student-teacher interactions. We take note of Schutte's (1997) observation that interactions among his students may have accounted for the superior performance of his Web course compared to his classroom section. The asynchronous interactions that the Internet excels at might further disinhibit the students who feel compelled to sit on the sidelines in the classroom by giving them time to formulate their contributions. Another possibility to consider is that many users seem to treat computers as if they were people (LaRose & Bates, 1990; Reeves & Nass, 1996) or even as extensions of their own psyches (Turkle, 1995). So the computer that delivers the Web course in a sense becomes the instructor and the sum total of our interactions with it establish (sometimes quite intense) feelings of closeness to it -- or alienation from it.
This leads us to a new proposition with which to assail the conventional wisdom that "media will never influence learning" (Clark, 1994). It is that the relationship with the instructor/computer is so close, so immediate, that it is really not mass mediated instruction at all, but interpersonal tutoring. Even leaving aside the possibility of "artificially intelligent" Web courses that tailor lessons to individual needs (e.g. University of Illinois, 1998), Web-based instruction could create an intimate dialog between instructor/computer and learner that might so heighten affective learning that it could also yield gains in cognitive learning when compared to conventional methods. However, the opposite outcome is also possible. Extreme levels of teacher immediacy beyond those commonly found in the classroom may actually inhibit learning (Comstock, Rowell & Bowers, 1995).
This argument also throws a new light on the issue of "time on task" -- the time that students are willing to allocate to instruction in the face of competing activities. In conventional studies of instructional technology, time-on-task differences are dismissed as artifactual. For example, when it was found that the apparent superiority of early computer assisted instruction (CAI) courses could be explained in terms of the extra time learners spent on CAI systems compared to conventional instructional media, researchers dismissed claims that the computer medium itself was effective (Clark, 1985). The argument was that if conventional instruction merely taught the learners longer, it would be equally effective.
We won't argue that time on task is a threat to internal validity in studies of instructional media effects. However, we can expect Web courses to be "taught" in quite a different environment than conventional instruction and one in which time on task differences could be vitally important. Given that the raison de etre of Web courses is to reach harried adult learners, we must assume that they will be consumed amid the competing demands of family and work -- to say nothing of the competing entertainment and education options that are always just a click away on the Web. Time on task therefore emerges as a potentially important mediating factor in Web course effectiveness, one that the Web instructor must seek to minimize rather than dismiss. In fact, it might be evaluated as a criterion variable in the assessment of competing Web course designs.
Footnotes
1
Your interest in learning the course material, your general attentiveness in class, the course as an intellectual challenge and improvement in your competence in this area due to this course, your enjoyment of the course2
The instructor's enthusiasm when presenting course material, the instructor's interest in teaching, the instructor's use of examples or personal experiences to help get points across in class, the instructor's concern with whether the students learned the material, the instructor's encouragement to students to express opinions, the instructor's receptiveness to new ideas and others' viewpoints, the student's opportunity to ask questions, the instructor's stimulation of class discussion.3
At the end of each web page, a "roll call" icon was provided. Students indicated that they had completed the lecture by clicking the icon, and the proportion of Web pages "attended" in this manner was used in cases where students failed to provide the relevant information on the exit survey. Responses were matched to students via the PID they entered to log into the Web course server for each session.4
Regarding power, there are only two studies of Web courses extant but Johnston (1987) provided a summary of metanalyses of effect sizes for three different types of computer based learning (CAI, CMI and simulation) and that found a range of .07 to 1.13 among various age groups. In our analysis of effects on student achievement, a cell size of 24 yields an adequate (over 80) power rating for an effect size of .5. For the analysis of attitudes, a cell size of 18 yields adequate power for an effect size of .55 (Cohen, 1988), both of which appear to be within the possible range of effect size.5
For example, the links to external Web pages often malfunctioned as they had to be loaded off the instructor's hard drive -- the classroom network connection never functioned until the last week of class. On another occasion, a totally unrelated taped lecture from another course blared uncontrollably in the classroom for half the class session. Events such as these had highly noticeable impacts on the instructor's vocal expression.6
Indeed, that was the initial plan for the present course, but the instructor's expression varied so little across the pictures of him taken in class that this idea was abandoned.7
In later offerings of the class, a staff of undergraduate teaching assistants, working only for independent study credit, responded to student questions on an 80 hour-per-week schedule to provide this service.
References
Althaus, S. L. (1997). Computer-mediated communication in the university classroom: An experiment with on-line discussions. Communication Education, 46. 158-174.
Bandura, A. (1986). Social Foundations of Thought & Action. Englewood Cliffs, NJ: Prentice Hall.
Benyon, D., Stone, D., Woodroffe, M. (1997). Experience with developing multimedia courseware for the World Wide Web: The need for better tools and clear pedagogy. International Journal of Human-Computer Studies, 47, 197-218.
Cape Software (1998). Internet University. Available: www.caso.com/iu/courses.html
Christensen, L.J. & Menzel, K.E. The linear relationship between student reports of teacher immediacy behaviors and perceptions of state motivation, and of cognitive, affective and behavioral learning. Communication Education. 47(1), 82-90.
Clark, R.E. (1983). Reconsidering research on learning from media . Review of Educational Research, 53, 445-459.
Clark, R.E. (1985). Evidence for confounding in computer-based instruction studies: Analyzing the meta analysis. Education communication & Technology Journal,33(4), 249-262.
Clark, R.E. (1994). Media will never influence learning. Educational Technology for Research and Development. 47(2), 21-29.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Comstock, J. Rowell, E. Bowers, J. (1995) Foood for thought: Teacher nonverbal immediacy, student learning, and curvilinearity. Communication Education, 44(July), 251-265.
DeLoughry, T.J. (1996, September 20). New School for Social Research bolsters flagging enrollments with 90 on-line courses. The Chronicle of Higher Education, 43(4), 27-8.
Gorham, J. (1988). The relationship between verbal teacher immediacy behaviors and learning: Monitoring processes and product. Communication Education, 39, 354-368.
Gubernick, L. & Ebeling, A. (1997, June 16). I got my degree through e-mail. Forbes, 84-86.
Guerrero, L.K & Miller, T.A. (1998). Associations between nonverbal behaviors and initial impressions of instructor competence and course content in videotaped distance education courses. Communication Education. 47(1), 30-42.
Hackman, M.Z. & Walker, K.B. (1990). Instructional communication in the televised classroom: the effects of system design and teacher immediacy on student learning and satisfaction. Communication Education 39 (July), 196-206.
Harris, D.A. & DiPaolo, A.. (1996). Advancing asynchronous distance education using high-speed networks. IEEE Transactions on Education, 39(3), 444-449.
Hiltz, S. (1986). The "virtual classroom": Using computer-mediated communication for university teaching. Journal of Communication, 36(2), 95-104.
Hovland, C.I., Lumsdaine, A.S. & Sheffield, F.D. (1965). Experiments on mass communication. New York: Wiley.
Johnston, J. (1987). Electronic Learning: From Audiotape to Videodisc. Hillsdale, NJ: Lawrence Erlbaum.
Kozma, R.B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development. 42(2), 7-19.
LaRose, R. & Bates, B. (1990) Virtual Social Distance and Adoption of nteractive Telecommunication Technologies, Southwestern Mass Communication Journal, 6 (1), 34-43.
McComb, M. (1994). Benefits of computer-mediated communication in college courses. Communication Education, 43, April, 159-170.
Mehrabian, A. (1981) Silent messages: Implicit communication of emotions and attitudes (2nd Edition). Belmont, CA: Wadsworth.
Moore, A. Masterson, J.T., Christophel , D. M. & Shea, K.A. (1996). College teacher immediacy and student ratings of instruction. Communication Education 45(1), 29-39.
Moshell, J.M. & Hughes, C.E. (1996). The virtual academy: A simulated environment of constructionist learning. International Journal of Human-Computer Interaction. 8(1), 95-110.
Olson, L.A. (1978) Behavior-specific items for student rating of instruction. Paper presented at the meeting of the American Educational Research Association, Toronto, March.
Parker, L.A. (1976) Teleconferencing as an educational medium: A ten year perspective from the University of Wisconsin Extension. In Parker, L.A. & Riccomini, B. (Eds). The Status of the Telephone in Education. Madison, WI: University of Wisconsin Extension.
Pipes, R.B. & Wilson, J.M. (1996). A multimedia model for undergraduate education. Technology in Society, 18(3), 387-401.
Quealy, J. & Langan-Fox, J. (1998). Attributes of delivery media in computer-assisted instruction. Ergonomics, 41(3), 257-279.
Ragsdale, R.G. & Kassam, A. (1994). The magic of multimedia in education: Promises of the 21st Century. In Reisman, S. (ed). Multimedia Computing-- Preparing for the 21st Century. Harrisburg, PA: Idea Group.
Reeves, T.C. (1986). Research and evaluation models for the study of interactive video. Journal of Computer-Based Instruction, 13, 102-06.
Reeves, B. & Nass, C. (1996). The media equation. New York: Cambridge University Press.
Richards, C.N. & Ridley, D.R. (1997). Factors affecting college students’ persistence in on-line computer-managed instruction. College Student Journal 31, 490-95.
Richmond, V.P., Gorham, J.S. & McCroskey, J.C. (1987). The relationship between selected immediacy behaviors and cognitive learning. In McLaughlin, M. (Ed.) Communication Yearbook 10. Newbury Park, CA: Sage.
Rodriguez, J.I, Plax, T.G. & Kearney, P. (1996). Clarifying the relationship between teacher nonverbal immediacy and student cognitive learning: Affective learning as the central causal mediator. Communication Education 45(4), 293-305.
Schramm, W. (1977). Big Media, Little Media. Beverly Hills, CA: Sage.
Schutte, J. G. (1997). Virtual teaching in higher education: The new intellectual superhighway or just another traffic jam? [On-line]. Available: http://www.csun.edu/sociology/virexp.htm
Shein, E. (1997, March 10). Anytime, anywhere. PC Week, 14(10), 115-116.
SPSS, Inc. Statistical Package for the Social Sciences, Version 7.5. Chicago: SPSS, Inc.
Storck, J. & Sproull, L. (1995). Through a glass darkly. What do people learn in videoconferences? Human Communication Research, 22: 197-219.
Straubhaar, J. D. & LaRose, R. J. (1997a). Communications media in the information society (Updated 1st Edition.). Belmont, CA: Wadsworth.
Straubhaar, J.D. & LaRose, R.J. (1997b). Instructor’s manual for Communications Media in the Information Society (Updated 1st Edition). Belmont, CA: Wadsworth.
Turkle, S. (1995). Life on the screen. New York: Touchstone.
University of Illinois (1998). Welcome to cyberprof (On-line). Available: http://cyber.ccsr.uiuc.edu/cyberprof/general/Newpage/toblevel/welcome.html
University of Minnesota (1998), Streaming Video Program. Available: http://www.unite.umn.edu/streaming-video/
University of Texas (1998). World Lecture Hall. (On-Line) Availabile: http://www.utexas.edu/world/lecture
Voss, L. (1995). Technology supported learning: education on the edge of transition. In Mauer, H. (ed). Educational Media and Hypermedia. Accociation for the Advancement of Computing in Education.
Webster, J. & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40(6), 1282-1309.
Wetzel, C.D. , Radtke, P.H., & Stern, H.W. (1994). Instructional effectiveness of video media. Hillsdale, NJ: Erlbaum.
Whittington, N. (1987). Is instructional TV effective? A research review. American Journal of Distance Education, 47-57.
Woolsey, K.H. (1994). Multimedia opportunities, In K. Beattie & C. McNaught (Eds.), Interactive multimedi in university education: Designing for change in teaching and learning. Amsterdam, Netherlands: Elsevier Science, 93-97.
Yaverbaum, G.J. & Nadarajan, U. (1996). Learning basic concepts of telecommunications: An experiment in multimedia learning. Computers in Education, 26(4), 215-224.
ZDNet. (1998). 100 Most Wired Colleges (On-line) Available: http://www.zdnet.com /yil /content/college/colleges98/100.html
A previous version of this paper was presented to the International Communication Association Jerusalem, Israel July 23, 1998. Preliminary results were presented at the Association for the Advancement of Computers in Education meeting, Calgary, Alberta, June, 1997. The authors would like to acknowledge the contributions of Dr. Carrie Heeter and her staff at of the MSU Communication Technology Laboratory to the design of this course. The authors wish to thank Kathleen Stuart for her assistance in interviewing students about their experiences as part of her course requirements for a graduate research seminar.
Robert LaRose is a Professor in the Telecommunication Department at Michigan State University where he teaches courses in telecommunication technology, social science research methods and the social impacts of information technology. He holds a Ph.D. in Communication Theory and Research from the Annenberg School at the University of Southern California.
Address: Department of Telecommunication, 409 Comm Arts Building, Michigan State University, East Lansing, MI 48824-1212Jennifer L. Gregg is a doctoral student in the Mass Media program at Michigan State University. She received her Master of Arts in 1996 from Iowa State University in Journalism and Mass Communication. Her research interests include technology applications for the elderly.
Address: Department of Telecommunication, 409 Comm Arts Building, Michigan State University, East Lansing, MI 48824-1212Matthew S. Eastin is a doctoral student in the Mass Media program at Michigan State University. In 1997, he received his Master of Arts degree in Journalism from the University of Nebraska-Lincoln. His research interests include mass media effects as well as the diffusion and use the Internet.
Address: Department of Telecommunication, 409 Comm Arts Building, Michigan State University, East Lansing, MI 48824-1212
