|
|
Educational Applications of CMCS:
Solving Case Studies through Asynchronous Learning Networks
Raquel Benbunan-Fich
Stillman School of Business
Seton Hall UniversityStarr Roxanne Hiltz
Computer and Information Science Department
New Jersey Institute of Technology
Table of Contents
Abstract
Case studies are an important component of many business curricula. However, in-class discussion of case studies suffers from temporal and geographical limitations. Computer-mediated communication systems (CMCS) can be used to overcome these constraints. An Asynchronous Learning Network, a CMCS supporting "anytime/anywhere" interaction and tailored for educational activities, may be used to expand and enrich case discussions. ALN-mediated discussions allow students and instructors from remote locations to participate in the discussion at their own convenient times. A field experiment was conducted to test the effectiveness of an ALN vs. traditional manual methods in individuals and groups discussing and solving a case study. Findings indicate that groups working in an asynchronous networked environment produced better and longer solutions to the case study, but were less satisfied with the interaction process.
Introduction
A growing number of articles in the academic literature (e.g. Alavi, et al. 1997; Althaus, 1997; LaRose, et al. 1998; Hiltz & Wellman, 1997; Webster & Hackley, 1997) and even in the news media (e.g. Arenson, 1998) are focusing on the use of Computer-Mediated Communication Systems (CMCS) in education. The convergence of technological and institutional factors is contributing to this rise in interest. At the technological level, rapid advances in telecommunications are linking not only individual students with their peers and instructors (Hiltz, 1994), but also entire schools with their counterparts in other locations (Perrone, et al. 1996; Wheeler, et al. 1995).
The new technological possibilities are attractive targets for exploration when some educational institutions are faced with declining resources and are looking for ways to reduce costs or to expand their markets (Alavi, et al. 1997; LaRose, et al. 1998; Leidner & Jarvenpaa, 1995). However, the educational challenge is to develop pedagogically effective technology-mediated learning environments that truly enhance the quality of education (Althaus, 1997).
There are many ways in which CMCS can be integrated into education. They can be used to transmit content (to deliver instruction) and/or to support the administrative and communication activities that take place in a course (Hiltz, 1994). As a content-transmission tool, IT can complement or completely replace the traditional role of textbook and teachers. As a communication-support tool, IT can be used to extend the availability of the professors beyond office hours and to accomplish administrative activities such as distribution of materials, reminders and notifications. These two roles can be effectively combined in the context of a Computer-Mediated Communication System (CMCS), tailored to support educational activities (Benbunan-Fich, 1997).
One of the terms used to describe CMC-based education is "Asynchronous Learning Networks" or ALN's. An ALN is a communication system designed to support "anytime/anywhere" interaction among students and between students and instructors. An ALN structures interaction by providing a combination of database and conferencing system that allows people to exchange messages and carry out asynchronous discussions in an organized manner (Hiltz & Wellman, 1997).
ALN's represent a new paradigm for teaching and learning, with both unique problems of coordination and unique opportunities to support active, collaborative (group or team-based) learning (Harasim, et al. 1995). ALN's can support entire courses or specific assignments such as the discussion and solution of case studies (Benbunan-Fich, 1998). This paper addresses this issue and reports the results of a field experiment in this area.
Literature Review
Computer-Mediated Learning
Computer-mediated learning can be classified in terms of the framework proposed by Johansen (1992). The framework was originally designed to describe different modes of interaction according to two dimensions: time and place. Interaction can occur at the same time (synchronous) or at different times (asynchronous). Members can meet in the same place (proximate) or in different places (disperse).
Figure 1 shows different alternatives for the integration of CMCS in the classroom. Same time/same place refers to situations in which traditional classrooms are furnished with computers for every student. In this mode of interaction, the traditional classroom environment is enhanced with a communication system (Alavi, 1994). Same time/different place situations occur when lectures are taught at the same time to students located in (at least two) different places, using video and data links across locations (Alavi et al. 1995). In this case, the traditional classroom is networked with other classrooms in different locations (Webster & Hackley, 1997).
Different time/same place normally refers to the case in which people work in shifts but share a common meeting room or project room where they leave messages for each other and share materials (Johansen, 1992). A possible application would be a class in which lectures are available on videotape and are offered at different times, but in the same physical location such as a library room (Hiltz, 1994). Another application would be a computer lab that students can use out-of-class for their computing activities (Chizmar & Williams, 1996).
Finally, different time/different place refers to totally distant classrooms in which students and professors rarely meet face-to-face. Content-transmission and communication-support take place in the context of an ALN (Benbunan-Fich, 1997). This category corresponds to completely electronic classrooms or Virtual Classrooms® (Hiltz, 1994).
Figure 1: Typology of Dispersion
Time
Same
Different
Place
Same
Synchronous/Proximate
Technology-enhanced classrooms
Shared physical workspace:
e.g. Video taped lectures in a single location or a networked computer lab.
Different
Synchronous/Disperse
Networked classrooms
Asynchronous/Disperse
Virtual Classrooms in ALN environments
Web Telecourse
Adapted from Johansen (1992)
Empirical studies in computer-mediated learning (e.g. Alavi, 1994; Alavi, et al. 1995; Alavi, et al. 1997; Hiltz, 1994; Leidner & Fuller, 1996; Webster & Hackley, 1997) can be organized in terms of this framework. Most of the research conducted in this area is based on the use of groupware systems in educational contexts. Consistent with the groupware literature (e.g. Fjermestad, et al. 1993; Pinsonneault & Kraemer, 1989) from which they are derived, studies in the area of computer-mediated learning are mainly concerned with two types of outcomes: performance and perception. Performance outcomes try to assess the effectiveness of the teaching/learning process from the student's perspective. They usually involve an assessment of task performance (e.g., quality of group report in case discussions) and assessment of learning (e.g., exam grades are typically used as proxy measures for learning achievement). Perception measures deal with attitudes toward the technology or system, satisfaction with the process and the solution, and other subjective indicators that could affect performance (Benbunan-Fich, 1997).
Same time/same place studies (e.g., Alavi, 1994; Leidner & Fuller, 1996) have been mostly focused on the use of synchronous group support systems 1 (in decision rooms) to support discussion and solution of case studies in MIS courses.
Alavi (1994) compared computer-supported vs. face-to-face unsupported groups of MBA students solving case studies in an introductory Management Information Systems (MIS) course. Computer-supported groups expressed higher levels of perceived skill development, self-reported learning and evaluation of classroom experience than non-supported groups. Although there were no significant differences in midterm scores, final test grades for students in computer-supported groups were significantly higher than those of the students in non-supported groups.
Leidner and Fuller (1996) examined whether a computer-mediated collaborative learning environment (same time/same place) involving case analyses was superior to individual learning involving individual case analyses. The study found that students who discussed the cases in groups were more interested in the material and perceived themselves to learn more than students working alone. However, students who worked independently outperformed students who discussed the cases in groups before solving them individually (Leidner & Fuller, 1996).
Same time/different place studies (e.g., Alavi, et al. 1995; Alavi, et al. 1997; Webster & Hackley, 1997) are based on the use of some form of audio, video and graphic link between two or more sites. The technology is used to support synchronized work (lecture delivery or case discussion) between local and remote participants.
Alavi and colleagues (1995) conducted a longitudinal field study to compare the effectiveness of three collaborative learning environments: computer-mediated proximate groups, computer-mediated non-proximate groups and face-to-face unsupported groups. The study found that the three environments were equally effective in students' knowledge acquisition and satisfaction with process and outcomes. However, students who worked in non-proximate groups showed higher critical thinking skills and were more committed to their groups than students in the other two conditions.
In another study, Alavi et al. (1997) investigated the effectiveness of networked classrooms at two universities (same time/different place). Results indicate no significant differences in mastery of the material due to the medium (face-to-face or videoconference lectures), but students were less satisfied with the videoconference lectures than with the face-to-face ones. This study included an out-of class team project using an asynchronous group support system (different time/different place) where students from both locations were grouped to discuss and solve a case study.
Webster and Hackley (1997) conducted an exploratory field study on several networked classrooms, same time/different place, in different subject matters such as accounting, chemistry, computer science, etc. Although this study did not include control conditions for comparison purposes, it served to develop recommendations to increase the effectiveness of networked classrooms.
Different time/different place studies (e.g., Hiltz, 1994; Hiltz & Wellman, 1997; LaRose, et al. 1998) deal with complete distance learning environments where the students get the lectures via videotapes (or electronic lectures via computer) and use the ALN or the Web to communicate with the professor, or with other students. A longitudinal study (Virtual ClassroomÔ in Hiltz, 1994) compared traditional classrooms with totally distant classrooms using a combination of videotaped lectures and ALN-supported communication. The study found no consistent significant differences between traditional (non-supported) courses and VC-supported classes in mastery of the material (actual learning) as measured by grades. Subjectively, however, most students reported that VC was overall a better way of learning than traditional classes.
Another interesting study in this category (LaRose, et al. 1998) compared a Web telecourse with traditional classroom instruction. Students in the Web telecourse condition listened to pre-recorded audio classroom interactions while viewing a detailed course outline and illustrative sites over the World Wide Web. Findings showed that students in the telecourse had test scores and perceptions (student attitude and teacher immediacy ratings) equal to those in the traditional classroom.
Findings across studies seem to suggest that the use of a computer-mediated communication medium is equally as effective as traditional methods in terms of mastery of the material (Alavi, et al. 1995; Alavi et al. 1997; Hiltz, 1994; LaRose, et al. 1998). In some cases (e.g Alavi, 1994), computer-mediated conditions outperformed their manual counterparts.
Most of the studies reviewed here are focused on the macro level (entire courses). We wondered what would be the effects of the communication medium at the micro level, i.e. on particular assignments such as the discussion of case studies.
Learning Through Case Studies
Case studies are the hallmark of business education. Many schools are using case studies in their curricula to transmit content and real life experiences that require student involvement (Leidner & Jarvenpaa, 1995). The case method of teaching seeks to enable students to process instructional inputs and assimilate course materials (Leidner & Fuller, 1996). Case studies present real or hypothetical situations that demand group discussion and the use of concepts to develop recommendations or achieve a preferred solution (Barnes, et al., 1994; Benbunan-Fich, 1997; Hashim, et al. 1991; Silver, et al. 1995).
The nature of case analyses can be proactive or reactive. Proactive analysis consists of anticipating the consequences of a situation presented by the case and to make decisions about what can be done. Reactive analysis is a retrospective analysis of the situation in order to identify the objectives and the outcomes, and to make recommendations about what could have been done differently (Silver, et al. 1995).
The use of case studies as teaching tools is based on four fundamental principles: situational analysis, student involvement, non-traditional instructor role and relationship between analysis and action. Situational analysis forces the student to deal with the characteristics of the situation presented by the case (absence of information, conflict of objectives and the imbalance between needs and resources). Student involvement is demanded by the very nature of the case method in which learning occurs through discussion and interaction with peers (Harasim, 1990; Hashim, et al. 1991). In this process the instructor is a mere facilitator of the discussion, not the traditional knowledgeable lecturer. Case discussions force students to bridge the gap between the academic goal of knowing and the practitioner's necessity of acting (Barnes, et al. 1987).
Preparation for case study discussions can be assigned to individuals or groups. Individual case preparation forces each student to think in isolation, using his or her own opinions, experiences and resources to analyze the situation and develop recommendations. In contrast, group case preparation is more enriching (Benbunan-Fich, 1997). Groups solving case studies are likely to experience process gains such as stimulation, synergy, more information and learning (Nunamaker et al., 1991), and the development of higher order cognitive skills (Hiltz, 1994). Teamwork produces the externalization of the thought processes, the comparison of alternative perspectives, social facilitation, better learning, high self-esteem and more positive attitudes toward the learning experience (Salomon & Globerson, 1989).
However, as in any other group endeavor, group discussions experience process losses such as information overload, and coordination problems (Nunamaker et al., 1991) and "free-riding" or social loafing (Shepperd, 1993). Instead of pooling their mental efforts, some team members may actually show reduced expenditure of mental effort, loafing behavior and even effort avoidance in ways that debilitate learning, just because solving the case is the responsibility of the whole group (Salomon & Globerson, 1989). To counteract free-riding and social loafing, team members are likely to exert a greater effort if their contributions are identified, if the outcome (or case solution) is important or personally relevant, and if they perceive a clear relationship between contribution and outcome (Shepperd, 1993).
Regardless of the approach to case preparation (individual or group), the case method of teaching is based on the case discussion, where participation is the crucial element. "The spirit of the methodology mandates people reacting to each other and learning through the synergies of conversation" (Hashim, et al. 1991: 374). However, interaction patterns are generally restricted to face-to-face lectures in which many factors such as social desirability, air-time fragmentation and blocking constrain the students' ability to participate in the process (Nunamaker, et al. 1991).
In general, lack of participation in a face-to-face case discussion is due to a number of elements, including fear of reprisals, fear of being evaluated or being mocked by peers, fragmentation of available speaking time, cognitive inertia (the tendency to think along the same lines), production blocking (inability to produce a meaningful contribution) and domination by more knowledgeable peers (Nunamaker, et al. 1991).
ALN's are designed to overcome many of the factors that constrain participation in a face-to-face discussion (Hiltz & Wellman, 1997). A communication system can increase group process gains, such as synergy, pooling of information, objective evaluation, cognitive stimulation and learning; and decrease group process losses, such as fragmentation, blocking, domination, evaluation apprehension and information overload (Nunamaker, et al. 1991). In particular, asynchronous interaction increases the time available to read or reread a message and formulate a comment. This can improve in-depth reflection and development of a topic (Harasim, 1990). Increased opportunity for member input may also enhance the quality of decision-making (Rice, 1984).
Probably the biggest advantages of ALN's to support case study discussions is teaming up groups of people impossible to assemble in a face-to-face meeting (Hiltz, 1994) and the integration of external expertise in a systematic way into the curriculum. Multidisciplinary teams composed of students, professors of different areas and practitioners can meet at their own convenience through an ALN to discuss case studies. These multidisciplinary groups can leverage the students' knowledge (Alavi, et al. 1997) and allow business partners to take an active role in the education of future professionals, as well as promote the emergence of new employment possibilities, which can flourish from these alliances.
The downside of ALN's includes procrastination. Since students do not have to participate at any specific time, they may not participate regularly at all. The anxiety produced by delays and different participation rates or "login-lags" (Dufner et al. 1994) may reduce the quality of decision making. Members may go along with an initial suggestion, even if they do not agree with it, in order to accelerate the process and meet a deadline (Harasim, 1990). In addition, students may feel that the medium is not as warm or personal as face-to-face classes, and this may also decrease motivation and satisfaction with the process. Instructors should develop the right incentives for the students, so that regular and legitimate participation is achieved (Benbunan-Fich, 1997).
Table 1 summarizes the advantages and disadvantages of using ALN's to support the solution of case studies.
Table 1: Solving Case Studies through ALN's
Advantages
Disadvantages
Increase group process gains
Decrease group process losses
In-depth reflection on topics
Higher quality decisions
Integration of external expertise
Procrastination
Frustration due to "login-lags"
Pressure to meet deadline
Impersonal medium
Incentives for participation
Based on this review of the relevant literature, the next section will present several hypotheses that require empirical testing. The independent variables will be medium (ALN-support vs. manual support) and type of work (individual vs. teamwork). The task will be the discussion and solution of a case study in computer ethics. The dependent variables will be consistent with prior studies in the area of computer-mediated learning (e.g. Alavi, 1994; Alavi, et al. 1995; Alavi, et al. 1997; Hiltz, 1994). The two categories of dependent variables will be performance and perception. In the context of case study discussions, performance outcomes could be subjective (e.g., grade as a proxy of quality of the solution) and objective (e.g., length of the final report). Perception outcomes refer to individual opinions about the process, for example satisfaction with the process of solving the case with the system or method provided (Alavi, 1994).
Hypotheses
Solution Quality
Groups are more creative at generating options and probing their advantages and disadvantages than are single individuals (Nunamaker, et al. 1991; Turoff & Hiltz, 1982). Groups are also better than individuals at making decisions (Hill, 1982; Rice, 1984). There is some support in the moral reasoning literature that groups produce better solutions to ethical and social dilemmas than individuals do (Peek, et al. 1994). At the individual level, moral judgment is the product of an individual's basic structure for perceiving reality, while at the group level ethical discussions force members to share not only facts but also values and work with different viewpoints and moral frames of reference (Nichols & Day, 1982). Therefore, the solution of a moral dilemma decided upon by a group should be superior to an individual's consideration of a dilemma. In line with these ideas, we hypothesize:
H1a: Groups will produce higher quality solutions to an ethical case study than will individuals.
When groups are working asynchronously, through a CMCS, members can reflect longer and in more depth about their contributions than when they are in a face-to-face discussion (Hiltz, 1994). In fact, other studies (e.g. Ocker et al., 1995) have reported that ALN-supported conditions will tend to produce higher-quality solutions than their manual counterparts. Therefore,
H1b: Participants working through an ALN will produce higher-quality solutions to an ethical case scenario than will their manual counterparts.
When individuals work alone and have to produce reports by hand, the quality of the solution to the case study will be a function of their own knowledge, understanding and resources. Furthermore, the benefits of working in a computer-supported environment will not be available to those individuals. Hence,
H1c: Individuals working alone in a traditional manual setting will produce a lower-quality solution to an ethical scenario.
Length of Reports
Since groups are able to pool more ideas and to combine information from different sources (Nunamaker, et al. 1991), it is expected that group reports will be longer than individual reports.
H2a: Groups will submit longer reports than individuals working alone.
Because of the ease of editing and improving the text using a computer editor as opposed to pencil and paper, we expect ALN-supporter participants to submit longer responses than their manual counterparts,
H2b: Participants working through an ALN will submit longer reports.
For ALN-supported groups, two effects will be present. On the one hand, due to the combination of contributions from different team members, longer responses could be possible (Hiltz, 1994; Nunamaker, et al. 1991). And, on the other hand, it will be easy to compile individual contributions and the content of the discussions due to the use of the communication system (Harasim, 1990). The combination of these two factors (more contributions and ease of compiling them) could produce an interaction effect that is more than the sum of the main effects:
H2c: ALN-supported groups will produce the longest reports.
Process Satisfaction
Satisfaction in computer-mediated groups tends to be lower than in face-to-face groups (Wilson, et al. 1997). In their review of the empirical groupware literature, Pinsonneault and Kraemer (1989) conclude that groups supported by a CMCS tend to be less satisfied than their manual counterparts.
In fact, when communication is mediated by an ALN, process satisfaction is affected by participation problems: absent members (Smith & Vanecek, 1988), "login-lags" (Dufner, et al. 1994), low degree of cooperation (Pinsonneault & Kraemer, 1989), and delayed feedback (Rice, 1984). Computer-mediated groups must work harder to accomplish the same results as face-to-face groups (Galegher & Kraut, 1994).
Students expected to produce individual output and submit it through the ALN are not supposed to be affected by this phenomenon. Even though the medium may be perceived as impersonal (Harasim, 1990), they are not suppose to interact or cooperate with others to produce results. Hence,
H3: ALN-supported groups will report the lowest levels of process satisfaction.
Methods
A quasi-experimental field trial was conducted to examine the effectiveness of different approaches to solving a case study. The experimental design was a 2x2 factorial crossing teamwork (individual vs. group work) with communication support (manual-offline vs. asynchronous computer conference). See Figure 2. The task was the solution of one of the Anderson et al. (1993) cases. Benbunan-Fich (1998) describes in more detail the nature and content of the case used in this experiment.
Figure 2: Experimental Design
Computer Support
Manual
Online
Teamwork
Individual
IM
IO
Group
GM
GO
In the individual/manual condition (IM), students solved the case individually in an open-book in-class exercise. In the individual/online condition (IO), students submitted their individual responses in a computer conference by using the "question-response activity" software on the conferencing system. This feature allows students to submit their individual responses without seeing what anybody else has written, but after their solutions are posted, they can read the answers of others (Hiltz, 1994). Students in the IM condition were given the case one week before the date of the in-class exercise and were advised to prepare the solution for the upcoming session. In the IO condition, the case was posted online in a computer conference one week before the due date.
In the group/manual condition (GM), team members solved the case by interacting in a face-to-face session. They discussed the scenario and wrote the final report without ALN support. Here again the students were given the case one week before the date of the group discussion and were advised to prepare their individual position statements for the group discussion. In the group/online condition (GO), team members interacted asynchronously using the computer conference as the only means of communication to discuss and solve the case. Each group was placed in a different computer conference. All conferences were seeded with the same comments regarding instructions on how to proceed and the text of the case.
Participants were undergraduate students in one of the core courses for computer science majors ("Computers and Society") at a major technological university. The task was implemented as one of the assignments in the course. The sample was composed of 140 students, distributed across conditions as follows: 42 in Individual/Manual, 42 in Individual/Online, 28 in Groups/Manual and 28 in Groups/Online. Due to scheduling constraints and the loss of groups in both conditions because of "no-shows", fewer participants completed the experiment in group conditions. Data collection instruments included a pre-test questionnaire to gather demographic information, and a post-test questionnaire to collect the students' perceptions.
Results
Since this experiment was conducted in an actual field setting, there was a limitation preventing a truly random assignment of subjects to conditions. Students in the distance section of the course could be assigned to online conditions only, while students in the traditional (on-campus) section could be assigned to any condition. As a result of this, most of the participants who ended up in online conditions came from the distance section.
Demographic data were used to identify differences between traditional and distance sections. These analyses showed that age and employment were the only variables significantly different. Students enrolled in the distance section were older and had much more work experience than those enrolled in the traditional face-to-face section. To control for these differences, the variable "age" and "months-of-full-employment" were used as covariates.
Analyses of covariance were conducted to test for main effects and interaction effects between the two factors (teamwork and computer support) on the following dependent variables: solution quality, report length and process satisfaction.
Solution Quality
Three computer professionals with extensive teaching and work experience were selected as "expert judges" to rate the quality of the final reports. The judges were unaware of the experimental conditions and each one received a complete set of identical looking printed versions of the reports. The judges were instructed to rate the overall quality of the reports using a scale from 1 to 100 (1 for very low quality solutions and 100 for very high quality solutions).
As there was a high level of agreement among the judges(inter-rater reliability = .85), the scores were averaged out to produce a measure of solution quality. Table 2 presents the results of this dependent variable.
Table 2: Solution Quality Results
Means by Condition
Manual
Online
Individuals
53.29
62.57
57.93
Groups
58.97
62.05
60.51
56.13
62.31
58.94
Model
F = 2.34
p = .04
*
Teamwork Effect
F = 0.80
p = .37
Online Effect
F = 3.88
p = .05
*
Interaction (TW*OL)
F = 1.14
p = .28
* = Significant at p=< .05
According to the score provided by the judges, ALN-supported participants (individuals and groups) submitted higher quality reports than their manual counterparts. The online mean (62.31) is significantly different from the manual mean (56.13) at p = .05, which supports the prediction of H1b.
Length of Reports
The number of words in each report was counted to obtain a measure of the length of the reports. Table 3 presents the results of this dependent variable.
Table 3: Length of the Reports Results
Means by Condition
Manual
Online
Individuals
380.94
461.84
421.39
Groups
390.51
756.02
573.27
385.73
608.93
480.82
Model
F = 8.98
p = .0001
***
Teamwork Effect
F = 11.60
p = .0009
***
Online Effect
F = 21.10
p = .0001
***
Interaction Effect
F = 10.02
p = .002
**
** = Significant at p < .01; *** = Significant at p<.001
The analysis shows that participants who worked in groups submitted significantly longer reports (group mean 573 words) than participants who worked alone (individual mean 421 words), which supports the prediction of H2a (p < .001). ALN-supported participants submitted longer responses (online mean 609 words) than those working manually (manual mean 386 words), thus supporting H2b (p < .001).
There is also a significant interaction effect. The combination of teamwork and computer support resulted in significantly longer reports for groups online (mean 756 words) than for any other condition. This supports H2c with a significance level of p < .01. Figure 3 illustrates this interaction effect.
Process Satisfaction
The post-test questionnaire measured satisfaction with the process of solving the case, using a three item semantic differential scale (anchored at five points), adapted from Green and Taber (1980). Since the scale was deemed reliable (Cronbach Alpha = .83), the scores of the three items were added up to create a composite measure of process satisfaction. The composite score, which goes from 6 to 15, was used to test the hypothesis. Table 4 presents the statistical results of process satisfaction.
Table 4: Perception of Process Satisfaction
Means by Condition
Manual
Online
Individuals
11.15
11.37
11.26
Groups
12.73
10.79
11.76
11.94
11.08
11.51
Model
F = 3.28
p = .01
**
TW (Teamwork Effect)
F = 2.62
p = .11
OL (Online Effect)
F = 3.78
p = .06
Interaction (TW*OL)
F = 6.33
p = .01
**
** Significant at p =< .01
According to these results, online groups were the least satisfied with the process; their mean (10.79) is the lowest among all conditions, which supports H3 at a significant level (p=.01). There is a significant interaction effect between teamwork and computer support. Figure 4 illustrates this "disordinal" interaction2 (Pedhazur & Pedhazur, 1991). In this case, the rank order of the process satisfaction goes from best to worst between manual and online groups. Manual groups were the most satisfied with the process (mean 12.73), while online groups were the least satisfied with the process (mean 10.79).
Summary, Discussion, Conclusion
Consistent with the literature (Hiltz, 1994; Ocker, et al. 1995), ALN-supported participants -- individuals and groups -- produced better reports than did their manual counterparts. It seems that in an ALN environment the potential visibility of individual responses (Hiltz, 1994) combined with in-depth reflection that can be achieved through asynchronous work (Harasim, 1990; Rice, 1984) resulted in higher quality solutions.
In terms of report length, as expected groups submitted longer reports than individual participants did. The contributions of different members in the discussion increased the length of the reports (Nunamaker, et al. 1991). Groups/online (GO) also benefited from the availability of a written transcript of the discussion produced by the system (Harasim, 1990). Due to the combination of these two factors (group input and written transcript of discussion), GO submitted the longest reports.
With respect to process satisfaction, consistent with the literature (Wilson, et al. 1997), online groups were the least satisfied with the process due to the nature of asynchronous interaction, characterized by delayed feedback (Rice, 1984) and "login-lags" (Dufner, et al. 1994). Apparently, groups working in an asynchronous environment had more difficulties coordinating the distribution of work and had to work harder than face-to-face groups (Galegher & Kraut, 1994). Since no other means of communication was allowed, it was up to each team to decide when to stop waiting for absent members (Smith & Vanecek, 1988). For these reasons, groups who used the ALN were the least satisfied with the process.
In summary, this field experiment has found that groups who used an ALN to discuss and solve a case study submitted better and longer solutions than their counterparts but were the least satisfied with the process. In fact, the combination of teamwork with the use of an ALN results in better and longer reports than if only one of these factors is present, but negatively affects process satisfaction.
The use of a field experiment to conduct this study is the source of its strengths and limitations. An experiment conducted in a real setting (a field experiment) has great potential for the generalization of results, but can be affected by the many factors that can not be controlled for in the real world (Pedhazur & Pedhazur, 1991). In this field experiment, some of the internal validity was lost because experimenters had no control over what students are enrolled in which sections (traditional or distance). But as in LaRose et al.'s (1998) study, having better potential for the generalization of the results compensates for this loss.
The implications of these findings are manifold. First, an ALN was found to be a feasible medium for collaborative learning activities such as case study discussions. Moreover, the use of an ALN enhances group performance, due perhaps to the potential visibility that the system can provide to each response, combined with deeper reflection in asynchronous work.
However, computer-mediated groups reported the lowest levels of process satisfaction because of the difficulties of interacting in an asynchronous environment. Therefore, one of the challenges for designers of ALN's is to provide effective coordination tools (such as agenda, voting, and polling) for structuring asynchronous interaction and overcome the inherent limitations of the medium (Dufner, et al. 1994).
In this field experiment, students who never met face-to-face were able to interact through the system and discuss a case study. By using an ALN, part-time students could team up with full-time students without the typical scheduling conflicts. But more importantly, the use of this system can open new possibilities for establishing partnerships with practitioners in different fields and leverage business education (Alavi, et al. 1997). Business practitioners could have a first-hand involvement in higher education and contribute to the development of the human resources they need for their own companies. For students, this would represent a definite enhancement in the quality of their education.
Footnotes
1 "A group support system is a special type of groupware to support face-to-face interaction, which usually consists of networked computer workstations for participants and a facilitator, a large public screen to display the results of group discussion, and software to support group processes such as brainstorming voting and ranking." (Alavi, et al. 1997).
2 Disordinal interaction means that the predicted scores for one condition will not be consistently larger than those for the other condition. (Pedhazur & Pedhazur, 1991: 49).
References
Alavi, M. (1994). Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly. June, pp. 150-174.
Alavi, M., Wheeler, B., & Valacich, J. (1995). Using IT to reengineer business education: An exploratory investigation to collaborative telelearning. MIS Quarterly, September, pp. 294-312.
Alavi, M., Yoo, Y., & Vogel, D. (1997). Using Information Technology to add value to management education. Academy of Management Journal, 40 (6), pp. 1310-1333.
Althaus, S.L. (1997). Computer-mediated communication in the university classroom: An experiment with on-line discussion. Communication Education, July, pp. 158-174.
Anderson, R., Johnson, D., Gotterbarn, D., & Perolle, J. (1993) Using the new ACM Code of Ethics in decision making. Communications of the ACM, 36(2), pp. 98-108.
Arenson, K. (1998). More colleges plunging into uncharted waters of on-line courses. New York Times, November 2nd.
Barnes, L., Christensen, R., & Hansen, A. (1994). Teaching and the case method. Boston, MA: Harvard Business School Press.
Benbunan-Fich, R. (1997). Effects of computer-mediated communication systems on learning, performance and satisfaction: A comparison of groups and individuals solving ethical scenarios. Ph.D. Dissertation, Rutgers University, Newark, NJ. UMI Dissertation Services # 9729583.
Benbunan-Fich, R. (1998). Guidelines for using case scenarios to teach computer ethics. ACM SIGCAS, September, 20-24.
Chizmar, J., & Williams, D. (1996). Altering time and space through network technologies to enhance learning. Cause/Effect, 19 (3), pp. 14-21.
Dufner, D.; Hiltz, R., & Turoff, M. (1994). Distributed Group Support: A preliminary analysis of the effects of the use of voting tools and sequential procedures. HICSS Proceedings.
Fjermestad, J.; Hiltz, S. R., & Turoff, M. (1993). An integrated framework for the study of group decision support systems. HICSS Proceedings, Vol. IV, pp. 179-188.
Galegher, J., & Kraut, R.R. (1994). Computer-mediated communication for intellectual teamwork: An experiment in group writing. Information Systems Research, 5(2), pp. 110-138.
Green, S., & Taber, T. (1980). The effects of three social decision schemes on decision group processes. Organizational Behavior and Human Performance, 25, pp. 97-106.
Harasim, L., Hiltz, S.R., Teles, L., & Turoff, M. (1995). Learning networks: A field guide to teaching and learning online. Cambridge, MA: MIT Press.
Harasim, L. Ed. (1990). On-line education: Perspectives on a new medium. New York: Praeger/Greenwood.
Hashim, S., Rathnam, S. & Whiston, A. (1991). CATT: An argumentation based groupware system for enhancing case discussion in business schools. Proceedings of the International Conference on Information Systems (ICIS), pp. 371-385.
Hill, G.W. (1982). Group versus individual performance: Are N+1 heads better than one?. Psychological Bulletin, 91 (3), pp. 517-539.
Hiltz, S.R. (1994). The Virtual Classroom: Learning without limits via computer networks. Norwood, New Jersey: Ablex Publishing Corporation.
Hiltz, S. R., & Wellman, B. (1997). Asynchronous learning networks as a virtual classroom. Communications of the ACM, 40 (9), September, pp. 44-49.
Johansen, R. (1992). An introduction to computer augmented teamwork. In Bostrom, Watson, & Kinney, Eds. Computer augmented teamwork: A guided tour. New York: Van Nostrand Reinhold.
LaRose, R., Gregg, J., & Eastin, M. (1998). Audiographic telecourses for the Web: An experiment. Journal of Computer-Mediated Communication [On-line], 4 (2). Available: http://www.ascusc.org/jcmc/vol4/issue2/larose.html
Leidner, D., & Jarvenpaa, S. (1995). The use of Information Technology to enhance management school education: A theoretical view. MIS Quarterly. September, pp. 265-291.
Leidner, D., & Fuller, M. (1996). Improving student processing and assimilation of conceptual information: GSS-supported collaborative learning vs. individual constructive learning. Proceedings of HICSS 29th. January, pp. 293-302.
Nichols, M., & Day, V. (1982). A comparison of moral reasoning of groups and individuals on defining issues test. Academy of Management Journal, 25(1), pp. 201-208.
Nunamaker, J., Dennis, A., Valacich, J., Vogel, D., & George, J. (1991). Electronic meeting systems to support group work. Communications of the ACM, 34 (7), 41-61.
Ocker, R.; Hiltz, S. R.; Turoff, M., & Fjermestad, J. (1995). The effects of distributed group support and process structuring on software requirements development teams: Results on creativity and quality. Journal of Management Information Systems, 12, 3, Winter.
Pedhazur, E., & Pedhazur, L. (1991). Measurement, design and analysis: An integrated approach. Lawrence Earlbaum Associates, Publishers.
Peek, L.E.; Peek, G.S. & Horras, M. (1994). Enhancing Arthur Andersen business ethics vignettes: Group discussions using cooperative/collaborative learning techniques. Journal of Business Ethics. Vol, 13, pp. 189-196.
Perrone, C., Repenning, A., Spencer, S., & Ambach, J. (1996). Computers in the classroom: Moving from tool to medium. Journal of Computer-Mediated Communication [On-line], 2(3).
Available: http://www.ascusc.org/jcmc/vol2/issue3/perrone.htmlPinsonneault, A., & Kraemer, K. L. (1989). The Impact of technological support on groups: An assessment of the empirical research. Decision Support Systems, 5, 197-216.
Rice, R. (1984). The new media: Communication, research & technology. Beverly Hills: Sage.
Salomon, G., & Globerson, T. (1989). When teams do not function the way the ought to. Journal of Educational Research, 13 (1), pp. 89-100.
Shepperd, J.A. (1993). Productivity loss in performance groups: A motivation analysis. Psychological Bulletin, 113 (1), pp. 67-81.
Silver, M., Markus, L., & Beath, C. (1995). The Information Technology interaction model: A foundation for the MBA core course. MIS Quarterly, pp. 361-390.
Smith, J., & Vanecek, M. (1988). Computer conferencing and task-oriented decision: implications for group decision support. Information and Management, 14, pp. 123-132.
Turoff, M., & Hiltz, S. R. (1982). Computer support for group versus individual decisions. IEEE Transactions on Communications, 30 (1), January, pp. 82-91.
Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, 40 (6), pp. 1282-1309.
Wheeler, B.C., Valacich, J.S., Alavi, M., & Vogel, D. (1995). A framework for technology-mediated inter-institutional telelearning relationships. Journal of Computer-Mediated Communication [On-line], 1 (1). Available: http://www.ascusc.org/jcmc/vol1/issue1/wheeler/essay.html
Wilson, E.V., Morrison, J.P., & Napier, A.M (1997). Perceived effectiveness of computer-mediated communications and face-to-face communications in student software development teams. Journal of Computer Information Systems, Winter, pp. 2-7.
Acknowledgement
This paper is based on the dissertation work conducted by the first author. We are very grateful to Eliezer Fich, Ron Rice and Murray Turoff for their comments and suggestions. Partial funding for this research was provided by a National Science Foundation Grant (NSF-IRI-9408805) and the Alfred P. Sloan Foundation. The opinions expressed here are those of the authors and not necessarily those of the sponsors of this research.
About the Authors
Raquel Benbunan-Fich is currently an Assistant Professor at Seton Hall University Stillman School of Business. During the 1997-98 academic year, she was a Visiting Assistant Professor in the IS Department at the NYU Stern School of Business. She received her Ph.D. (1997) in Management Information Systems from Rutgers University, her MBA (1989) from IESA,Venezuela, and her BS in Computer Engineering Cum Laude (1986) from Universidad Simon Bolivar ,Venezuela. Her publications include journal articles conference papers, and several publications in Spanish. Her current research focuses on Educational Applications of Computer-Mediated Communication Systems, Asynchronous Learning Networks and Evaluation of Web-based Systems.
Address:W. Paul Stillman School of Business, Office KH 614, Seton Hall University, South Orange, NJ 07079Starr Roxanne Hiltz is Distinguished Professor of Computer and Information Science, New Jersey Institute of Technology, where she also directs the Collaborative Systems Laboratory. She has spent most of the last twenty years engaged in research on applications and social impacts of computer technology, publishing widely in journals including JMIS, MISQ, Communications of the ACM, and Management Science. Her research interests currently include Group Support Systems and Asynchronous Learning Networks. In particular, with major funding from the Corporation for Public Broadcasting and the Alfred P. Sloan Foundation, she has created and experimented with a Virtual Classroom [TM] for delivery of college-level courses. This is a teaching and learning environment which is constructed, not of bricks and boards, but of software structures within a computer-mediated communication system.
Address:Computer & Information Science, New Jersey Institute of Technology, University Heights, Newark, NJ 07102
