Abstract

Students are often used in research as research subjects or to validate/pilot questionnaires. It is known that response rates to requests to participate in research projects vary as a function of a number of factors. This research brief examines the effect of the communication medium on response rate by comparing an oral request for participation with an email request. Email and oral communication, specifically public oral communication, are the two easiest and presumably most common approaches faculty members have to access students to request their participation in research. Results show that an impersonal email to a mailing list is the worst way researchers can approach students to request participation, with there being no difference between making the request by personalized email or orally.

Background

Students are often used in research as research subjects or to validate/pilot questionnaires. It is known that response rates to requests to participate in research projects vary as a function of a number of factors such as: a letter sent to forewarn that the request is to be made (Brunner & Carroll, 1969), incentives such as money (Singer, Van Hoewyk, Gebler, Raghunathan, & McGonagle, 1999), the questionnaire length, design and the number of difficult questions (Dillman, Sinclair, & Clark, 1993), how interesting the topic is to the respondent (Groves, Singer, & Corning, 2000), and the use of follow up methods (Roscoe, Lang, & Sheth, 1975). Persuasiveness has been found to be dependent on a large number of factors including the vividness of the message (Frey & Eagly, 1993), the attractiveness of the speaker (DeBono & Harnish, 1988) and the mood of the recipient (Raghunathan & Trope, 2002).

This research brief examines the effect of the communication medium on response rate by comparing an oral request for participation with an email request. Numerous studies have compared one communication channel with another (e.g., Lana, 1963; McGinnies, 1965). However, there is little point in comparing two channels without considering how appropriate they are to communicators, in this case faculty members, to meet their specific goals (Rosnow & Robinson, 1967). The choice of email and oral communication, specifically public oral communication, is therefore not arbitrary. These are the two easiest and presumably most common approaches faculty members have to access students to request their participation in research—by asking students in a lecture or tutorial, or by using a class email list. (Asking students in a tutorial to participate is a public oral request as opposed to asking one student after class, which would be private oral.) Both of these techniques are more convenient than making the request by some other means, for example, telephone or letter.

This work offers a straight comparison of the effectiveness of the two techniques that will be of interest to researchers who use students in their studies.

Oral versus Email

It is difficult to predict whether an email communication will have a better or worse response than an oral one. Previous research into response rates has provided mixed and unexpected results. For instance, Yammarino, Skinner, and Childers (1991), in a meta-analysis of 115 studies, found preliminary notification and follow-ups, inclusion of a return envelope with postage, and monetary incentives, were effective in increasing mail survey response rates. However Tuten, Bosnjak, and Bandilla (2000) found that response rate was lower when respondents had a chance to win a prize than when it was pointed out that they were being given the chance to contribute to science. Conventional wisdom says that in surveys, personal questions should be asked last (see for instance: http://www.deakin.edu.au/buslaw/bowater/research/pdf/questionnaire_design.pdf), but Frick, Bächtinger, and Reips (1999) found that when they were asked first in web-based surveys, fewer drop-outs occurred.

When a request is made orally in a lecture hall a number of factors that have a potential impact on the decision to participate are on display that are less visible in an email. The recipients will all be aware of the source's age, sex, and attractiveness. In addition, the source will be close to the recipients. Leverage-saliency theory (Groves et al., 2000) states that people vary in the importance they assign to characteristics of requests such as these, and that this importance influences whether they agree to participate. Previous studies suggest that a recipient relies more on these characteristics to make a decision when they cannot understand the message (Chaikin, 1980; Petty & Cacioppo, 1981), or when the characteristics are more vivid (Pallak, 1983).

Moon (1999) studied the impact of two of these characteristics on persuasion via computer-mediated communication: the physical distance between source and recipient, and response latency (RL) (how long it takes the source to respond to any messages the recipients may send). She found a relationship between closeness and persuasion: Persuasion was greater the nearer the source was to the recipient. A relationship was also found with RL: Medium RL led to increased persuasion compared with long and short RLs.

It is difficult to generalize from these findings to the real world communication of the oral request. It may be expected that an oral request will have a better impact than an email one as the source is close to the recipients. However, any response to questions about the request would be immediate (short RL) and might negate this. Closeness may have an impact because it affects attraction (Pallak, 1983) and likeability (Chaiken, 1980). Therefore if the source is likable and attractive, the oral request may be expected to do better.

Latane (1981) proposes a theory of social impact that specifies the effect one person has on another. The theory states that the target of a request will be affected by the number of people making the request, the immediacy of the request, and the strength of the request. The theory proposes that the more people who are present making the request, the more likely the target is to agree to participate. When an email is sent, even if the sender's address indicates that it comes from a group or an organization, it is likely that most people will assume that only one person has written it. This might mean that, in terms of the number of people making the request, there would be no perceived difference between an email and oral communication. The immediacy of the request refers to how close the source is to the target. The oral communication will naturally will be more immediate than an email communication. This suggests that, all else being equal, the oral channel will have more success than the electronic one. The strength of the request refers to the importance of the source to the target and encompasses factors such as age, sex, status, and power. So a head of school might expect to get a higher response than a junior member of staff. Some of these indications of strength will naturally be more visible in the oral communication than in the email which suggests that if the source has high strength, oral will be more successful than email communication.

This leads to the first hypothesis. Given the mixed results of previous research, it has been left directionless:

H1: There will be a difference in the responses received when a request for participation is made orally and when it is made by email.

If this is correct, it will be of interest to examine the email and oral conditions more closely. Latane's (1981) theory also states that the source's request will be diminished according to the size of the target audience. It might be expected, therefore, that a personal email to each individual in the group will have a higher response rate than one email sent to a mailing list:

H2: Emails sent to individuals will get a higher response than emails sent to a group.

Experiments on conformity such as that conducted by Asch (1955) show that the responses of others in a group have an impact on the response of an individual. It is possible that in a lecture/tutorial, the more people who agree to participate, the more likely an individual will be to participate:

H3: Responses to an oral request will be greater when potential respondents perceive that a large number of their peers have already agreed to take part.

Method

282 final year business school undergraduate students taking a strategic management module were used as the sample. The author was not a tutor on this module and did not teach this group of students on any other module. Each student was requested to participate in a research project. The dependent variable was their response, yes or no. For the purposes of teaching, the class was split into 18 tutorial groups. All the students in each tutorial group were randomly assigned to one of four conditions. Those assigned to Condition 1 were sent an individual, personal email with the request shown below. The email began with 'Dear' followed by the student's name. Those in Condition 2 were put into a mailing list which was used to send one, impersonal email with the same request. This email started 'Dear all'. Students in Conditions 1 and 2 were asked to indicate their agreement to participate by emailing back and saying so.

Under Conditions 3 and 4, an oral request was made to the tutorial class. To ensure that all conditions received the same message, exactly the same request was spoken out loud in the tutorials using exactly the same wording as the email. After the request was spoken, if there were any questions, they were either answered by referring to the text that had just been read out, or they were told that answering the question at this time would interfere with the research. Under Condition 3, a blank sheet of paper was passed around the class immediately after the request was made. People willing to participate were asked to write their names on it. Under Condition 4, the sheet of paper already had a large number of names signed up to participate.

It should be noted that assigning tutorial groups to each condition does not give a truly random sample for each condition, as at the start of the module, students were free to sign up for any tutorial slot they wanted as long as it was not already full (the maximum size was 25). This meant that the tutorials might contain groups of friends and people who signed up because the tutorial time was convenient for them. There was no way of preventing this.

The wording of the request was as follows: "I would like to request your participation in a research project that will be happening over the next few weeks. If you agree to participate, all that you would have to do is read some text and then fill in a questionnaire about what you have read. It will only take about ten minutes and you can do it at a time convenient to yourself. All I'd like from you at this stage is to indicate whether you would be willing to participate by [INSTRUCTIONS DEPEND ON THE CONDITION] and then I will contact you nearer the time."

Results

The results are analyzed using a chi-squared test for independence with phi used as a measure of association. The results of testing H1 are shown in Tables 1 and 2. Table 1 is the cross tabs of comparing the oral request with the email request. The results reveal a significant difference between the two media, with the oral request out-performing the email request. However, the email to group condition achieved a very low response rate (less than 1%) and this alone may be contributing to the significance of the relationship. To test this, the analysis was repeated comparing the oral request with just the email to individuals request. The results are shown in Table 2. This time, no significant difference was found.

The results of testing H2 are shown in Table 3. The email condition is examined to determine the impact of emails sent to individuals versus one sent to a group. The results show emails to individuals are much more successful at eliciting a positive response than an email to a group.

Table 4 shows the testing of H3, examining the oral request. The results do not show a statistically significant difference between the responses of people who were given a blank sheet to sign and those who were given a sheet already full of names to sign. .

Conclusion

Support was found for H1 and H2 but not H3. There is a significant difference between the email and oral request, but only because of the poor performance of the email to the group. Emails to individuals are significantly better at eliciting a positive response than one email to a group. There is no statistically significant difference between oral requests when potential respondents perceive a large number of people have already agreed to take part and when they do not. However, as seen in Table 4, the number of yes responses for the full name sheet were much higher than expected and the percentage of yes responses for the full name sheet is nearly twice that for the blank name sheet. Therefore, research to examine this result further would be useful. Overall, the findings show that to achieve the highest response, researchers should avoid asking students to participate in research by sending one impersonal email to the entire group. There is no difference in response if the request is made by personalized email, or orally in tutorials, so this choice may be made based on convenience.

References

Asch, S. E. (1955). Opinions and social pressure. Scientific American, 193 (5), 31-35.

Brunner, G. A., & Carroll, S. J. (1969). The effect of prior notification on the refusal rate in fixed address surveys. Journal of Advertising Research, 9 (1), 42-44.

Chaikin, S. (1980). Heuristic versus systematic information processing and the use of source versus message cues in persuasion. Journal of Personality and Social Psychology, 39, 752-766.

DeBono, K. G., & Harnish, R. J. (1988). Source expertise, source attractiveness, and the processing of persuasive information: a functional approach. Journal of Personality & Social Psychology, 55 (4), 541-546.

Dillman, D. A., Sinclair, M. D., & Clark, J. R. (1993). Effects of questionnaire length, respondent-friendly design and a difficult question on response rates for occupant-addressed census mail surveys. Public Opinion Quarterly, 57 (3), 289-304.

Frey, K. P., & Eagly, A. H. (1993). Vividness can undermine the persuasiveness of messages. Journal of Personality and Social Psychology, 65 (1), 32-44.

Frick, A., Bächtinger, M. T., & Reips, U. D. (1999). Financial incentives, personal information and drop-out rate in online studies. In U. D. Reips, B. Batinic, W. Bandilla, M. Bosnjak, L. Gräf, K. Moser, & A.Werner, (Eds.), Current Internet Science - Trends, Techniques, Results. [Aktuelle Online-Forschung - Trends, Techniken, Ergebnisse.] Zürich: Online Press. Retrieved April 10, 2006 from http://dgof.de/tband99/

Groves, R. M., Singer, E., & Corning, A. (2000). Leverage-Saliency theory of survey participation. Public Opinion Quarterly, 64 (3), 299-308.

Lana, R. E. (1963). Interest, media, and order effects in persuasive communications. Journal of Psychology, 56 (1), 9-13.

Latane, B. (1981). The psychology of social impact. American Psychologist, 36 (4), 343-356.

McGinnies, A. (1965). A cross-cultural comparison of printed communication versus spoken communication is persuasion. Journal of Psychology, 60 (1), 1-8.

Moon, Y. (1999). The effects of physical distance and response latency on persuasion in computer-mediated communication and human-computer communication. Journal of Experimental Psychology: Applied, 5 (4), 379-392.

Pallak, S. R. (1983). Salience of a communicator's physical attractiveness and persuasion: A heuristic versus systematic processing interpretation. Social Cognition, 2 (2), 158-170.

Petty, R. E., & Cacioppo, J. T. (1981). Attitudes and Persuasion: Classic and Contemporary Approaches. Dubuque, IA: William C. Brown.

Raghunathan, R., & Trope, Y. (2002). Walking the tightrope between feeling good and being accurate: Mood as a resource in processing persuasive messages. Journal of Personality & Social Psychology, 83 (3), 510-525.

Roscoe, A. M., Lang, D., & Sheth, J. N. (1975). Follow-up methods, questionnaire length, and market differences in mail surveys. Journal of Marketing, 39 (2), 20-27.

Rosnow, R. L., & Robinson, E. J. (Eds.). (1967). Experiments in Persuasion. New York: Academic Press.

Singer, E., Van Hoewyk, J., Gebler, N., Raghunathan, T., & McGonagle, K. (1999). The effect of incentives on response rates in interviewer-mediated surveys. Journal of Official Statistics, 15 (2), 217-230.

Tuten, T. L., Bosnjak, M., & Bandilla, W. (2000). Banner-advertised web-surveys. Marketing Research, 11 (4), 17-21.

Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991). Understanding mail survey response behaviour: a meta-analysis. Public Opinion Quarterly, 55 (4), 613-639.

About the Author

Thomas Chesney is a lecturer in information systems at Nottingham University Business School. He holds a PhD in information systems from Brunel University, an MSc in informatics from Edinburgh University, and a BSc in information management from the Queen's University of Belfast. His research interests are adoption of recreational information systems, use of blogs to share knowledge, and determinants of research participation decisions. For more information, see: http://www.nottingham.ac.uk/~liztc/Personal/index.html
Address: Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, U.K.