Abstract

This study examines students' motives for use and perceptions of the web site http://www.ratemyprofessors.com, one of the main sites that allow students to post anonymous ratings of college professors in the United States and Canada. The contents of comments posted to the web site are examined for relationships to numerical ratings on the site and motivations expressed by students for participating in the rating process. Qualitative (focus group interviews) and quantitative (content analysis) investigations are presented. Students' motives for accessing Ratemyprofessors.com include information seeking, convenience, and interpersonal utility. As regards teacher ratings, the results indicate that students were primarily concerned with issues such as instructor competence and the classroom experience.

Introduction

In the past five years, several teacher-rating sites have emerged on the World Wide Web and their popularity is growing daily. Sites such as http://www.pick-a-prof.com, http://www.professorperformance.com, and—probably the largest and best known—http://www.ratemyprofessors.com give students the opportunity to evaluate their teachers through numerical ratings as well as open-ended comments. In addition to posting their own ratings of their professors, students can read teacher ratings to help in the selection of courses. No longer does a student have to know people who have had a particular professor; information is only a click away.

A primary reason for the popularity of online teacher rating sites is, arguably, students' desire for information about classes and professors. While most schools conduct teacher evaluations, the results of these traditional end-of-course evaluations are generally not made available to the students. John Swapceinski, founder of http://www.ratemyprofessors.com (hereafter, Ratemyprofessors.com), argues that, "students are demanding more information because they see themselves as customers who want the most value for their dollars" (Gilroy, 2003, para. 5). However, not everyone is enamored with such public access to teacher evaluations. Ratemyprofessors.com gets regular threats of legal action as well as complaints about specific evaluations, although all comments and ratings are screened so that offensive or potentially libelous comments can be removed before being posted (Giegerich, 2003).

As of November 22, 2004, there were over 2.5 million ratings posted on the site, representing approximately 450,000 professors and 4,200 schools from across the country (http://www.ratemyprofessors.com statistics). Students rate professors on three dimensions: easiness, helpfulness, and clarity of teaching. The helpfulness and clarity ratings are combined to give an overall quality rating. In addition, students can rate attractiveness by assigning a "chili pepper" to an instructor to denote "hotness," or physical (especially, sexual) attractiveness. Hotness scores for professors are the total number of "chili peppers" each instructor has. In addition to the numerical ratings, students can post a brief descriptive comment about the professor.

While there is a large body of research on traditional methods of students' evaluations of teachers, Ratemyprofessors.com and other similar sites have been mostly limited to reviews and articles in the popular press. As these sites become more popular, however, it becomes necessary for scholars to investigate how and why students are using the sites to rate teachers and what information students value. Unlike traditional course evaluations where the audience is the teacher and perhaps administrators, on Ratemyprofessors.com and other similar sites, the intended audience is other students.

This public, computer-mediated forum presents an anonymous platform on which students can express themselves about their instructors with great freedom, international reach, and almost absolute impunity. In the context of this anonymity, reach, and freedom, the current study investigates how often students comment on professors' physical appearance, race, gender, age, intelligence, competence, and personality in ratings posted to the web site Ratemyprofessors.com (RMP), and how these factors correlate with assigned scores. In addition, the study draws on the uses and gratifications theory of media use in order to understand the perceptions and motivations of students reading and posting to the RMP site.

Literature Review

In this section the uses and gratifications theory, specifically motivations for Internet use, is reviewed to provide general insight about students' potential motivations for using and posting specific comments to the web site http://www.ratemyprofessors.com. Prior research related to student evaluations of teaching is then summarized briefly, including the use of ratings in course selections, perceptions of evaluation systems, and correlates of high and low teacher ratings.

Uses and Gratifications

The uses and gratifications theory describes how audience members use media, and the gratifications they obtain from those uses. Several assumptions underscore the theory (Rubin, 1993): Audience members are active and goal-directed; specific communication messages are selected in order to fulfill various needs (e.g., information seeking or entertainment); social and psychological factors influence media selections; and media are constantly in competition with other types of communication. Uses and gratifications has long been applied to traditional media. More recently, the theory has also been applied to motives for using the Internet, for example: uncovering motives for hosting a personal web page (Papacharissi, 2002), determining the Web as an alternative to television (Ferguson & Perse, 2000), investigating the uses and gratifications sought from accessing political web sites (Kaye & Johnson, 2002), and understanding Internet usage in general (LaRose, Mastro, & Eastin, 2001).

Papacharissi and Rubin (2000) researched uses and gratifications of the Internet in order to provide "greater understanding of the personal and social attributes that affect why people use computer-mediated communication (CMC) and the outcomes of CMC-related behavior" (p. 2). While their study did not focus on specific Internet sites, they identified five primary motives for using the Internet: interpersonal utility, to pass time, information seeking, convenience, and entertainment.

The first motive, interpersonal utility, includes such needs as to participate in and belong to a group, to express oneself freely, to give input, and to find out what others say. The motive "to pass time" is manifest when one is bored or has nothing better to do. Information seeking includes looking for and receiving free information, and the fact that information gathering is easier online than with other media. Convenience as a motive includes communicating with friends and family because it is cheaper than other modes of communication, as well as the convenience of not having to be in the same time and space as others in order to interact. Finally, entertainment motives include using the Internet because it is fun, entertaining, and enjoyable. Students obviously use the RMP site to seek out information about professors, but they may also like the freedom of expression the anonymous format allows. In addition, one can get several opinions at once (convenience) and students may find the site entertaining due to the creative phrasing of some of the comments (as in the title of this article).

Several of the motivations summarized above have been found to be reasons for choosing the Web as a source of political information. Kaye and Johnson (2002) found four motivations: guidance, information seeking, entertainment, and social utility. The authors conclude that "the uses and gratifications approach is well suited for studying the World Wide Web as a whole and for examining specific types of sites within the web" (Kaye & Johnson, p. 1). The uses and gratifications perspective may be particularly relevant with regard to the content and users of Ratemyprofessors.com.

Student Evaluation of Teaching

Numerous studies have been written on the merits and drawbacks of student evaluation of teaching. McKeachie (1990) notes that evaluating teachers has been thoroughly studied since the late 1920s, and concludes that "despite faculty doubts about the ability of students to appreciate good teaching, the research evidence indicates that students are generally good judges - surprisingly so" (Evaluation of teaching section, p. 6). More recently, Hobson and Talbot (2001) reviewed research on the reliability and validity of student evaluations and concluded that "well-developed student evaluations with adequate reliability and validity data may provide some of the best measures of teaching effectiveness" (Recommendation section, p. 4).

Student Course Selection

Past research has explored student course selection (CS), which generally focuses on how students utilize prior student ratings of teaching (SRTs) to aid them in making decisions about which courses and which instructors they will take. Babad, Darley, and Kaplowitz (1999) conclude that while a primary objective of student ratings of instruction (specifically, those that are made publicly available) is to provide information to assist in student course selection, the research is limited and shows that students may rely on and value other sources of information more than empirical student evaluation data. For example, they cite Borgida and Nisbett (1977), who found that "the provision of mean SRT ratings had little effect on subsequent CS, whereas brief and vivid face-to-face comments from another person had a greater impact despite their informational deficiencies" (Babad et al., 1999, p.11).

Babad et al. (1999) argue that the statistical format of SRTs may be an unsuitable way to present information to students; therefore, it may not be used as a primary source when selecting courses. For their content analysis study, they focused on non-statistical data about courses and professors-narrative types of data that more often are being provided to students, including the "informal, juicy, and palatable information based on the experience of their peers" (Babad et al., 1999, p. 8). Although some colleges now make such information publicly available, many more still do not. Thus, professor rating sites like Ratemyprofessors.com serve, in general, the same purpose-to aid students in course selection. Ratemyprofessors.com contains some statistics, but also includes narrative descriptions and visual icons. Research has also shown that students are in favor of publishing ratings of instructors, while professors are not (Howell & Symbaluk, 2001). In the present study, interviews with students are conducted to uncover motives for using Ratemyprofessors.com, what information presented by the site is valued (statistics versus comments), and students' use of the site in conjunction with other sources of information (e.g., face-to-face interactions with peers).

Student Perceptions of Teaching Evaluations

Another area of investigation with regard to student evaluations of teaching is student perceptions of the evaluation process, although some authors have commented that student attitudes have received little research attention (Ahmadi, Helms, & Raiszadeh, 2001; Dommeyer, Baum, & Hanna, 2002). Abbott, Wulff, Nyquist, Ropp and Hess (1990) argue that this line of research is important because "student satisfaction may affect students' willingness to participate, may influence their opinions about teaching effectiveness, or both" (Discussion section, p. 1). Abbott et al. found that students preferred mid-term versus end-of semester evaluations. In addition, students reported being more satisfied with a group interview method of providing teacher evaluations than with the standardized questionnaire format. The authors explain this finding through social comparison theory, arguing that the group interviewing method of gathering data "allows students to compare their opinions about teaching effectiveness with the opinions of their classmates" (Discussion section, p. 5). Ratemyprofessors.com also allows students to compare their ratings of a teacher with those provided by other students.

Regarding students' perceptions of the comment section of the faculty evaluation form, Ahmadi et al. (2001) found that the students only sometimes complete this portion of the form. Their reasons include lack of time, fear of responses not being anonymous, belief that the form is useless, indifference, belief that their comments will have no influence, and the notion that it is only necessary to complete the written section when something about the class or professor is exceptionally good or bad. One interesting finding in this study is that when asked if the results should be made public, over two-thirds said yes; students suggested publishing the information in newspapers and on the Internet. The Ahmadi et al. study provided guidance for some of the discussion questions for the focus group interviews we conducted for the present study.

Factors Related to Student Evaluations of Teaching

Much research has been done on how student ratings of instructors can be influenced by specific factors; however, findings at times appear to be inconsistent. Read, Rama, and Raghunandan (2001) suggested that student evaluations of instructors can be influenced by student characteristics (such as expected course grade), instructor characteristics (such as gender, race, rank, appearance, and personality), characteristics of the course (such as difficulty), and even environmental characteristics (such as size of the classroom). These and other factors may also influence ratings posted on Ratemyprofessors.com.

The communication literature has investigated what communication and relational factors may affect student evaluations of an instructor and a course. Scott and Nussbaum (1981) argued that:

while the instructional environment is unique in many respects, it is also a microcosm of the larger interpersonal communication environment. Therefore variables which influence interactants in the interpersonal communication environment can also be expected to influence interactants in the instructional environment. (pp. 51-52)

Scott and Nussbaum found that perceptions of overall communication competence by instructors (for example, honest self-disclosure and competency in verbal and nonverbal communication) were directly related to students' evaluation of an instructor. Rice, Stewart, and Hujber (2000) found that sociability and interaction with students, among other factors such as organization and preparation, do impact student evaluations, suggesting that "being an effective communicator is an essential component of being an effective instructor" (p. 253). Sociability and interaction with students could be perceived as outgoing or extrovert behavior. Radmacher and Martin's (2001) research support this notion. They found that teachers who are perceived as extroverts receive significantly higher evaluations than those who are not. This study will examine how often posted comments related to these types of communication and personality characteristics are found at Ratemyprofessors.com.

Researchers have also focused on how instructor appearance may influence student evaluations of teaching. For example, recent research has questioned if attractiveness correlates with ratings. Montell (2003) reviewed the results of a study that found that "attractive professors consistently outscore their less comely colleagues by a significant margin on student evaluations of teaching" (Hammermesh & Parker, 2003, cited by Montell, 2003). The findings, they say, "raise serious questions about the use of student evaluations as a valid measure of teaching quality" (Montell, 2003, p. 2). Feeley (2002) found significant relationships among instructor level of attractiveness and vocal clarity and such non-related dimensions as teaching effectiveness, affective learning and nonverbal immediacy, arguing that "professors seen as particularly effective can do little wrong in student-raters' perceptions" (p. 232).

Attractiveness is an important dimension to consider as students posting to Ratemyprofessors.com can rate attractiveness of a professor by assigning a "chili pepper" icon to indicate "hotness." This concept is not defined for users anywhere on the site but is generally understood as (and used in reference to) the physical attractiveness of the instructor. Felton, Mitchell, & Stinson (2003) content analyzed data from Ratemyprofessors.com in order to determine if any relationship exists among perceived quality, easiness, and "hotness" scores. The 25 institutions with the most student evaluations constituted their sample, which totaled 65,678 posts. They found that overall there were significant correlations among the three variables of quality, easiness, and "hotness." Professors who were rated as easy were perceived as sexier and of higher quality than professors rated as not easy. The present study will also probe for any influence of attractiveness on ratings.

In addition to appearance, race and gender have also been shown to have an impact on student ratings. Rao (1995) investigated the reactions of American students to encountering instructors who were "foreign" in appearance and in language. In what Rao termed the "Oh No!" syndrome, students had negative reactions to instructors who were perceived as requiring a greater effort to understand. Gender has also been shown to be a factor in student evaluations, although Ahmadi et al. (2001) argue that the research is mixed and inconclusive. For example, Bachen, McLoughlin, and Garcia (1999) found female teachers were rated higher on several teaching dimensions than male teachers when rated by female students. They concluded that evaluations of teachers are somewhat guided by students' expectations of appropriate gender role behavior. However, other research has shown gender to be not as important in teacher ratings as other factors, such as overall teaching effectiveness (Dukes & Victoria, 1989). In the present study, comments at Ratemyprofessors.com are coded for any mention of race and gender.

Finally, students are also concerned about intelligence and overall competence when rating instructors. Bachen et al. (1999) summarizes research that illustrates how students generally describe ideal professors as being competent in their jobs and knowledgeable in their subject matter. Competence and knowledge, while important to students, are also characterized by some faculty as criteria that students are not qualified to judge (Ahmadi et al., 2001). Related to judgments of competence and intelligence are evaluations of perceived "easy" instructors. In other words, concern for validity of ratings might occur if easy or lenient instructors consistently receive higher ratings than difficult instructors. Difficulty of a course and workload expectations, while arguably important considerations for students, have been shown to have little to no real effect on student evaluations of teaching (Beatty & Zahn, 1990). Posted comments to Ratemyprofessors.com will be examined to see if competence and intelligence of the instructor are mentioned. Comments related to competence and intelligence, along with the other factors described above (personality, appearance, gender, and race) will be correlated to the established ratings of "Easiness," "Helpfulness," and "Clarity" that are present at Ratemyprofessors.com.

Research Questions

Drawing on the uses and gratifications theory, this study aims to elucidate how students perceive the teacher evaluations posted on the Ratemyprofessors.com website, as well as their motivations for accessing and reading the evaluations. We thus posed the following Research Questions:

RQ1a: What are students' perceptions of the numerical and written evaluations at Ratemyprofessors.com?

RQ1b: What motivates students to access the site and read the evaluations?

We were also interested in the contents and motivations of student postings to the site. We therefore posed Research Questions 2a and 2b:

RQ2a: What do students post to the site Ratemyprofessors.com?

RQ2b: What motivates students to post to the site?

In the context of the anonymity provided by the computer-mediated nature of postings to the site, and based on research related to the influence of various factors on student evaluations of teachers, the present study was concerned with the content of the comments that form part of the teacher ratings on Ratemyprofessors.com. We thus posed Research Question 3a as follows:

RQ3a: How often are factors such as personality, appearance, race, age, gender, intelligence, and competence mentioned in ratings posted to the web site Ratemyprofessors.com?

The investigation also sought to determine any relationships between such factors and the ratings given to instructors, and posed Research Question 3b as:

RQ3b: How do these factors correlate with an instructor's easiness, helpfulness, clarity and/or "hotness" scores?

Methodology

In order to examine the content of postings at Ratemyprofessors.com as well as uncover student perceptions and motivations for accessing and posting to the site, we combined focus group interviews with content analysis.

Focus Group Interviews

Four focus group interviews (FGIs) were conducted with students from a small private liberal arts college in the Northeastern United States. This school was chosen because it was the highest posting school per capita on Ratemyprofessors.com (i.e., had the largest number of postings relative to school size) at the time of the data collection; it is also the home institution of the authors. Two pilot focus groups were conducted in June 2003, shortly after the end of the spring semester when students, having finished their semester courses, may have recently posted ratings about their professors. However, it was difficult to obtain participants because most students had returned home for the summer break.

Based on information obtained from the pilot focus groups, two additional focus groups were conducted in late October 2003 immediately after students created their schedules for the upcoming Fall Spring semester. A recent graduate of the college facilitated the June focus groups, and a student research assistant facilitated the October focus groups. All FGIs were videotaped to allow for transcription and analysis.

We recruited the June focus group participants from the population of communication students staying on campus to take summer courses, and offered extra credit for participating. These two groups consisted of only juniors and seniors. Six students participated in the first group, and four students participated in the second. The October focus group participants were also recruited from communication classes, and were awarded attendance points or extra credit for participating. These students were all freshmen or sophomores. Eleven11 students participated in the first group, and 11 participated in the second. All students were familiar with the web site and had experience reading or posting to the site, or both. Each of the four FGIs lasted approximately one hour. The Ratemyprofessors.com website was accessible to the facilitator during the FGIs to use for examples and illustrations.

Two specific topics were addressed in the FGIs: students' reading of the ratings and students' posting of the ratings. For example, regarding students' reading of the ratings, we asked how often students read the ratings and why, as well as how students perceived and how much value they placed on the numerical ratings, the icons (smiley/frowny face, etc.), and the actual posted comments. With regard to posting, we inquired as to reasons for posting, when they post, and what kinds of information they are most likely to post. Focus group data were analyzed using the constant comparative method (Strauss & Corbin, 1998), which involves examining the data for specific categories, and formatting those categories into relevant themes.

Content Analysis

The comments posted to the web site Ratemyprofessors.com were content analyzed in order to determine what factors the students mention in their comments, and whether these factors correlate with the rating categories the site itself measures: easiness, helpfulness, and clarity. We analyzed the assigned scores and content analyzed the comments of a sample of 1,054 ratings from the RateMyProfessors.com web site. We then investigated the relationships between and among the scoring items and the comment themes.

Data Collection and Sampling

Ratemyprofessors.com arranges colleges and universities by state and lists the schools in each state alphabetically. Under each hyperlink to a school, the professors at that school are listed alphabetically and hyperlinked. The hyperlink for each professor leads to the ratings (and associated comments) for that professor. The data in the present sample were collected directly from the Ratemyprofessors.com web site from July to September, 2003.

The sampling process began with identifying the states to be coded through the use of a random number generator against a list of states. We identified the first school to be recorded within a state from the web site's list of schools for that state using a random start from one to ten. We selected other schools from the state using an interval of ten from that random start. For each school, we identified the first professor to be sampled with a random start from one to ten and with an interval of ten thereafter. We continued the sampling process until we had collected more than 1,000 entries in total. After cleaning and editing the data for illegal codings and errors, we ended up with 1,054 cases.

Prior to the final sampling and collection, the authors performed a pilot coding exercise to test the coding and sampling schemes. We examined each other's coding of the comments on thematic lines to ensure common understanding of the coding criteria. Following the pilot exercise, the authors collected data for the sample and coded independently. Each author then reviewed the other's coding to ensure agreement. Overall intercoder reliability using the Holsti (1969) method was .94. We entered ratings, comments, and thematic codes directly into spreadsheets. These records included direct pasting of comments from the web site with spelling and grammatical nuances intact. We then analyzed the data using the Statistical Package for the Social Sciences (SPSS).

Coding

We coded comments on gender, race, age, appearance, intelligence, competence, and personality as positive, negative, or neutral. Under "gender" we coded all comments that made reference to an instructor's gender (excluding common pronouns). These included comments such as "She is a total bitch," and "You will learn lots from this classy lady." We coded comments under "race" if they made reference to an instructor's ethnicity or national origin (e.g., "…something cute about his German accent..."). Comments referring to the instructor's actual or apparent age were coded under "age" (e.g., "…if only he were a few years younger!" and "Time for this professor to retire"). We coded for appearance whenever a comment made reference to the instructor's physical appearance (e.g., "Looks a lot like Jerry Garcia" and "What is up with the uni-brow!"). The category "intelligence" referred to comments regarding an instructor's knowledge about course topics (e.g., "She knows her stuff"). We used the category "competence" for comments dealing with the professor's ability in the classroom (e.g., "…he is so good at explaining such a difficult subject"). The category "personality" covered all comments that made reference to the instructor's personality, communication style, and personal traits (e.g., "She relates to her students as a person and truly cares about them" and "…powerful, focused, energetic and inspiring").

Findings

Focus Group Interviews

Research Question 1a asked: "What are students' perceptions about the numerical and written evaluations at Ratemyprofessors.com?" Overall, students reported that they tend to trust their friends' opinions of professors more so than opinions at Ratemyprofessors.com, but the online ratings are a good supplement. Some participants saw web site evaluations as lacking credibility because of the anonymity of the postings. Knowing the identity of the person making the recommendation was important to these students, thus the reason they trusted and relied on traditional "hearsay" more than Ratemyprofessors.com. One student offered this explanation: "I probably take my friends more seriously because on the web site it might say "oh, he's a jerk" but you don't know if that kid…did bad in class and hates the teacher, so I'd probably rely on my friends more."

The above comment also illustrates how students were wary of trusting the evaluations at Ratemyprofessors.com, but continued to use them as a supplement. In other words, they want as much information about courses and teachers as they can get, even if the source is somewhat questionable. They noted, for example, how conflicting ratings posted to the site diminish credibility. They do, however, appear to be very critical readers of the messages. For example, several students noted that it is easy to tell the "real" comments from the sarcastic or angry ones, so they take the comments many times "with a grain of salt." Students are careful and critical when reading the ratings, because, as one student noted, "you don't exactly know who's writing the responses or why they wrote them."

Students also commented on what information they value more at Ratemyprofessors.com: the statistical and visual elements (smileys, numerical ratings, chili peppers, etc.) or the actual written comments. Not surprisingly, students rely more on the written over the non-written portion of the evaluations. The students basically ignore the numerical ratings, but if they look at any, the easiness score is the one most consulted. The smileys, the students noted, are hard to ignore and are good for creating a first impression of the teacher and the course. For example, if the teacher has many "angry" versus "smiley" faces, this appears to influence their opinion of the teacher initially, but they will investigate further to read the actual written comments. The chili peppers are generally disregarded; students reported that they do not place importance on whether or not a teacher is rated as "sexy." One student summarized this idea by stating, "I think the hot tamale thing kind of takes away from the credibility of the site. If you're looking for a professor, obviously their level of attractiveness isn't really a top priority."

Students read the posted comments with caution. They were confident that they could pick out the ones that are fair and honest and the ones that are vengeful and sarcastic. The consensus was that one must read many comments in order to get valuable information; information gleaned from professors with only one or two ratings was deemed less credible. The kind of information students are looking for in the written comments not only is concerned with the professor, but also information about the class itself (number of tests, readings, assignments, etc.). Another concern of students is what class the poster is writing about. Students noted they become frustrated when reading posts where the specific course is not indicated. Students were aware of the fact that comments about one course may not necessarily apply to another. Finally, students reported that most of the comments they read represent the extremes-either very positive or very negative-and they view this as not a full and accurate reflection of the teacher and the course.

Research Question 1b asked: "What motivates students to access the site and read the evaluations?" Students indicated that their primary motive in reading the professor ratings was to gather information, mainly information that aids in course selection. Students seek out information not only about professors specifically, but also information that might be posted about the class itself (tests, readings, etc.). Students check for this information usually during the time when they are creating their future course schedules. "I just check usually when I'm making my schedule; I'll check it a bunch of times" one student reported. Another commented, "when I was a Fall semester freshman, I didn't know how teachers taught or anything so I checked for how they tested and stuff like that. Then for Spring semester, for making my schedule, just to see how they're rated." The theme of seeking out information during course selection time and/or during add/drop was prevalent.

Students also reported a convenience motive for accessing the site. That is, they like the fact that there may be several posts to read, constituting more opinions than they might get from just asking one or two friends. They also indicated that Ratemyprofessors.com opinions were easier to access than asking several people. For example, student comments included, "I rely on Ratemyprofessor for the variety," "It's all right there on the site," and "It's just a lot easier." This last comment is referring to the fact that it is easier and quicker to check the site and find several ratings, versus asking several friends one at a time.

Additionally, students reported reading the comments at the web site in order to see what others had to say about certain professors. One student reported "During the year, if I really like a professor or if I really don't like a professor, I'll go on and see what other people thought." The fact that students are curious as to what others say about a specific professor illustrates an interpersonal motive for accessing the web site.

Entertainment was not specifically stated as a motive, but the students did discuss how the site itself possesses some entertaining aspects. For example, one student commented on the hot tamales: "The chili peppers are there just for fun. Obviously it's not relevant at all." The motive "to pass time" was not expressed by the FGI participants.

Research Question 2a asked: "What do students post to the site Ratemyprofessors.com?" Most of the students who had posted ratings about professors were participants in the October focus groups. Students reported that it was important to post about the professor, but also to post information about the course itself (e.g., readings, number of tests, etc.). "I like to post around the time that everyone is making up their schedules. I'll go into how their tests are, or attendance policies, just general things about the class." Overall, they post what they themselves would be interested in finding out about a teacher and a course.

The content of what is posted is directly connected to the reasons for posting. Research Question 2b asked: "What motivates students to post to the site?" Although several reasons were given, all of them seem to fulfill the interpersonal utility motive as identified by Papacharissi and Rubin (2000). The interpersonal utility motive includes such needs as to participate in and belong to a group, to express oneself freely, to give input, and to find out what others say. One theme that emerged when discussing posting practices was the notion of posting a comment about a professor only if the students really liked or really disliked the professor. "I only post when I have a really strong opinion of a teacher, either really good or really bad," one student reported. In other words, neutral feelings about a professor did not motivate students to post. Students felt reporting about these specific instances would be most useful for other students. Particularly in the case of "bad" professors, other students could be warned about that teacher and the class itself. This finding confirms what Ahmadi et al. (2001) found; that is, students write specific comments for only exceptionally good or bad professors.

Besides posting to pass along important information to other students, several of the students mentioned revenge or venting. If they had a bad experience in a certain class with a certain professor, posting to the site was their way passing along information, but also of "getting back" at that instructor. For example, one student shared her reason for posting. "I do it so people won't take that professor, but I think it's more my revenge in a way. It's my way of getting back at them." Other reasons noted for posting include a feeling of obligation (if I read the posts, I should contribute to the site by posting) and to balance an instructor's ratings. For example, if a student had a favorable impression of a professor, but many of that professor's ratings were negative, then the student would post a favorable rating in order to offset the negative comments.

In general, then, students want to provide information to others, especially about particularly good or bad professors/courses. They also feel the need to participate; if they read the posts, then they are part of a student community, and therefore they should reciprocate by posting. "I do post for other people, to help them out…I think it's very helpful to other students and if no one was reading it, I wouldn't post," one student reported.

Content Analysis

Research Question 3a asked: "How often are factors such as personality, appearance, race, age, gender, intelligence, and competence noted in ratings posted to the web site Ratemyprofessors.com?" Our sample included 1054 ratings for 626 professors. The total of 626 included 390 male professors, 216 female professors and 20 instances where the gender could not be determined. As detailed in Table 1, 260 of the professors (41.5%) had only one rating, 121 (19.3%) had two ratings up to a maximum of 25 ratings for one professor (0.2%). Male and female professors did not differ significantly on any of the RMP standard rating criteria in the present sample or with regard to the dimensions coded from comments in the sample.

A total of 788 ratings (74.8%) included comments. Comments were coded on seven main content dimensions-personality, appearance, race, age, gender, intelligence, and competence. Mentions of these themes were recorded as positive, negative, or neutral. The presence of comments was not significantly associated with any of the comment content dimensions and only weakly related to one rating item, i.e., average clarity (r[335]=.11, p<.05).

The most common concern among raters was instructor competence. Approximately 42% of the ratings coded in the present sample (437 ratings) contained statements pertaining to the competence of the instructor. Twenty-nine percent of these comments (127) were negative while seventy-one percent (310) were positive. Some 250 ratings (23.7% of all ratings) contained comments pertaining to the personality of the instructor. Of these, 63 (25.2%) were negative while 185 (74%) were positive.

More than six percent of ratings (70 ratings) indicated something about the intelligence or knowledge of the instructor. These statements about the instructor's intelligence were about five times as likely to be positive as negative. Statements pertaining to appearance were present in 3.6 percent of all ratings in the sample and these were more than four times as likely to be positive as negative. A small number of neutral statements on appearance were also recorded.

Race- or nationality-related statements were present in 1.1% of all ratings, and tended to be negative (0.9%) rather than positive (0.2%). Statements on age were present in less than 1% of ratings but were also more likely to be negative than positive. The data set revealed few comments about gender overall, representing less than 1% of all ratings.

The category "class" was added to the coding scheme in response to numerous instances during coding where students commented on the class and class procedures (e.g., numbers of tests) instead of, or in addition to, commenting about the professor. The "class" item was coded only for occurrence. A total of 107 class-related comments were recorded from the 1054 cases in the present sample representing 10.2% of all ratings coded and 13.6% of all cases with comments. Multiple codings were allowed.

A total of 158 professors of the 626 in the sample received a "hotness" total rating ranging from 1 to 10. The mean hotness total among those receiving a hotness rating was 1.7; the modal score was 1 (SD=1.3). The hotness ratings were posted as hotness total scores for the professors and not identified with individual comments (a change from earlier layouts on the same site that showed a hot pepper icon alongside a rating that included a hotness item).

Research Question 3b asked: "How do these factors correlate with an instructor's easiness, helpfulness, clarity, and/or "hotness" scores?" A number of significant relationships emerged from correlation analysis of the three main rating elements at Ratemyprofessors.com (Easiness, Helpfulness, and Clarity) and four of the comment coding dimensions used for the sampled comments (Appearance, Intelligence, Competence, and Personality). The Ratemyprofessors.com rating items all demonstrated strong and significant relationships among each other. As shown in Table 2, the Easiness, Helpfulness, and Clarity items were all positively related to one another with Clarity and Easiness showing the strongest relationship (r[1052]=.86, p<.001).

The four main comment coding dimensions also showed two significant inter-relationships, namely that between the Competence and Intelligence items (r[38]=.54, p<.001) and between the Competence and Personality items (r[124]=.71, p<.001).

Significant relationships among the rating elements and the comment coding dimensions (see Table 2, Appendix A) included a very strong positive association between Competence and Clarity (r[435]=.85, p<.001) and a strong positive association between Personality comments and Helpfulness scores (r[248]=.78, p<.001) as well as weaker but still significant relationships among Personality comments and Easiness scores, Competence comments and Easiness scores, and Personality comments and Clarity scores, among others.

Finally, the data in the present sample demonstrated a strong and significant positive correlation between total hotness scores and the number of ratings posted for a professor with r(156)=.608, p<.01. However, hotness scores did not show any significant relationships with either the main rating elements or the comment themes.

Discussion

Perceptions of Postings and Motivations for Reading Postings

Overall, the main reason for accessing Ratemyprofessors.com is because students want information to aid in course selection; they also desire a variety of opinions and easy access to information. In addition, students are curious about others' opinions. These findings, which illustrate students' motives for reading the ratings, include three as identified by Papacharissi and Rubin (2000): information seeking, convenience, and interpersonal utility.

Students felt that the posted ratings were important for community dispersion of information about teachers and courses, particularly at course selection times. However, study participants also indicated further social functions such as affirmation in their propensity to consult the website to see how others' ratings matched their own perceptions of a professor. The Ratemyprofessors.com website also extends the well-documented process of peer inquiry about teachers among students discussed earlier. Through the website, students feel that they are able to extend the process of inquiry beyond their immediate circle of friends. Yet, students place greater trust in the information provided by their friends directly.

The investigation of comments posted on Ratemyprofessors.com proved to be a useful departure from previous investigations into numerical scores only, as students indicated a strong preference for the guidance provided by written comments over the numerical or graphical components of the ratings. However, the comments themselves are viewed by students with some suspicion and thought to be likely to reflect extreme views, including venting and revenge. This suspicion may also account for the propensity of students to cross-check the ratings with the experiences of their personal friends.

What is Posted

The content of students' comments on the website and the statements recorded in our focus groups demonstrated an overall concern for teacher competence above other considerations such as appearance, race, or gender. Issues such as appearance and personality were less important to students as reflected in the present study. However, appearance and personality were related to ratings and to perceptions of instructors in general. We find confirmation of the primacy of teacher competence in the fact that focus group participants in the present study independently indicated that their posting priorities had to do primarily with the quality of the professors and the content of the courses. We cannot, however, rule out the importance of other factors such as perceptions of a teacher being easy, as we found a strong positive relationship between easiness ratings and overall quality ratings (cf. Felton et al., 2003).

Motivations for Posting

Focus group inputs suggested that students post to Ratemyprofessors.com primarily for interpersonal utility reasons (Papacharissi & Rubin, 2000). They want to provide information to others, and they also feel part of a community of posters. Primarily, students appear motivated to post ratings for teachers who are perceived as being either very good or very bad. This explains why the number of ratings per professor did not show linear correlation with the perceived quality of that person's teaching. The data show that the only significant relationship with regard to the number of posts was that of the "hotness" rating. Professors with higher hotness ratings received more ratings on average. However, while perceived hotness seems to relate to the propensity to post ratings, this factor did not seem to affect the average quality rating as there was no significant relationship between hotness scores and overall quality scores. This suggests that perceived attractiveness of professors is related to students' propensity to post about them, but is not sufficient to influence what is posted.

In addition, it is important to note that approximately 13% of the descriptive comments focused on the course itself (for example, number of tests, required readings, etc.) in addition to or in place of specific attributes of the teacher. Therefore, some students appear to be posting in order to pass along general course information regardless of specific teacher characteristics. This finding was confirmed by our focus group participants who indicated that they post information about the instructor specifically, but also about the course in general.

Additionally, our focus group interviews suggested that students reading postings on Ratemyprofessors.com do so with some awareness of the possible motivations for ratings and comments. Readers are aware that ratings and comments on the site could reflect students exacting revenge or venting. Students confirmed this by stating that they do, in fact, post to vent their frustration at a teacher or course. One aspect of the interpersonal utility motive is the ability to express oneself freely, and venting appears to fulfill this function.

Limitations and Further Study

A number of the results in the present study differ from those in previous research on Ratemyprofessors.com by Felton et al. (2003). Felton et al. found that overall there were significant correlations among the three variables of quality, easiness, and "hotness." Professors who were rated as easy were perceived as sexier and of higher quality than professors rated as not easy. However, the Felton et al. study involved selected groups of ratings (such as those for professors with more than ten ratings, or from certain schools) while the present investigation sampled broadly and randomly. Moreover, the present study focuses on empirical data coded from comments on the web site while the Felton et al. study did not. These differences notwithstanding, some similarities were found such as the fact that the present study confirmed the Felton et al. correlation between easiness and quality ratings.

The size of the present quantitative sample, while useful for initial investigation, may need to be increased for further analysis of the Ratemyprofessors.com site and related sites, especially as the numbers of ratings at such forums increase. In addition, the information gleaned from the thematic coding could be used to further refine the category scheme to test for more specific themes.

The focus groups in the present study, both at the pilot stage and in the final data set, were all conducted at one school. While the school in question had the highest number of ratings per capita in all of Ratemyprofessors.com, the insights gathered are still limited to the ideas of members of one student body. Further investigation might focus on a wider body of students from a number of different schools. Additional and more varied focus group data would be helpful in confirming the motives for accessing and posting to the site found here (information seeking, convenience, interpersonal utility), and in uncovering additional motives.

Finally, college administrators and professors might themselves be candidates for focus groups and or survey investigation. Such investigation could focus on administrators' and professors' perceptions of web sites such as Ratemyprofessors.com and their estimation of the value of such forums compared to traditional teacher evaluation mechanisms.

Conclusion

The present research revealed that student postings on the website Ratemyprofessors.com closely matched student's real life concerns about the quality of instruction in the classroom. While issues such as personality and appearance did enter into the postings, these were secondary motivators compared to more salient issues such as competence, knowledge, clarity, and helpfulness. Students post comments as a way to both receive and share information about instructors. They value the perspective of their peers, but are also critical consumers of the posted comments.

As the impact of new communications technologies continues to be realized and examined, the emergence of public forums such as Ratemyprofessors.com becomes important not only for educators, but for communication scholars and other social scientists as well. Our combination of qualitative and quantitative analysis allowed us to gain insight into the uses and motives behind using Ratemyprofessors.com, thus providing for a greater understanding of these complex processes. Our findings prompt questions that may be answered by scholars in diverse fields in the future.

Acknowledgments

A previous version of this paper was presented at the Annual Conference of the National Communication Association in November 2003 at Miami Beach, FL.

References

Abbott, R. D., Wulff, D. H., Nyquist, J. D., Ropp, V. A., & Hess, C. W. (1990). Satisfaction with processes of collecting student opinions about instruction: The student perspective. Journal of Educational Psychology, 82 (2), 201-206.

Ahmadi, M., Helms, M. M., & Raiszadeh, F. (2001). Business students' perceptions of faculty evaluations. The International Journal of Educational Management, 15 (1), 12-22.

Babad, E., Darley, J. M., & Kaplowitz, H. (1999). Developmental aspects in students' course selection. Journal of Educational Psychology, 91 (1), 157-168.

Bachen, C. M., McLoughlin, M. M., & Garcia, S. S. (1999). Assessing the role of gender in college students' evaluations of faculty. Communication Education, 48 (3), 194-210.

Beatty, M. J., & Zahn, C. J. (1990). Are student ratings of communication instructors due to "easy" grading practices? An analysis of teacher credibility and student reported performance levels. Communication Education, 39 (4), 275-282.

Dommeyer, C. J., Baum, P., & Hanna, R. W. (2002). College students' attitudes toward methods of collecting teaching evaluations: In-class versus on-line. Journal of Education for Business, 78 (1), 11-15.

Dukes, R. L., & Victoria, G. (1989). The effects of gender, status and effective teaching on the evaluation of college instruction. Teaching Sociology, 17 (4), 447-457.

Feeley, T. H. (2002). Evidence of halo effects in student evaluations of instruction. Communication Education, 51 (3), 225-236.

Felton, J, Mitchell, J., & Stinson, M. (2003, June). Web-based student evaluations of professors: The relations between perceived quality, easiness and "hotness." Social Science Research Network Electronic Library. Retrieved September 17, 2003, from http://ssrn.com/abstract=426763

Ferguson, D. A., & Perse, E. M. (2000). The world wide web as a functional alternative to television. Journal of Broadcasting and Electronic Media, 44 (2), 155-174.

Giegerich, S. (2003, Mar. 30). How does your teacher rate? Students turn to Internet to find out the good, the bad and the ugly. The Washington Post, p. A14.

Gilroy, M. (2003, Sept. 8). Rate-a-prof systems: Here to stay; venting, or valuable consumer feedback? The Hispanic Outlook in Higher Education, 13 (24), 10-12.

Hobson, S. M., & Talbot, D. M. (2001). Understanding student evaluations. College Teaching, 49 (1), 26-31.

Holsti, O. R. (1969). Content Analysis for the Social Sciences and Humanities. Reading, MA: Addison-Wesley.

Howell, A. J., & Symbaluk, D. G. (2001). Published student ratings of instruction: Revealing and reconciling the views of students and faculty. Journal of Educational Psychology, 93 (4), 790-796.

Kaye, B. K., & Johnson, T. J. (2002). Online and in the know: Uses and gratifications of the web for political information. Journal of Broadcasting and Electronic Media, 46 (1), 54-73.

LaRose, R., Mastro, D., & Eastin, M. S. (2001). Understanding Internet usage. Social Science Computer Review, 19 (4), 395-413.

McKeachie, W. J. (1990). Research on college teaching: The historical background. Journal of Educational Psychology, 82 (2), 189-200.

Montell, G. (2003, Oct. 15). Do good looks equal good evaluations? The Chronicle of Higher Education. Retrieved October 20, 2003 from http://chronicle.com/jobs/2003/10/2003101501c.htm

Papacharissi, Z. (2002). The self online: The utility of personal home pages. Journal of Broadcasting and Electronic Media, 46 (3), 346-370.

Papacharissi, Z. & Rubin, A. M. (2000). Predictors of Internet use. Journal of Broadcasting and Electronic Media, 44 (2), 175-196.

Radmacher, S. A., & Martin, D. J. (2001). Identifying significant predictors of student evaluations of faculty through hierarchical regression analysis. The Journal of Psychology, 135 (3), 259-268.

Rao, N. (1995). "The Oh No! Syndrome: A Language Expectation Model of Undergraduate Negative Reactions Toward Foreign Teaching Assistants." Paper presented as the 79th Annual Meeting of the International Communication Association, Albuquerque, NM, May 25-29, 1995.

http://www.ratemyprofessors.com statistics (2003). Retrieved November 22, 2004, from http://www.ratemyprofessors.com

Read, W. J., Rama, D. V., & Raghunandan, R. K. (2001). The relationship between student evaluations of teaching and faculty evaluations. Journal of Education for Business, 76 (4), 189-192.

Rice, R. E., Stewart, L. P., & Hujber, M. (2000). Extending the domain of instructional effectiveness in student evaluations of communication courses. Communication Education, 49 (3), 253-266.

Rubin, A. M. (1993). Audience activity and media use. Communication Monographs, 60 (1), 98-105.

Scott, M. D., & Nussbaum, J. F. (1981). Student perceptions of instructor communication behaviors and their relationship to student evaluation. Communication Education, 30 (1), 44-53.

Strauss, A. L., & Corbin, J. M. (1998). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Thousand Oaks, CA: Sage.

About the Authors

Jeannette Kindred is Assistant Professor of Communication at Marist College in Poughkeepsie, NY. Her research interests include qualitative investigations of online relationships and the use of computer-mediated communication in education.
Address: LT 216, Marist College, Poughkeepsie, NY 12601 USA

Shaheed N. Mohammed is Assistant Professor in the Multimedia Department at Marist College in Poughkeepsie, NY. His research interests include content analysis of the World Wide Web and the social impact of new communication technologies.
Address: LT 215, Marist College, Poughkeepsie, NY 12601 USA