Weird of Oz on the Art of Rating

Weird of Oz on the Art of Rating

siskel-and-ebertAs a film and book reviewer for a number of periodicals and websites over the years, I have often wrestled with the art of rating. To some, the awarding of stars to a particular work might seem a simple matter, but there is a craft to it, and it is one of those tasks that can be as complicated as you care to make it — you can assign a rating on gut instinct, jotting down the first number that pops into your head, or you can (as I often do) vacillate back and forth over whether you should add that extra half star.

It is also one of the most subjective undertakings. It is one thing to decide whether you enjoyed a movie; it is quite another to assign it some value on a fixed scale. First off, you, the reviewer, must decide on what criteria and within what framework you are going to base your ratings. In fact, this varies so dramatically from one reviewer to the next that the best you can hope for is to be as consistent as possible with yourself.

Believe me, there is no set, agreed-upon code among professional critics to which you need worry about conforming; you just need to make sure your readers can understand your reasoning. It is also helpful to communicate your personal tastes and preferences insofar as they influence your assessments, so that readers know where you’re coming from. Here are some other considerations…

I. Weighing the Scales

The first question is which scale to use. This is pretty arbitrary and varies greatly from one publication to the next. Perhaps the most simple was Siskel and Ebert’s “Thumbs Up/Thumbs Down.” But this was a bit too austere, so they added a further designation of “Thumbs Way Up” — else how could you differentiate  a film that was an amusing diversion from a film that blew your mind?

joe bobMost use either a 4-star or 5-star scale — with other signifiers sometimes substituted for stars. A horror site might award gravestones. One online B-movie site awards Godzilla heads. The most unique rating symbol may have been Nathan Shumate’s on his Cold Fusion Video Reviews site. The cold fusion symbol was reserved for those films so jaw-droppingly bad they achieved the mythical “cold fusion” — or, at least, they guarantee an hour or two of unintentionally uproarious entertainment.

Joe Bob Briggs, the “Drive-In Movie Critic,” notoriously adds to his ratings a tally of “Drive-In Totals” (here is an example from his one-and-a-half star [out of four] review of The Devil Inside: “5 Dead Bodies. No Bare Breasts. 3-4 Beasts. Priest-Flinging”). By the way, Shumate and Briggs are two of the most consistently funny film reviewers out there, both of them exercising their wit at the expense of the lowest of low-brow cinema.

I’ve always opted for the 5-star scale, with the additional parsing of half-stars (essentially making it a 10-point scale), but I like being able to make fine distinctions. As a college professor, it drove me nuts to have, for example, a student who just missed a B grade with 79%, while another student barely eked out 70% with some eleventh-hour extra credit, yet their transcripts would record them equally as C students. How I wished the university used plusses and minuses, because clearly one was a C + while the other was a C -, and a 9% spread in quality of work — that’s a pretty significant range!

II. The Stingy Critic versus the Generous Critic

Once you have your rating scale, the next issue is how you are going to deploy those stars. Here, again, we have a strong dose of subjectivity. Sometimes reviewers are stingy with their stars, reserving that fourth or fifth one for only the distinguished few. Others are more generous — but if you leniently hand out five-star ratings to just about any work you enjoy, how do you give special distinction to that rare work that comes along and rocks your world? (Stephen King complained about the trend of movie-ratings inflation in a 2007 op-ed for Entertainment Weekly. More on that next week.)

the name of the windWhat really got me thinking about this was that a few months ago, Patrick Rothfuss (The Kingkiller Chronicle) posted on Goodreads an explanation for how he rates works. Now, it so happens that I’ve posted reviews and ratings of both his novels on Goodreads, so I’m pretty sure he’s seen them (rare is the author who doesn’t read all the reviews of his/her work, even those anonymous ones on Amazon. Oh, maybe Stephen King or Margaret Atwood can’t be bothered to read all of them, but the rest of us — yeah).

I gave his first book, The Name of the Wind, five stars and raved about it. And, like virtually everyone else who read that book, I eagerly anticipated the second installment. The Wise Man’s Fear delivered its share of pleasures, but this time out I had a few minor criticisms; nor was it the groundbreaking work the first was (although since this is the second book of a trilogy, it is somewhat unfairly artificial to weigh the pieces without weighing the whole).

Hence, I gave it 4 stars (according to the Goodreads definition for each rating increment, I “really liked it,” but would not go so far as to say “it was amazing”). This was an instance when I wished Goodreads allowed the parsing of stars, because I might have gone 4.5. But if you give something the highest increment, aren’t you saying there’s nothing better? I try to be fairly stingy with my five-star reviews, to avoid being the critic who cried wolf by declaring any book that tickled his fancy a landmark of the genre!

Reading Rothfuss’s explanation for how he approaches rating, I discovered he is a self-professed generous star-giver. A book really has to fall short in some major way for him to knock a star off. He freely admits he wishes there was a “sixth star” option for him to distinguish the truly seminal works, being that he gives five stars to so many books and graphic novels. It seems that for a work to get two or three stars from him it really has to stink, and I’ve read enough of his reviews to recognize that he reserves one star ratings for moral condemnation — to rebuke a work for racist or sexist overtones.

If I had rated The Wise Man’s Fear according to Rothfuss’s scale, I’d have given it five stars, no question. Hey, I’m not too concerned — I’m sure Rothfuss isn’t hurt that I shorted him a star — but it does serve to illustrate the continuum that exists between the stingy and the generous reviewer.

Next week, I’ll wrap up my thoughts on reviewing with parts III and IV, which will cover critical transparency (why one should be forthcoming about personal taste and biases that might influence one’s judgment) and framework (should one judge Bill and Ted’s Excellent Adventure by the standard of 2001: A Space Odyssey?).

Subscribe
Notify of
guest

3 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Joe H.

Myself, the only “reviewing” I do is on GoodReads for books I read. (I really wish they allowed for half-star increments in their ratings.) Most of my ratings seem to cluster around four stars with some fives and threes and very few lower than that. But it’s not just because I’m a “generous” review, necessarily — it’s because these are books I’m reading for pleasure, so I’m naturally going to try to choose things up front that I’m going to enjoy.

(I also find that, for me, at least, it does change the dynamic in some cases to know that the author may very well read my review.)

[…] recap, in last week’s post I covered the selection of a ratings scale, which can vary from the most simple “thumbs up/thumbs […]

[…] were watering down or diluting the weight of a top rating (a valid point that I addressed in part 1 under the section The Stingy Critic versus the Generous […]


3
0
Would love your thoughts, please comment.x
()
x