Should a review be "objective"?

[valondar]

Originally Posted by VivisQueen

Valondar: I like your scale. Very flexible but still ranked in a way that would be helpful. Hmmm... I might switch to something like that at some point. By the way, shouldn't this be going into the 'how do you use your ratings thread? GAH!'

Hm. I guess. I've put it on my profile anyway. As far as the argument about numbers goes, I'm fine with the the way it is now - where reviewers have their own internally consistent scores - but I wouldn't mind standardisation either. I do think the content is the important part of the review.

[therik]

Originally Posted by VivisQueen

Therik: Gundevil does have a point. I'm only guessing this, but I do wonder whether one of the reasons sothis hasn't created a standard scale for the ratings (yet) is because she doesn't want to lower our freedoms regarding reviews. Site reviewers are hard to come by (literally finding you and valondar around the same time was like sothis striking gold). The last thing you want is to put reviewers off with lots of rules etc. However, theoretically speaking that would be the best way to ensure continuity for the site. So we all know that five means average (or whatever she would set it as).

On the other hand, standardised ratings aren't half as important as ensuring your ratings are INTERNALLY consistent i.e. that whenever someone looks at your ratings they know when you say 5 = average, it means average ALL THE TIME for you. My guess is you pluck the ratings out of the air because you've only done a handful. With a scale of 10, that's a lot of space to fit your reviews. After you starts getting into the tens level, my guess is you'll probably start giving it a bit more thought to it. Otherwise you might find your favourite anime being lower than some you thought average just because there wasn't much thought put into it.

Okay, you've got me there, and you certainly have a point. Perhaps I will become more consistent with my numbering as time goes on. However, I don't think I'll ever manage to be consistent to the nearest 0.5, or even 1.0 and if I can't do that, then all of my numbers will have to be taken with a pinch of salt. In turn, that somewhat defeats the point of using them, after they're already superfluous to the review itself.

The fact is, I'd rather write without giving ratings at all, for some of the reasons you've stated. I use numbers because the review format compels me to, but I can't honestly vouch for their accuracy and I'd hate for people to take that number as an absolute indicator of my opinion. It's not like I'd intentionally mislead anyone with the numbers I give out, but I can't categorically promise that the numbers will mean anything.

When it comes down to it, using numbers to rate things is a clear, clear indicator of subjectivity. I don't know if someone's already brought this up. The thread's length seemed a little daunting, so I've not read everything, but here's my take. If I write, for example, that I got on with Kanon's characters, and that I didn't have any problems with Ayu repeating "uguu" over and over again, then that gives my opinion on the matter. What it also does is give the reader a chance to say "hang on, I think I probably would be annoyed by a repetitive catchphrase. Maybe I won't enjoy the characters as much as therik did." If I give the characters an 8.5 and that's all the reader observes, then it's entirely subjective and it's impossible for someone else to make a personal judgment. Numbers give people an idea of the reviewer's subjective opinion. Words give the objective grounds for this. I think that's pretty much the opposite of what gundevil began by saying, so it might be worthy of note.

I guess my point is that - apart from if you're using some "sort by ratings" function - a score out of 10 is completely unnecessary when placed next to the review itself. If something's unnecessary, and it's impossible to be perfectly accurate with it, and it might be misleading, I'm led to wonder why it should be included.

~D

[VivisQueen]

Hehehe. Precisely everything you said. As long as it's not too much effort, try not to pluck ratings out of the air. Give them SOME thought, but other than that, it really doesn't matter. And yes, I mentioned that ratings are subjective on page 16.

[therik]

Originally Posted by VivisQueen

Hehehe. Precisely everything you said. As long as it's not too much effort, try not to pluck ratings out of the air.

I'll do my best

Originally Posted by VivisQueen

Give them SOME thought, but other than that, it really doesn't matter. And yes, I mentioned that ratings are subjective on page 16.

Heheheh

[vivafruit]

I wouldn't mind standardization if everything was standardized to my rating scale. Otherwise, I'm against it. :P In practical terms it's just a huge pain in the ass to change the scores to some other arbitrary scale when my arbitrary scale already does its job perfectly (quickly summarize my opinions).

Also, as a statistician I feel compelled to correct this statement:

Originally Posted by GunDevil

All it all depends on how you view the ratings on the site, as either Nominal (which is Vivis arguement) meaning that numbers are only assigned to abstracts or Interval (which is my argument assuming the site had its own definition of categories) meaning that you can tell which score is better than the other based on a certain scale.

Number ratings are ordinal, and anyone who tells you otherwise is wrong. Vivi was not trying to claim that they are nominal, she was claiming that ordinal scales from two different people are not the same scale. Yes, even if you standardize scores this is true, i.e if you force everyone to define a 7 as "good," what does "good" mean? In physics terms, you can think of different rating systems as having different units. 5 miles != 5 kilometers, and a 7 from me != a 7 from Sothis.

Moreover rating scales are not interval ratings because it's impossible to know what the difference is between two "measurements." If rating was truly an interval scale than the difference between a 6 and a 7 would have to be numerically comparable (if not equal) to the difference between a 9 and a 10, when this clearly doesn't have to be the case. Just look at how most people define their scales: they do stuff like put a 6 as "OK" and a 7 as "good." What is the difference between "OK" and "good," and is it comparable to the difference between "good" and "excellent?"

In the future, it might be possible to actually objectively measure pleasure (perhaps by detecting activity of certain parts of the brain), but for now all we can do is approximate using subjective scales.

Therik: I think the primary benefit of using a numerical scale is that it lets people compare the different anime you've rated. For instance, I can easily find out that, even though you enjoyed both Kanon and Kyouran Kazoku Nikki, you enjoyed Kanon more.

[VivisQueen]

Originally Posted by vivafruit

I wouldn't mind standardization if everything was standardized to my rating scale. Otherwise, I'm against it. :P In practical terms it's just a huge pain in the ass to change the scores to some other arbitrary scale when my arbitrary scale already does its job perfectly (quickly summarize my opinions).

Also, as a statistician I feel compelled to correct this statement:



Number ratings are ordinal, and anyone who tells you otherwise is wrong. Vivi was not trying to claim that they are nominal, she was claiming that ordinal scales from two different people are not the same scale. Yes, even if you standardize scores this is true, i.e if you force everyone to define a 7 as "good," what does "good" mean? In physics terms, you can think of different rating systems as having different units. 5 miles != 5 kilometers, and a 7 from me != a 7 from Sothis.

Moreover rating scales are not interval ratings because it's impossible to know what the difference is between two "measurements." If rating was truly an interval scale than the difference between a 6 and a 7 would have to be numerically comparable (if not equal) to the difference between a 9 and a 10, when this clearly doesn't have to be the case. Just look at how most people define their scales: they do stuff like put a 6 as "OK" and a 7 as "good." What is the difference between "OK" and "good," and is it comparable to the difference between "good" and "excellent?"

In the future, it might be possible to actually objectively measure pleasure (perhaps by detecting activity of certain parts of the brain), but for now all we can do is approximate using subjective scales.

Therik: I think the primary benefit of using a numerical scale is that it lets people compare the different anime you've rated. For instance, I can easily find out that, even though you enjoyed both Kanon and Kyouran Kazoku Nikki, you enjoyed Kanon more.

Was it in statistics school that they taught you to put maths in such a sexy way?

And how did I know you'd respond to this thread?

A gold star for each question correctly answered. Bwahaha!

[Gundevil]

I started the wildfire again! Lol!

[lagore]

I agree with Vivafruit's first post. True objectivity is impossible to obtain, so it is better to embrace your own opinions in a review, rather than lie.

There are many, MANY shows I have seen that I personally hated, but I can understand their artistic merit, and I can see how other people might like them (FLCL is perhaps chief among them). I do not consider it a bad show, I just didn't like it. And I would try to rate it properly, while conveying that I did not personally enjoy it.

[Maniak]

i really think that each review should reflect the author's opinion, so that's why i usually don't care about what others say about anime series, and just watch them myself...if there is one thing i learned, it's that tastes differ

[CrazyAce01]

I've not read the thread but for the first page and this last page; here's my take.

A review cannot be objective because a review is an opinion. An opinion is, by definition, subjective. Therefore, a review is subjective. I would point out that the leading question of this topic doesn't make any sense, but vivafruit used quotations around the word "objective" (like that) which I would assume acknowledges this fact.

Using facts to back up your opinions can lend them actual validity, but then, people interpret things differently. Even with facts used as proof of an opinion, differing paradigms will inevitably generate different opinions from the same facts. With a little work, you can argue anything about anything.

Objectivity in a review is like a scale, with baseless opinions at one end and a summary of the work at the other. The less facts the reviewer references as backup, the more subjective the review is, stating opinion based only on what the reviewer thinks seemingly without evidence. The more facts posted, the more objective the review is as brief synopses flood the body. If everything was completely objective, it would be not a review but a summary.

A good review is somewhere in the middle, neither ranting nor summarizing. Facts should be used appropriately, supporting the opinions which should make up most of the review.

So, should a review be "objective"? Not completely, no, as then it wouldn't be a review. However, a review without objective facts is just a rant. The good review makes good use of the objective, but is not objective itself.