[TO DO] Anirec - Average Rating (trusted users)

[JAhU]

I get the felling that I'm one of those people who rate everything 4 or 5 (ok, my scores are usually from 2.5 to 5, but I have a number of anime rated 4 and 5). I honestly like most of the stuff I watch, but if I don't like a series, I don't force myself to watch it (and thus I have only a few titles rated below 2.5).

I have a dumb question : What does the rating system represent? The quality of an anime or how much one likes it (thats not necessarily the same thing)?

I'm not really concerned about the average rating system, but I hope users' ratings will be displayed on those "Users who watched this anime" pages, as it is now.

[cassiesheepgirl]

Originally Posted by JAhU

I get the felling that I'm one of those people who rate everything 4 or 5 (ok, my scores are usually from 2.5 to 5, but I have a number of anime rated 4 and 5). I honestly like most of the stuff I watch, but if I don't like a series, I don't force myself to watch it (and thus I have only a few titles rated below 2.5).

Same here really. I like the majority of the series that I watch so most of my ratings are 3 - 5, and with the ratings its a very personal thing, not only are peoples personal tastes different but people's ratings mean different thing. for me a 5 means oh my god I LOVE LOVE LOVE this series, 4.5 means i really really like it, 4 means like a lot, 3.5 means like, 3 means ok, kind of average and 2.5 and below are generally meh not too fussed with. I tend to know what sort of stuff im definately not going to like so I don't watch it.

I'd like to think I rate well but I know I have a few series that have decent ratings off me because they made me laugh despite being technically rubbish (damn uni for making me analyze stuff when I watch it). In fact I look at my top two anime, Mushishi is technically AMAZING and gorgeous, one of the best anime I've ever watched, but the only reason it didn't make my #1 spot was because I have more of an emotional attachment to Kyo Kara Maoh, it was the first series I loved, I really got into the story and cared about teh characters, despite it not technically being amazing.

I think this'll be a great idea, because it will have the personal touch, and yes selecting the people whose ratings will contribute to the average rating is important so you don't get people skewing the stats just because they can. Great way of seeing at a glance what people think of a specific anime. I don't think it's too unreasonable to select the people whose ratings will count, it's no different to selecting people to write site reviews.

I think I'm going to stop now because I think I've lost my train of thought...

Have a pickle.

[Teddy]

Just a thought , maybe it would be better to make categories for the trusted users..like for example romance anime, u make a group that likes romance animes and only their vote counts since they watch alot of the genre , they could tell if its good or bad , same for other categories. Dunno just a tough , i for one don't really look at the other user's ratings and i watch them regardless if it has a good or bad rating , happend a lot to find underrated and overrated animes , im well avare that i, too have overrated some of them
So to make it short, i like the ideea of trusted user ratings cuz when evrybody gets to rate it just gets crazy and those ratings aren't to be trusted. Hope nobody gets upset but those are...my thoughts.

[Betelgeuse]

How about making an special statistics page for each anime showing all/much of the distributions of the votes?
Showing them (also) for different categories:
the 'jury' and everyone, and also these distributed over: watched, want to watch, watching, stalled, dropped and maybe also won't watch. Watching, stalled and dropped could also be subdivided in how much (relatively) has been watched.

By showing the distributions people should be able to see the effect on the distribution of black-white thinking and of the fact that people like the anime they watch and then they can compensate if they want to compare the anime with others. (Have you ever seen & compared the distribution diagrams on Anime News Network?)

[alexander]

Originally Posted by sothis

this is how statistics work too.. when you hear statistics on the news, on the web, anywhere - they aren't polling every person in the country. they use a target group of a few hundred or maybe a few thousand people, and assume that those people represent a wide sample of the population.

You might have misunderstood that. Statistics are representative when samples were either completely random or chosen in proportion to the whole population (i.e. when half the country is female, half your sample has to be). 'Trusted Users' would not be a valid approximation of all site users.

But we're not doing science here, and I'm all for filtering out people who don't use the rating system as intended.

i would argue if we do the "real score" it will be a popularity contest, and filled with terrible ratings by people who don't fairly give things a score of 1-5

Did you try it? I mean manually getting a few averages out of the current database, to see how they look. Most series seem to be rated by about a hundred people, it would be hard for any individual to skew that. The site could be made to only display an average when a certain number of ratings are in.

Originally Posted by JAhU

I have a dumb question : What does the rating system represent? The quality of an anime or how much one likes it (thats not necessarily the same thing)?

There's a whole thread on How do you use your ratings?

I rate personal enjoyment, because the numbers are primarily for me. If other people find them useful, that's a bonus.

It's hard to rate quality. A compromise might be two ratings: personal enjoyment and general suggestion. You could then give a series personal 5 stars, because you like stupid girls with huge...guns, but only suggest it for 3 stars, because you know it's a niche and not that well drawn.

Originally Posted by Hannigaholic

Just a thought but you could always include both sets of scores. Show both "average score (all users)" and "average score (trusted users)".

Bad idea. What would be the benefit to the user? Having two different numbers wouldn't induce trust in either.

Originally Posted by Betelgeuse

By showing the distributions people should be able to see the effect on the distribution of black-white thinking

Yes! AP already has a coloured bar showing status distribution. I'd use the same UI element to display ratings, and put it directly below.

(Have you ever seen & compared the distribution diagrams on Anime News Network?)

The standard version is nice, the details are useless: click on it and you're swamped in decimal places and mathematical terms.

[Johnnyflickan]

I think it is a great idea to make a trusted users rating, but then it would be positive if you could see how many trusted users there are(not WHO they are, just how many) so you can imagine how average the rating is.

[Worosei]

Im probablly really really slow in noticing this,
but I've just realised that when you view an anime entry in a-p, and click on one of the user selections (such as 'watched', 'watching..."),
it comes up with a list of users and what they've rated that series as.

In that way you can see how well rated a series is. And see if the people who rated a series highly rated anime you've rated high as well...

If the list of users can somehow be sorted in more ways (such as by sort by ranking, or sort by users with the highest number of recommendations submitted, or by number of series that person has seen), then 'popularity' of a series can be measured using one's own criterias rather than a single 'percentage' that is skewed...

Lol, sorry for bringing that up, i just didnt realise i could see the users who've watched a series and how they rated it :p

[alexander]

If you prefer custom criteria, how about a rating average that isn't based on all or somehow trusted users, but on your friends list. And if that's too few people (with many ratings) there is a setting "nth degree" to include friends of your friends.

Personally I like the trusted thing better, but now I think that friends should definetly be highlighted (different row colour) in the user lists of series.

[sloggerK]

I am assuming ratings and criteria will be on a mathematical basis.
For example, to include a person's ratings in the group average, they'll have to have rated above a certain # of shows, the average rating will have to be below a 4.25 or some numbers, and the standard deviation of the ratings given (or some other mathematical quantifier of variability) will have to be above a certain value to insure that they are giving a decently valid set of ratings?

If so I would like to see this feature work with color codes:
The average rating could be expressed in decimal value from 1-5 like the number of stars and given a color that matched the variability of its ratings. IE if cowboy bebop averages 4.2 stars, and the indicator of variability (std deviation) among the pool of valid ratings is low (meaning most reviewers give at about the same rating), its rating would be green whereas a show that people don't agree about and has ratings all over the place in the rating pool (has a high indicator of variability) could be displayed in red while shows on the scale inbetween could be some shade of orange. That way the average ratings feature could indicate not only the average rating, but how much of a consensus that rating is.

If you wanted to combine this with the people who previously suggested genre fan rating subgroups, you could let people who enter the ratings people select "genre fan" preferences that identify them as particular fans of certain subgenres. You could then not only see the overall rating and color variability, below the overall rating the site could display "by category" and then have the same rating + color code and you could see that say self identified shonen fans give a particular series higher marks with less variability, while others give it much lower marks overall. You could even have a system where users do not have to select what categories they like, but put in place automated criteria using tags. For example, if the user's average rating of anime tagged "shonen" is 4+, or maybe if 40%+ of animes they've watched ar tagged shonen, then they become part of the "shonen" group/category sub-rating automatically.

[Spunky]

After reading this thread in the hopes it wouldn't be as retarded as I initially thought, I understand your point of view but still feel that ratings should be more open to people. Look at my anime list. I can justify the ratings for all of them perfectly. They aren't perfectly spread out, but that doesn't make a bit of difference. It's how I feel about them. People will rate based on their feelings toward a series. You don't have to be good at explaining those feelings, because you shouldn't have to. You know how you feel, and that should be good enough.

Now, granted, if someone goes through and rates every single anime a 1, that person is simply being a dick and his/her ratings should become void. But I see nothing wrong with rating a series (or all of the series you've watched) a 5 if you enjoyed it. That's how you feel about it. Just because you rate every series you've seen as a 5 and some as a 1 doesn't make your rating any less valid than anyone elses. I don't consider this abuse. Rating systems are implemented to let users express their feelings toward a series. You shouldn't dictate who gets to rate based on your own biased feelings toward the rating process itself.

I hope you understand. Really I do.