www.Tobold.com
Tobold@GMail.com
A blog with my thoughts regarding games I am playing and other stuff in life. Please read my Terms of Service
RSS Feed
Thursday, July 19, 2012
Metacritic
IGN had an interesting article this week, where they accused Metacritic of being the reason for boring games and endless sequels. Well, not Metacritic itself, but game studios in which the developers are paid according to the Metacritic score their games achieve. Which is a rather stupid measure, because it only loosely relates to sales numbers, which is what game developers' bonuses *should* depend on. As IGN says, if you target a high Metacritic score, you have to play it safe, as anything controversial can quickly result in a few negative reviews that mar your score.
The good news is that this sort of playing it safe only applies to the big budget games, and that indie games have gotten a lot better over the last couple of decades. With distribution platforms like Steam you can now actually find them, or buy them in cheap and cheerful bundles if you want to be surprised.
Metacritic also has another problem: Either fan groups or protests manipulating user review scores. For example Diablo III has a critic score of 88, and a user review score of 38, with part of that downgrade being due to protests against always-online DRM or the first days of server problems. World of Warcraft has only 883 user reviews, which is hardly representative, and then ends up with a lower score than TERA. The same game can also have very different scores, e.g. ranging from 46 to 80, if it is listed multiple times on different platforms. On the critic review side of things, some games end up with improbably high scores because they only got reviews from a handful of niche game reviewers, but presumably wouldn't do as well with a larger audience.
While the scores and averages on Metacritic just plain suck, I like the site for the links to the professional reviews, and the user reviews I can read on the site itself. While the numbers don't tell you anything, the specific points about which the reviewers talk tell a lot. If the main complaint about a game is its DRM, or that it is turn-based, I can conclude that maybe the game isn't as bad as the score suggests. If the main positive point is how you can gank your fellow players, it tells me that this isn't the game for me.
The good news is that this sort of playing it safe only applies to the big budget games, and that indie games have gotten a lot better over the last couple of decades. With distribution platforms like Steam you can now actually find them, or buy them in cheap and cheerful bundles if you want to be surprised.
Metacritic also has another problem: Either fan groups or protests manipulating user review scores. For example Diablo III has a critic score of 88, and a user review score of 38, with part of that downgrade being due to protests against always-online DRM or the first days of server problems. World of Warcraft has only 883 user reviews, which is hardly representative, and then ends up with a lower score than TERA. The same game can also have very different scores, e.g. ranging from 46 to 80, if it is listed multiple times on different platforms. On the critic review side of things, some games end up with improbably high scores because they only got reviews from a handful of niche game reviewers, but presumably wouldn't do as well with a larger audience.
While the scores and averages on Metacritic just plain suck, I like the site for the links to the professional reviews, and the user reviews I can read on the site itself. While the numbers don't tell you anything, the specific points about which the reviewers talk tell a lot. If the main complaint about a game is its DRM, or that it is turn-based, I can conclude that maybe the game isn't as bad as the score suggests. If the main positive point is how you can gank your fellow players, it tells me that this isn't the game for me.
- posted by Tobold @ 2:28 PM Permanent Link
Comments:
<< Home
Newer› ‹Older
It is like accusing IMDB of misrepresenting movies. Older movies or movies with a huge fan base (like LOTR), get sky-high ratings and newer releases almost never score higher than 8.
It is up to the discerning user to read user reviews and professional critics before making up his/her mind. And if you want a quick check just to make sure the movie isn't completely rotten (i.e. another Uwe Boll masterpiece), the rating is there for you.
I don't think anyone in their right mind in Hollywood is going to offer Tom Cruise his next paycheck (say MI5) to be based on its IMDB rating.
It is up to the discerning user to read user reviews and professional critics before making up his/her mind. And if you want a quick check just to make sure the movie isn't completely rotten (i.e. another Uwe Boll masterpiece), the rating is there for you.
I don't think anyone in their right mind in Hollywood is going to offer Tom Cruise his next paycheck (say MI5) to be based on its IMDB rating.
There's an old quote, that I sadly can't remember who its attributed to, but it says
"Tell me how you measure me, and I'll tell you how I'll behave"
"Tell me how you measure me, and I'll tell you how I'll behave"
Well there's no solid connection between ratings and the lasting quality of an MMO, is there?
Most AAA MMOs will rack up a tremendously high score, whether or not the game is actually worth playing. Almost no ratings take end-game into account. How could they? You'd have to play the game for 200+ hours to actually be qualified to rate then end-game.
Personally, I think an player-written, amazon-style (but more complex) rating would provide a more accurate review. Much more comprehensive than a handful of employees playing the starting areas.
Most AAA MMOs will rack up a tremendously high score, whether or not the game is actually worth playing. Almost no ratings take end-game into account. How could they? You'd have to play the game for 200+ hours to actually be qualified to rate then end-game.
Personally, I think an player-written, amazon-style (but more complex) rating would provide a more accurate review. Much more comprehensive than a handful of employees playing the starting areas.
# posted by 
: 19/7/12 15:30
: 19/7/12 15:30
I used to be a big fan of the "user reviews" because even when they varied wildly they did give an overview of real players experiences with the game. The recent trend of polluting user reviews has really downgraded their value however particularly since the polluters have got cleverer. It used to be a 100 instances of "Spore Sucks because of DRM". Now you will get a 100 randomised complaints making it impossible to sort out genuine players from pressure groups.
I would very much like a system like Netflix ratings that involves comparisons to "users like you" based on what other things you have liked or not liked. Rather than attempting to declare some objective "rating" for all games, it simply offers suggestions that the system predicts you will also like.
I've worked at game studios with metacritic-related target bonuses. We actually had two targets for bonuses, but they were related to each other: One based on sales, and one based on metacritic. If we reached a certain metacritic number, the sales number target was lower. Otherwise, it was higher.
The general thought, as it was explained to me by my coworkers, was that the marketing department and the development studio both share credit for sales. The metacritic score is (roughly) inversely proportional to the amount of sales success due to marketing. Conventional wisdom suggests that if the game is good on its own, marketing doesn't have to do as much to sell it.
This is why you see games like 50 cent: Blood on the Sand selling millions of copies, despite the game being garbage. The publisher will credit the marketing team for that success. The studio itself won't receive a pat on the back, but the marketing team will.
The general thought, as it was explained to me by my coworkers, was that the marketing department and the development studio both share credit for sales. The metacritic score is (roughly) inversely proportional to the amount of sales success due to marketing. Conventional wisdom suggests that if the game is good on its own, marketing doesn't have to do as much to sell it.
This is why you see games like 50 cent: Blood on the Sand selling millions of copies, despite the game being garbage. The publisher will credit the marketing team for that success. The studio itself won't receive a pat on the back, but the marketing team will.
I've seen several examples where a user has written a meaningful, relatively-fair review with reasoning in place and an evaluation of the various pros and cons of a game... then given it either a ten or a one. People are voting binary instead of giving accurate number representations of their opinion of the score. I half-suspect it's an evil side-effect of facebook 'liking', but also an awareness that in order for their vote to affect the overall score positively or negatively, it needs to be at either extreme.
It's a little sad how common this is, and should probably be adjusted by metacritic. Adjusted out of existence. If users are going to behave as if they're just an aggregate of ones and tens, not an average actual score, make them just an aggregate. Like or dislike.
It's a little sad how common this is, and should probably be adjusted by metacritic. Adjusted out of existence. If users are going to behave as if they're just an aggregate of ones and tens, not an average actual score, make them just an aggregate. Like or dislike.
IGN is complaining about MetaCritic? That's priceless, since IGN has consistently been one of the worse game review sites for years, often giving sponsors better scores (same as GameSpot), being far from unbiased, have reviewers who didn't even play the game they review.
I never put much faith in professional reviews, they've been lying to me for years. If I want to know what a game is like, I ask people I know, guildies, and those who have the same taste in games as me.
Professional game reviews are worthless.
I never put much faith in professional reviews, they've been lying to me for years. If I want to know what a game is like, I ask people I know, guildies, and those who have the same taste in games as me.
Professional game reviews are worthless.
"Well, not Metacritic itself, but game studios in which the developers are paid according to the Metacritic score their games achieve."
Alt 789 Great game, played it till 4 am! Would recommend! 10/10
Alt 790 Wonderful game. My whole family loves it, 10/10.
Screw this, I'll buy 1000 likes off a Chinese clickfarmer.
Average metacritic rating 9.99
I can't believe people actually run businesses in the way you describe. Or let me say I believe you but find it hilarious and ridiculous. I've met seven year old girls who are better project managers.
Post a Comment
Alt 789 Great game, played it till 4 am! Would recommend! 10/10
Alt 790 Wonderful game. My whole family loves it, 10/10.
Screw this, I'll buy 1000 likes off a Chinese clickfarmer.
Average metacritic rating 9.99
I can't believe people actually run businesses in the way you describe. Or let me say I believe you but find it hilarious and ridiculous. I've met seven year old girls who are better project managers.
<< Home
