Tobold's Blog
Friday, May 29, 2015
 
Politics in review scores

Imagine that you created a game playing in 2015 Europe. As a dev your bonus from the publishing company depends on the Metacritic score of the game. And then you read a review in a major publication where the reviewer gave you a lower score because he didn't agree with a map in the game showing the Crimea as being Russian (or alternatively as being not Russian). Bonus gone because of a difference in political opinions. How would you feel?

In reality the game developer in question, Adrian Chmielarz, and the reviewer from Polygon had a political difference about sexism and equal representation of minorities in the game The Witcher 3. But otherwise the story remains the same, the reviewer gave a lower score to The Witcher 3 than other reviewers because of politics. And because this is the post-Gamergate era, any discussion of gender / minority politics in games always ends up exploding in a huge shitstorm. The problem with those shitstorms is that people only ever discuss minor details like some statement not being 100% accurate, or the credibility of this or that person, and totally fail to discuss the core issue.

I have no interest whatsoever in discussing the details of the Chmielarz / Polygon spat, and will delete all comments trying to derail this thread towards those details. What I would like to discuss is whether it is justified to give a worse review score or better review score to a game because you disagree or agree with the politics of the game.

Games have come a long way from Pong, Pac-Man, and Tetris. So when a game stops being about the interaction of abstract shapes, but instead shows cinematic quality stories, it is only natural that the reviewer has an opinion about the stories that are being told. And it is nearly inevitable that those stories in some way touch on political issues, because everything in life does. Would you expect a book review of "Capital in the 21st Century" by Thomas Piketty (or the earlier incarnation "Das Kapital" by Karl Marx) to be politically neutral and only talk about whether the book is well written or not?

On the other side threatening a developer with bad review scores if he isn't politically correct is clearly a form of censorship and attack on artistic freedom. I remember people complaining about the promotional material for Warlords of Draenor, because it showed only male orcs, and they wanted equal representation: Some male, some female characters, and preferably two gay orcs holding hands and another one in a wheelchair to represent the handicapped demographic. Personally I don't think we should repaint Da Vinci's "The Last Supper" to include apostles of different skin color or gender. I believe that any artist, including game developers, should have the artistic freedom to say that *his* vision of warlords is one of blood-thirsty male brutes that just aren't very inclusive as a club. In particular I believe that if you tell a historical story, you should have at least the freedom to depict gender and race relations in a historically correct way, even if we all agree that those relations have progressed since. If gender and race relations in medieval times weren't very enlightened, that isn't exactly the fault of the artist who depicts those times. You *can* create a story based on the premise "what if people in medieval times would have been totally enlightened", but you shouldn't be forced to.

I think that while a reviewer could well mention his politics and his political opinion on things shown in a game in the text, it is somewhat unfair to then let those politics affect the review score. Review scores are simple numbers that don't reflect the details of how a reviewer got to them, especially once they are aggregated. The most common use of a review score is for a customer to decide whether a game is any good and whether he should buy it (thus the link to bonuses). Personally I prefer reviews without scores, but if you have to put a score, that score should say more about the quality of the game than about the politics of the reviewer.

Comments:
"Would you expect a book review of "Capital in the 21st Century" by Thomas Piketty (or the earlier incarnation "Das Kapital" by Karl Marx) to be politically neutral and only talk about whether the book is well written or not?"


No, I would not expect the review to be politically neutral. In fact, I would expect that many reviewers would not only be politically non-neutral, they'd express their opinions on whether the author got their facts and political interpretations correct.

I would, of course, then take any such review with a grain of salt because the reviewer is likely to have a political bias going in. A socialist reviewing a book touting capitalism (or vice versa) is going to write a review influenced by their base opinions.

We ultimately cannot expect game reviews to be any different. Reviewers are already self-selected out (hopefully) by having the person doing the review be someone who actually enjoys playing this type of game. Even within a game type, reviewers are not all looking for the same thing. More than once I've read a review and seen a reviewer talking up a feature I see as unnecessary or detrimental; I've also thought features a reviewer has panned were actually positives for the game.

In the end, I believe the problem is not about whether politics should be in the review (as long as it's clear how the reviewer's beliefs are affecting the review), but instead is about the fact that game developers have to rely on reviews they have zero real control over to determine their bonus. Or even worse: Determine whether the company behind the game will continue their employment blindly based on review aggregation.

If we want to have a variety of games to enjoy, game developers need to have some freedom in what they create, not just keeping them stuck in a tiny mold likely to be accepted by the most reviewers. If we want to have quality reviews, both buyers and developers need to accept that not all reviews or reviewers are equal. And if we ultimately want things to be fair, the Metacritic crowdthink needs to be given a lot less value than it currently holds.
 
Note on the above: I am not against what Metacritic does. I am opposed to how that data is sometimes used by others.
 
I don't see how game reviews or game development itself could be separated from politics. Expressing an opinion publicly is inherently a political act.

Also, the very act of tying bonuses or even employment status to aggregate review scores is political; It tells the developers that the bosses expect them to please everyone. That is the power imbalance that restrains artistic freedom.. and I can't think of any cure to that issue that wouldn't be worse than the issue itself.

Thankfully, it's the golden age of crowdfunding, where even industry veterans appeal to their patrons directly to fund games that they they all like. And even that is not apolitical; They've just exchanged one patron for a group of smaller patrons.

The only apolitical developer (or reviewer) is the one who never allows others to see their work.
 
Welcome to the world of rating based on proxies and scoring systems....

It's not like we didn't have the exact same problem in science with bibliometrics.

To be more on-topic: ideally, a reviewer shoult not be biased by politics, but reviewers are human, so they're not perfect and it's inevitable that prejudices/politics/morality/whatever WILL factor in the comment they make of whatever they are reviewing and in the final score.
But the real problem is the one I started with: the fault lies with the company which just wants to cut corners and judges the result by taking a number from an internet website, number which is generated by a self-selected sample. They should dig deeper to see where that number comes from.

 
The problem is not the review scores themselves, but the stupid expectations video game companies and consumers place on them.

In any other creative medium, be it books, tv, art, or film, politics DO in fact play a part in it's review and score.

A movie like say Brokeback Mountain got vastly different reviews and scores depending on the political/religious beliefs of the people that reviewed it. To pretend, let alone to ask, for these views and beliefs to not influence scores and reviews is naive at best and ignorant at worst.

As long as the reviewer is transparent about what is affecting their scores it doesn't matter why they score something the way they do. It should be up to the consumer whether they put value on that review or not. Somebody score a game down because it lacked women and you think that's a dumb reason? Easy; don't rely on that site or reviewer for your information.

A site like Polygon is very clear that it is left leaning and liberal. There is absolutely nothing wrong with them judging games by what they believe. What is insane is the fact that people know this about the site and then get up in arms when they publish a review incorporating those beliefs. If you know you don't agree with the way Polygon writes it's articles why visit the site at all? Why not find a site or reviewer that is more in line with how you see things. God knows there is no shortage of reviewers out there. Polygon not your thing? Go with Total Biscuit. He doesn't fit? Find someone else.

Now the issue of companies basing employee bonuses on this review is even more insane. I hope employees of these companies will eventually be able to band together and fight back against such a dumb practice. Reviews of games should not in any way affect whether or not employees are paid bonuses. If anything should be used as a metric for something like that it should be sales numbers or something more tangible. Whether Joe Blow the reviewer scores your game a 7 or a 8 should not determine whether hard working people earn some pay.
 
On the other side threatening a developer with bad review scores if he isn't politically correct is clearly a form of censorship and attack on artistic freedom.

What? How in the world does this constitute censorship? The game has already been made and is being sold in retail stores (or online). What censorship is taking place? Reviews are always 100% subjective interpretations of a given experience, so to suggest that it's censorship is to state I'm censoring you for disagreeing with your argument.
 
I am extremely pleased that Polygon gave the information in the review that they did, and strongly believe they were right to do so. *Of course* political perspective plays a part in a review (on *anything*): as if it couldn't! If I'm reading a deep green ecological site, I would absolutely not expect a review of the new Apple watch. A critique sure, but a product review? A product review is the publisher saying 'dear target audience, this is something that might be of interest to you.' That process is - precisely - a political process, and reflecting it in the final product review is, in fact, a feature of good reviewing.

So site x, with target audience skewing left-liberal politically, points out that game y has elements that said target audience are unlikely to find appealing: that is the very *definition* of a good review. Is it possible to enjoy said game inclusive of those unappealing parts? Of course (and the review in question ended up very positive on the game); is the presence of those unappealing (to the target audience) parts worth mentioning to the target audience: it would be dishonest *not* to mention them.

I'm a huge RPG fan but actively dislike the world of The Witcher: I don't wish to play a power fantasy where I'm the law-breaking outsider anti-hero, transgressing against the bougeois rules of society because *someone* has to keep the milk-sops safe. Other people can want to play them, and more power to them - I hope games keep getting made that serve their fantasies. *I* don't wish to play them - and being told that Witcher III is the same, in this respect, as Witchers I and II, was precisely the information I needed to guide my purchasing choice. SO yes: both in the general case and the specific case, reviews *must* point out issues - politics - that are relevant to the target audience.
 
"The most common use of a review score is for a customer to decide whether a game is any good and whether he should buy it "

That might be the most common use, but it is not how many reviewers see it. That's the lowest form of the matter, reviews as objective buyer's guides, like Consumer Reports' roundups of the best vacuum cleaners.

They see themselves as critics. They're providing their criticism of the subject matter, just like a book review in the new york times magazine. Criticism is always inherently personal and subjective. That's what this is _really_ about-- criticism.

It sucks that many studios are paid bonuses based upon review scores. That should change. But does that mean that reviewers should strive to be objective? Is that even possible, to eliminate your personal biases? Would we want Roger Ebert's reviews in the Chicago Sun-Times to be identical to A.O. Scott's in the NYTimes? Why are game reviews treated differently?

It all comes down to lack of respect for games as artforms. But they deserve that respect. They deserve to be properly criticized. That offers real value.
 
This comment has been removed by a blog administrator.
 
The creative artist of a review is the reviewer. How the reviewer chooses to express opinions, including what he or she includes or omits is an artistic decision first and foremost. As a consumer of reviews my primary expectation and requirement is to be entertained not informed. I'll go to the press release and the news pages for information.
 
Well that's kind of the whole point of critics isn't it? You find o e that you generally agree with (be it gameplay, or politically if you are so inclined) and that critics score has meaning for you.

Major companies have to try to find ways to reward people for the work that they do. User feedback (or in this case reviews) is one of those half passed ones that is an easy way for companies to say"you could have gotten more but..."
 
"On the other side threatening a developer with bad review scores if he isn't politically correct is clearly a form of censorship and attack on artistic freedom"

I disagree with this statement in the strongest possible terms. It is not "clearly" either any more than you saying you won't link to a blogger because you don't like his views is censorship. Criticism is not censorship, and anybody saying that criticism of their speech or artistic expression is such is grossly misinformed. Hurt feelings or lower review scores do not represent attacks on artistic freedom.

Speech, and by extension, artistic expression, has consequences. You make your choices and you either stand by them or you don't. People are allowed to disagree, that isn't censorship in any way.

Meanwhile, reviews are completely subjective endeavors. The idea that one can have an objective review of art/entertainment is ludicrous. The purpose of art is to engage with people, and how that happens will differ from person to person based on their experience and their current context. If a reviewer wants to be honest and give their personal context as to what may have influenced their review, such as the current allegiance of Crimea, fine. But I don't think a detailed listing of the reviewer's personal context is a requirement.

You read a review and you evaluate whether or not the reviewer values the same things you do. I have read low score reviews that made games sound good to me and high score reviews that have steered me away from a game. If all you do is look at scores though, well, you're taking a short cut that indicates that you are not that interested in the subject at hand.
 
I expect the sensibilities of the reviewer to affect the review.

1) It's going to anyway; why pretend differently

2) Criticism/reviews are not "should I buy it". Whether something is a good book/movie/game and whether I should buy it are very, very different questions.

3) If you design a game with a map of the Crimea in it,and don't expect that will cause controversy and reduce sales, then you are not a good developer. Your shareholders care what affects sales, not what should affect sales.
 
Just out of curiosity, but did someone really criticize Warlords of Draenor for not including a couple gay orcs, female orcs and a handicapped orc, or was that just a bit of hyperbole? I mean....I know of websites where that sort of narcissistic criticism sounds about on target, but never actually encountered the specific criticism you mention.
 
re Nicholas Bergquist

My recollection was there was a couple of contretemps when a Bliz employee said something "unmodern." They were criticized. Some of the defenders used the Reductio ad absurdum defense of that if games had modern sensibilities would it mean requiring a couple gay orcs, female orcs and a handicapped orc.
 
There was an actual demand of having female warlords on the Warlords of Draenor poster. The gay and handicapped part was the reductio ad absurdum.
 
One person's 'political nonsense' is another person's 'important factors'. It's impossible for everyone to decide and agree what should matter and what shouldn't, so it becomes a personal matter for the reviewer to decide and a personal matter for the reader to decide if the reviewer's review has merit or not.

The real problem is metacritic, or scores in general.
 
That's the lowest form of the matter, reviews as objective buyer's guides, like Consumer Reports' roundups of the best vacuum cleaners.

Scores are the problem, because they make reviews appear as if they were objective consumer reports. As much as I agree with that being stupid, the industry clearly treats review scores like consumer report scores. And so do many readers. If a reader expects to find out which is the best vacuum cleaner, he could well be annoyed if a particular brand has been scored down because there aren't any women on the board of directors of that company.

If review authors strive for something higher than a consumer report, they should not give a review score. Knowing how industry bonuses works it gives reviewers a tool for political correctness blackmail.
 
While it's unjust, it's impossible to avoid. A review score, unless created by completely measurable criteria like graphic quality of the engine, area of roamable space and such, will show the subjective opinion of the reviewer who has his own biases.
 
Essentially everybody in the industry agrees that game reviews shouldn't contain arbitrary scores. Every publication tried that, at one point or another, and public outcry forced them to retreat back to scoring their reviews.

One site, I believe it was the late/great C|net Gamecenter, had a scoring method that met my approval. Every game was scored Hit, Direct Hit, or Miss. So it was fine, amazing, or stay away. I haven't seen anyone try this more recently, but I do prefer it. While it's still a score, it avoids the arbitrary nature of "What's the difference between a 7.5 and a 7.9, really?"

This is one of those circular arguments that comes up every year or two in gaming journalism discussion forums, and the same arguments are made every time, everybody agrees, then goes back to scoring reviews because they have no choice in the matter.

The political aspect with gamergate and whatnot is an unusual, and sad, spin on a very old argument.
 
You hit the nail on the head there Tobold. Unfortunately what I've heard many sites say is that not having a score negatively impacts site views... Consumers for whatever reason want that stupid number next to reviews.
 
You hit the nail on the head there Tobold. Unfortunately what I've heard many sites say is that not having a score negatively impacts site views... Consumers for whatever reason want that stupid number next to reviews.
 
There was an actual demand of having female warlords on the Warlords of Draenor poster.

Were people making demands, or just saying that they were confused?

If the Warlords had been a representative cross-section of Draenor society, then only about half of the characters would have been men. For some reason, somebody decided that men needed more representation than that, so they shoehorned in a bunch of gratuitous male characters. It was obvious that they had only been put there to fill a quota, and I thought it was weird and jarring. I still don't understand why it was done that way. I guess somebody had a headache, and didn't want to have to deal with a bunch of whiny dudebros shrilly demanding more men?


Other than that, instead of looking at "bonuses based on review scores" as another way to lose, why not look at it as another way to win?

The alternative is to pay bonuses based strictly on the number of sales. But when you do that, you lose the ability to reward "noble failures" - games that are too ambitious, or too inventive, or too controversial, or otherwise ahead of their time. Reviewers might give them glowing reviews, but that won't translate into a lot of sales. So this kind of bonus system gives management a way to acknowledge that. (And of course there can be a separate bonus structure to reward "big dumb blockbusters" that get bad reviews but lots of sales.)

Of course, if you write "Concentration Camp Simulator XV", and the critics hate it, and gamers don't buy it, and the non-gaming media uses it as an example of what's wrong with gaming, then maybe you shouldn't get a bonus. That holds true even if most of the objections to the game are political.
 
A lunatic fringe complained that all the warlords were male, yes. Which is silly, because the warlords of draenor expansion is a callback to the old warcraft games and there actually weren't any female warlords at the time. It wasn't a big outcry or anything, not that they didn't put in the effort to drum up support.
 
A developer is free to develop the game they want.

A critic is free to rate the game they want with the criteria they want.

A publisher is free to grant bonuses based on the criteria they want.

And that is that.
 
@Vitoly

Considering that this was an alternate universe where many of their histories are very different (Nerzul is no a spiritual leader of all the clans for example), it would have not been very hard to at least include a token female warlord that for instance, usurped power from the previous male warlord. Also, I thought they made up a few warlords because some of the clans created in later games? And good job calling people you disagree with lunatics.
 
Oh wait there are only seven warlords? For some reason I thought the thunderlords were a warlord for some reason.
 
Sure. Make a bunch female, one trans, put one in a wheelchair, one with Downs syndrome, a hemophiliac with a giant padded helmet, and variously assign skin colors according to Skittles flavors. Why not?

Well, because it's silly. None of the warlords were made up, they had lore going back 20 years. If you deviate too much from the lore, you lose the nostalgia factor entirely.

I think it's worthwhile to make a real effort to be inclusive. I really do. But in this specific case, replacing a warlord would have been a mistake. A "power behind the throne" type subplot led by a female orc would have been compelling, though, I'll give you that.
 
Well that's kind of the whole point of critics isn't it? You find o e that you generally agree with (be it gameplay, or politically if you are so inclined) and that critics score has meaning for you.

Major companies have to try to find ways to reward people for the work that they do. User feedback (or in this case reviews) is one of those half passed ones that is an easy way for companies to say"you could have gotten more but..."
 
Well that's kind of the whole point of critics isn't it? You find o e that you generally agree with (be it gameplay, or politically if you are so inclined) and that critics score has meaning for you.

Major companies have to try to find ways to reward people for the work that they do. User feedback (or in this case reviews) is one of those half passed ones that is an easy way for companies to say"you could have gotten more but..."
 
Hmmm, I think that...when games become political, it is open season to critique them on political grounds. The problem here is that you don't have to INTEND something to be political as the creator for the final product to be political. But the same is true of movies, television shows, books, etc.

In particular I believe that if you tell a historical story, you should have at least the freedom to depict gender and race relations in a historically correct way, even if we all agree that those relations have progressed since. If gender and race relations in medieval times weren't very enlightened, that isn't exactly the fault of the artist who depicts those times.

That's kind of dangerous, because you run into the argument that women and PoC WEREN'T treated the same way historically as the traditional narrative depicts. You have the great work of sites like http://medievalpoc.tumblr.com/ to catalog that europe wasn't as lily white as some people so...what exactly do you mean "a historically correct way" exactly? Even something like that is not apolitical.
 
Hmm wouldn't forcing a reviewer to ignore certain aspects of his oppinion of a game in his review be closer to censorship than him making a bad review?

I would turn the issue on its head and claim that political oppinion ought to influence reviewscores. At least as long as the "problem" is something contained in the game as in your examples in the article. In a prior comment however you mentioned someone giving a lower score to a game due to the company not having any women on its board... This seems like a completely different thing.

As an (to me at least) obvious example on when political oppinion really ought to have influence could be the following: imagine a game about an (insert evil stereotype, could be nazi) torturing a prisoner of (insert deplorable victim, could be a child). Maybe this game actually has really good gameplay, but as the story/theme of the game goes against anything i believe I would still want to rate such a game very low. And I would want review scores to be allowed to represent a similar sentiment in the reviewers. I dont claim they MUST rate it badly, merely that they ought to be allowed to. Of cause this example is something most people might not even recognize as a political issue, as the scene displayed is one that is more or less generally accepted as bad, but the reason that reviewers should be allowed to score this game low, is the same reason that they should be allowed to detract from reviewscores due to political controversies, the experience of playing the game rubs them the wrong way. Even if the gameplay as such might be objectively "good".

As a less hyperbolic example any game in which the act of torturing is displayed as a "good act" would still rub me the wrong way. Imagine getting "light side points" in swtor, (or any similar endorsement that the act is considered good by the developer) for choosing to torture a person into surrendering information that leads to saving people, something I am aware certain political viewpoints would endorse (but which I dont). This amounts to the game enforcing its ideology upon me, which detracts from my experience. As this very clearly colours my experience of the game, it ought to be allowed to influence the score that is supposed to represent my oppinion of the game.

(Just to clarify, i am not against allowing games to display torture, nor even against them displaying it as a "good act", I am however against the notion that such a choice should not be allowed to be reflected in review scores. And as this is just my own personal pet peevee i am certain that others feel similar sentiments against other (maybe less radical?) viewpoints, being enforced upon them. Not allowing such design choices to detract from a reviewscore seems to be to be the only thing resembling censorship)

Tldr: Of cause political oppinion should be allowed to influence review scores, at least as long as the point of controversy is with the game (as compared to being with the developer only, and not portrayed in the game).


 
A lot of those demanding 'representation' take an absurdly cherry-picking approach to history. For example, I first heard of the site Sine Nomine mentioned in the context of people demanding that a game set in a medieval Bavarian village should have black characters. But what they mainly linked to on that site was a painting of the Queen of Sheba that was held somewhere in Bavaria. The Queen of Sheba may have existed, and if she existed she may possibly have been black as in the painting, though she was probably Arab - but in either case it's unlikely that she traveled 2000 years forward in time to the middle of a different continent! Their other big example was a saint from Roman times.
 
"On the other side threatening a developer with bad review scores if he isn't politically correct is clearly a form of censorship and attack on artistic freedom."

This is an idiotic claim. The developer is not being prevented from presenting their artistic vision; nothing is withheld from publication. Moreover, as further evidence of the intellectual barrenness of Tobold's view, the only way to prevent the phony "censorship" of the developer is to *actually* censor the reviewer. What is actually happening in the situation described is that the developer has exercised his/her free speech, the reviewer did not appreciate what was said and, as should be done, the reviewer attempted to counter the developer's free speech with more free speech. Working as intended and as is proper.
 
If there is an aspect of the games story that negatively impacts on your enjoyment of the game-play then I think the score needs to be lowered to match the actual enjoyment level. I applaud the reviewer for being transparent and explaining why he marked the game down.
I'm thinking along the lines of "The game had really good mechanics and everything made sense but I felt uncomfortable because it was making me do X"
 
Post a Comment

<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool