Tobold's Blog
Friday, May 08, 2009
The reason I don't rate games

There is a huge fight going on between Eurogamer and Aventurine, after Eurogamer rated Darkfall 2 out of 10. In response chief developer Tasos Flambouras (I'm not kidding you, that is his real name!) from Aventurine posted a reply saying that the reviewer only spent 2 hours in the game, and thus wasn't qualified to judge it. Eurogamer replied that Aventurine faked or misread their server logs, because the reviewer had spend 9 hours in the game, and now they plan to do a second review. But at least Aventurine managed to divert the attention away from the arguments of the review. So much more fun to discuss over incompetent game reviewers!

If that review had been posted by a blogger, and not given a rating, it would have been rather unremarkable. It is an opinionated rant from somebody who obviously hates the game, and that is a writing style which is quite frequent in the blogosphere. But it isn't actually lying, just putting too much drama towards Darkfalls existing and very real flaws. But as that article appeared in Eurogamer, and put a horrible score on Darkfall, it caused a huge outcry. The author could have written exactly the same review mincing his words, and given the game a fairer score, Keen proposes 4/10, and it would have looked more like game journalism.

But this is why I don't write for game sites, and I don't rate games. Don't you hate these official game magazines and sites which rate EVERY major game between 6/10 and 9/10, and you really need to read between the lines to find out what the author liked or disliked? What games you like is an intensely personal thing (watch Paul Barnett on that subject), and there simply is no way to judge it objectively. In defense of Darkfall, syncaine calls Free Realms a gay simulator, but the simple fact is that Free Realms would probably have been a better game for that Eurogamer reviewer, while Darkfall is a better game for syncaine. Reading somebody's rant about a game that he doesn't like can be quite informative. Even better are posts from people who *would like* to like a game, but are struggling with the game's flaws (Keen's blog is excellent for those kind of posts). But carefully measured, emotionless pieces of pseudo-objective game journalism are as useless as fanboi posts to understand a game you haven't played yourself.

Darkfall can be a great game in specific circumstances, which necessarily include being part of an active guild. Under those circumstances it has features few other games offer, like your actions having potentially serious consequences. I can fully understand some people liking these unique features so much that they are willing to overlook some other obvious flaws of the game, like the skill system. And I understand that as a consequence of game design some actions have to be cumbersome in the user interface: It makes sense that looting is so complicated in Darkfall, because it prevents you from looting stuff quickly in a danger zone. But the average gamer entering Darkfall unprepared, and looking for a solo game, is likely to have a strongly negative reaction not unlike the Eurogamer reviewer. So how can you possibly put a score on a game like that? Posting just the negative drama review sure looks a bit unfair, but you certainly wouldn't want some Darkfall fanboi writing that review, glossing over all the bad parts, and giving the game a 9/10 score. And some bland compromise with neither emotion and some average score isn't going to tell the reader much about the game either. You're much better off reading a bunch of blogs spanning a wide range from haters to fanbois, and at least you'll know what to expect if you decide to buy the game. Whether you'll ultimately love or hate this game, I can't tell you, because it depends a lot more on you than on me. So, no game rating from me.
The problem wasn't just that the reviewer only spent a minimum of time in-game, he also posted outright incorrect information about darkfall, and his editor didn't bother to fact check any of it with the second account that aventurine gave them...

Darkfall being a good/terrible game wasn't the true issue here, it was the complete lack of professionalism or journalistic decency.
I agree on the lack of professionalism, but I didn't see anything in that review that was "outright incorrect". It was opionated, exaggerated, judgemental, and everything, but not untrue. Of course if somebody says "Darkfall has a horrible UI", some people will consider this statement true, and others will say its false. But the truth is that such statements don't deal with facts at all, only judgements, and thus can't be true or false.
Well, since the EG forum is my regular Internet hangout I've followed this spectacle closely from the start and it is most amusing. Now this isn't the first time the site has gotten spammed by raving fanboys, and not the worst. The Darkfall fans seem to be a special kind of raving mad though, which makes it a bit different this time.

Now the most amusing part of this whole kerfuffle is Tasos meltdown over one bad review. It wouldn't be the first time some journalist wrote about things they haven't experienced, but Aventurine really hasn't provided any actual evidence for their accusations. They have merely claimed that their logs, which they won't publish or send to EG, prove their point. So right now it's a publications word against a developer, where the developer seem to think have everything to gain from discrediting the publication.

But it gets funnier still. Now Tasos has decided to decline the offer of a new review. He seems to be trying to be claiming some high ground, while clumsily insinuating that the new review will be a new hatchet job. However he only manages to come across as extremely unprofessional and and overly defensive.

That, combined with all reports about instability issues, a lack of a free trial and the staggering lack of opportunity of buying the game makes me strongly suspect that things arent right over at Aventurine. It seems like they arer almost looking for something to blame an imminent failure on, instead of their own mistakes. If Darkfall lasts the summer I'd actually be mildly surpised.
This issue is similar to film critics. I have been many times to films which critics gave "a rave review" but in all honestly I felt should have had 5/10. Other films have been deemed as "pants" but I watched them and put them up with the best ;), however if i was to review them, I would probably be slated and biased.

Its simply personal preference and everyone to their own tastes I guess. I would almost imagine this Darkfall rater to perhaps be trolling a bit with his review for what ever reason, we may never know.
The bottom line of a review is "do I advise other people to play this game?". And a rating is a quick way to see that without having to read through the entire review. Of course a review is personal as *you* advise other people to buy or not buy a game.

In this case, he didn't like the game and advises people not to play it, that's OK. But only playing the game for 2 hours is not done. It's the equivalent of a movie reviewer leaving the theater before the end credits. Such reviews can not be taken seriously and puts a blame on the reputation of any reviewer.
At the end of the day you should never take a review simply upon its own merits but also look at other reviews and form a rough opinion yourself on would it be the sort of game you would like to play when you look at the features of the game.

One review is bad but not the end of the world. I mean if the evidence is correct Eurogamer has hurt themselves just as much as Aventurine, although game reviewing is a correct business anyway.

Personally, I've never heard of Eurogamer. I tend to get my reviews from smaller, indie sites, or from blogs, as a general choice, so this wouldn't of impacted upon me anyway. I know none of my friends rate a game on any review bar that of word of mouth and actually playing it.

Games Reviews don't carry as much weight in the world now, at least not the newer released ones. They tend to only hold a game above water after a couple of years and only then it's when scores will 'matter'.
Regarding the review, I understood where the user came from when he mentioned player-based economy as a flaw of the game.

For the rest, I can't understand why people speak of "opionated, exaggerated, judgemental, and everything, but not untrue" when in fact the review is full of inaccuracies and opinions disguised as fact.

For feth sake, the reviewer didn't even know how many servers Darkfall had. He didn't play the game enough to realize he could use the "F" key to interact with objects. He took screenshots with the graphics/resolution at the lower settings to demonstrate the graphics were terrible. And people hand wave this as opinionated and exagerated?

And the fething nerve on calling Darkfall players rabid fanbois when not so long ago Richard Bartle was steamrolled by the WoW community due to most of them being unable to read a sentence within the context.

The fact is, Darkfall should be praised for being a work from a small independent developer. Everybody was calling it vaporware but Aventurine came through and they made the game they could with the conditions they had. I don't know, do people have nightmares with FFA-PVP and full looting?

Of course the game has flaws and of course some of them are critical and may break the game if not addressed soon enough. But does Darkfall hurts anybody by being around?
"but Aventurine really hasn't provided any actual evidence for their accusations"

Wrong. AV has actually sent a copy of the logs to EG. And when EG claimed they could be incorrect or editted in some way. AV has offered to pay all costs to fly out a member of the EG tech team to examine their entire logs backup system etc and prove that the logs are infact legit.

I have every faith in AV that the logs are correct. The number of incorrect statements he made in the review suggest strongly that AV is right. Here are a few examples of where Ed is wrong.

"At times you'll be left waiting to die - for up to a minute."
Wrong, You can press "space" once the dying bar is at 25% to die and go back to your bindstone. This is no more then 10-15 seconds.

"Darkfall even lacks the basic lore that even the worst games have. There are wolf people, orks, humans, elves, and dark elves. There hasn't even been an attempt to construct a faux-story"
There is a long story/timeline on the darkfall website giving a vast ammount of law about agon the races and the 4 sub continents.

"The developers have taken the classic stance when faced with the echoing cries of "you barely have any content", and claim that the "core" of Darkfall is clan warfare."
Which it is. Darkfall isn't meant to be a game with raiding content and stuff like that. Its a pvp game. and the fact the clans can build entire citys and hanlets and siege them right from Day 1 is awesome.

Even if Ed had played the game for 9 hours, thats still not enough time to review a game like darkfall. in my first 9 hours of gameplay i didn't siege any cities or jump in and brutally slaughter a pve group (not the starter goblins btw) i didn't get into any huge scale pvp battles. and thats what the game is about.
Yes, but don't you think that not being able to figure out what the game is about in your first 9 hours is a fundamental weakness? We discuss how long a professional game reviewer should play a game before judging it, but forget that the average player isn't going to spend that much time. If after 9 hours the average player still has only seen a few goblins, hasn't found the lore because it is hidden on the forums and not presented in the game, has been ganked a few times, and has never participated in the large scale PvP content Darkfall is really about, isn't he going to simply quit the game and never come back?

You can laugh about games like World of Warcraft or Free Realms, but at least everybody "gets" them in the first hour of gameplay.
You really should understand the story before making statements like "he isn't lying". I know it's a lot to read, but you make yourself look foolish talking about something you skimmed over (but at least it puts you closer to the EG viewpoint).

Plenty of sites have already pointed out the FACTUAL information he got wrong, not just his opinions on it. Also rather basic to the story is he lied about his actual in-game time, which you yourself use. He never played 9, or even 1 hour on a character. He spent 2 hours with the game, most of that time at the character creator screen.

It helps to understand the issue before you comment on it, and you of all people I figured would be the last to put up a 'tldr' post about this.
Tesos claims he has sent logs with proof to EG and EG has respondnded that they doubt the accuracy of the logs, yes. We however haven't really seen shit in the form of evidence from either side. Now since Tesos doesn't seem, judging by several comments over att, to be a perfectly reliable source it really isn't possible to judge who is telling the truth here. So, lacking actual proof shown to us, no one of us should take Aventurine's words here as gospel.

Now assuming that even if Ed Zitron played the 9 hours he claims would that still not be enough to make a proper judgment? Well, that depends. Usually I'd say it's a bit short for an MMO. However if the games fundamental are basically broken to the core, which seems to be what ED's opinion about Darkfall is, then a longer playtime really isn't called for. What is being reviewed isn't what the fans might someday potantially construct with a set of broken tools but the tool's potential in helping the players in their endeavor.

Syncanine, the factual errors people have actually pointed out have been minor issues regarding the UI, which could easily be explained by badly designed instructions and a counterintuitive interface. Apart from that everything Darkfall fans have actually claimed as faults are matters of opinion. They rave about how Zitron's statements about floaty controls, bad hit detection poor graphics and so forth are factually incorrect, when in reality it's just a matter of opinion.

Eurogamer won't get hurt in the slightest here. They merely get some more page hits to show the advertisers. Aventurine however end up with getting a public image of being that company with a hysteric lead developer/spokesperson and a rabidly insane fanbase. Anyone objective can easily see that this whole debacle is a sign that things over at Aventurtine are not as they should.
Claiming we don't know if the logs are real is just denial. Anyone who has actually played the game (and even those who have not) were able to point out errors in his piece. Know why? Because the errors are not high-end minor detail stuff, they are 'first hour' basics.

Lying that you played something for X hours, when in reality it was X minutes, is not a 'minor UI' issue. The review would have been perfectly fine if the first line was "After 14 minutes of playing DarkFall, here is my review...", but that's not the case here, he tried to pass it off as a legit review, and got called out on it.
Then why haven't the players actually managed to point out what these glaring flaws are? You sure as fuck haven't, here or on your blog. Basically the game's fans have disagreed about matters of opinion and then overstated the importance of a few minor issues that might be factual errors.

Go to and you will find just as many people, among those who claim to have played the game, who see Zitron's review as spot on as you'll find Darkfall supporters. As an outsider it really isn't easy to get the impression that the review is factually incorrect or based on merley two hours of play based on public information.

But then, you seem to think that Tesos latest crash and burn failure of a hissy fit was some sort of game set and match, so I really dooubt you are viewing this issue objectively. I mean come on, you can't seriously think that his reaction is in any way benificial and constructive here.
I've listened to the folks at 1up discuss their review philosophy for a while now and upon reflection I think there methodology is the best compromise when reviewing games. There are aspects to any game - technical issues, the amount and depth of content, the consistency of the experience - that will be the same for almost everyone who plays them. Beyond that, so much of one's experience and enjoyment of a game is conditioned on their own aesthetic preferences that to put oneself, as a games critic, in the position of the "objective game buyer" is impossible. Stripping away one's personal reaction to the game doesn't make the review more objective. Instead, the better game critics that I read describe those components that are invariant between people and then go on to describe the subjective experience of playing the game. This means that every game review will not be useful for you since that reviewer might have sharply divergent tastes but as you say in the post, if you read a number of reviews you will get the more complete perspective that you desire or at least one that is closer to your own.
Nils, I'm not going to do the work for you, but plenty of sources (including the comment section on my blog) have pointed out the errors.

Aventurine pointed out the main one, he claimed to play 9 hours, he loaded the game for two, with most of that time spent at the character screen. Explain how that's "minor issues that might be factual errors", or how after knowing that, you can read the review and consider ANYTHING written as valid?
I have already done that work. I've read the bullet point lists DF fans spam all over the place. The lists, without fail, contain the same complaints about how Zitron didn't figure out the hotbars and how you press F to do things with the user interface or something along those lines. That is the full extent to which all these critics can point to what can be seen as actual errors. The rest is just fans who disagree on wether combat, control, graphics and UI are unintuitive and crap or just right. The latter is simply opinion. It has fuck all to do with any factual errors.

Saying that "reviewer X thinks the hit detection is wonky but I don't" does not mean you have provided an example of an error. It simply means you have a different opinion compared to reviewer X.

And once again, do you really believe that Tesos has handled himself in a manner that has helped his little niche title in this affair?
I'm with Nils on this one. I haven't seen a single fact in the review which anybody could prove to be wrong. There were a couple of "the reviewer didn't figure out how to do that" points, and a lot of "the reviewer called something bad which I consider good" points. But as the review contained very few facts to start with, and lots of opinion, there wasn't really much you could honestly claim to be untrue. You're simply counting every time where the reviewer said he couldn't do this, or didn't like that, as a lie.

The original review didn't even say how many hours the reviewers later claimed to have spent in the game, so if that was a lie, it was one not contained in the review.

So you see, syncaine, if anyone did a tldr post here, it was you, not me.
What amazes me is what passes by "opinion" these days.

If some reviewer said that the controls of CoD4 were clunky/floating (whatever that means) everybody would call him nuts, not saying a complacent "it's a matter of opinion".

Anyway I would laugh my ass off if SOE or EA bought Darkfall. All of a sudden we would see better reviews all over.

"PvP in Darkfall is very rewarding albeit some minor bugs which are not deal breakers. Typical MMO gamers may feel some discomfort at the FPS oriented controls but once getting used to the movement will feel smooth and responsive. The player driven economy is a plus making crafting a rewarding activity and a solid source of income..."

You get the picture. We all have to eat don't we?
If a reviewer would call the controls in CoD 4 floaty and clunky it would be met with disbelief because most people don't share his opinion. It's still just an opinion though.

Complaints regarding controls in games, based on abstract criticisms such a perception of "floatiness", "unresponsiveness" or whatever, are standard. Read any review of a Japanese survival horror game and you'll find it there.

The CoD 4 example you provided, wyrm, is simply an example of where the majority wouldn't share the opinion of the reviewer. What that has to do with the opinion on a game where several players share the reviewers opinion, several others disagree violently and the majority doesn't even know the game exists is beyond me.
Simple facts: You can't right click to interact with NPCs as the author claims, that's done with the F key.

Please tell me how that's just 'his opinion'? (and this is not some 'minor UI' thing, since almost everything in-game revolves around which UI mode you are in, and it's impossible to do much of anything in the game without using the F key)

If you say you claim something is one way, and it's not, that's called a lie, not an opinion. Claiming black being lighter than white is a lie, not just 'an opinion'.

The issue about time played came up when he got called out for clearly not even playing the game beyond logging in. EG said he did, Tasos proved he did not.

So again, it seems tldr got in the way of your post. Like EG, get the facts straight before you make a statement like "he isn't lying".
And once again, do you really believe that Tesos has handled himself in a manner that has helped his little niche title in this affair?Possibly, if the publicity gets enough attention to the game to outweigh the negatives. (Most darkfall news I've seen recently involves people on blogs arguing about whether it is good or not, for some reason this game seems to bring out stronger emotions than usual on the MMO blogs I've read.)
Tobold, how do you hand wave the fact that he took the screenshots with all the graphic settings on low?

I say that Darkfall graphics are worse than Pong's. Is it a matter of opinion?

I say the collision detection doesn't work when, afaik, that was never a issue with the game. And do you really think that even the most hardcore Darkfall fan would ignore that. They're fething PvP'ers for feth's sake.

The blanket of "opinion" is a very big. So big that sometimes people try to pass slander as an opinion.

And the final straw that most of you Darkfall haters tend to ignore is that DArkfall players are very aware of the problems of the game and they discuss it on forums and blogs. Even Syncaine who is a fan of the game regularly blogs about Darkfall issues and issues more serious that the ones pointed by the reviewer.

Anyway, is pointless. People have bad dreams with FFA-PVP and any game or developer who dares to create such a game will be the target of hate. The gods will know why...

The "majority" of the players currently on the game, you know, the ones that even provide their in-game character name and guild, say he is wrong.

For all the rest, it's just "i have a lvl 80 in another server."

So, yes, maybe the reviewer only has 3 fingers in each hand, that is why the controls are not responsive...
"Unbelievably, to do anything that involves any interaction at all, you have to stop still - this includes any and all inventory management, looting, chatting - anything interactive. It isn't totally clear whether this was a decision or a design oversight, as many other elements of Darkfall don't quite work."

This is one of Ed's very first comments on the game. It's blatantly false. Not an opinion...not something that takes 5 hours to figure's just completely wrong. And it is one of the "lies" in the article that Ed puts there to make the game look give his infamous 2/10 rating.

Another one off the top of my head...he says there is no tutorial to teach you the UI. But Ed, that is the very first think that shows up on the screen when you log in, and it is always there by default. Again, this isn't opinion. it's there on the screen every time you log in and you have to turn it off manually.

So, how are these items "opinions"? And again, you don't have to play the game 5 hours or more to figure this stuff out.

Oh well...people will see what they want to I suppose. I'm done arguing with people who refuse to see this review for what it is and think it's just a subjective piece with no supposed "facts" that a blind person can see is rotten from a mile away.
This comment has been removed by the author.
Why are we spending so much time talking about Darkfall when the real travesty is Gamespot's review of "Stalin vs Martians"?;read-review

A 1.5? How can such slander be called journalism? Just because it's not like that Blizzard RTS that shall not be named doesn't mean it's a bad game. I hereby call upon Gamespot to have someone actually play SVM for at least 50 hours and then re-review the game.
Terathis, why do these posers ONLY give such ratings to independent games that aren't backed by large multinational companies?

Reviews? Gamespot? LoL, as credible as W speaking of WMA's in Irak...

(the game appears to sucks ass, though... but kudos on the it's title, pure win)
Sorry, but I'm calling horseshit here. What is Aventurine's (and its supporters') point, anyway?

Is it that telling "lies" about the UI is bad? Well telling "lies" on about all the amazing game features must be bad too.

Is it that driving away potential Darkfall buyers is bad? No disrespect to Eurogamer but however many people are driven away by this review, it surely PALES in comparison to those dirven away by (1) years of "go back to WoW, carebear" on ForumFall and (2) Aventurine's shall we say novel approach to "selling" its game.

Sorry, but Aventurine doesn't have a leg to stand on in the righteous indignation department.
Yes... Darkfall sucks and is a horrible game... you would never want to play that game... WoW is so much better...


Seriously, I'm having more fun in Darkfall in the first month than I had in my 4,000 hours of WoW...

Don't take another persons review as gospel on whether or not you'll enjoy a particular game!
I think Tasos is simply bringing the hardcore pvp social style to his professional work. Waiting for him to start teabagging the reviewer's cold dead body.

Wierdly while most of us will just laugh at Aventurine it's probably quite appealing to Syncaine types - a games dev with the balls to start a fight with a games journalist. RARRRRRGHH!
I don't see the distinction between reviewers who give "bad games" low reviews and sites that give everything 6-9/10. If you're being fed inflated scores, then just treat everything on the low end of the scale as being bad. There's always a reason why something scored lower rather than higher. Or, just find a reviewer who uses the entire spectrum. Generally, any review worth it's salt IMO takes into consideration mass appeal versus genre specific appeal. Games that transcend genres score higher. In the past, you said that Darkfall is a niche game. A low score seems fair then, because the game won't appeal to most people.

I haven't read the review nor have I played Darkfall. If the reviewer said things which were "incorrect" is it because the game hid those features away so they were never discovered? In this day and age, that's another valid point deduction.

I think the real problem is reviewing an MMO. 2 hours, 9 hours, all of that is completely insignificant. You can't get anywhere in an MMO with such little time. OTOH, if the game isn't fun and appealing at the start, it tends not to be later either, as far as the base game mechanics go. But the most fun and rewarding events usually only exist at endgame.
Syncaine is hilarious. His blog is trolling, but even better than normal trolling, you actually have to go to it willingly to get trolled. The man is a genius.
In one of the first comments Carra says
The bottom line of a review is "do I advise other people to play this game?". And a rating is a quick way to see that without having to read through the entire review.I was wondering why we even need a rating scale at all? Why not a simple yes or no. Would the author of the review advise you to pick up this game, yes or no? I don't need to know how much it leans in one way or another, if it sucks it sucks.

There may even be a better indicator of how much the reviewer recommends the game. At what point did they want to stop playing? As Tobold mentions, most people playing a game will give up in a frustrating game long before a reviewer will. So, when did the reviewer want to put down the game and call it quits? The 1 hour mark? 10 minutes into the game? Are they still playing it at home even though they are done with the review? I think that is a far greater indication of merit than a number between 1 and 10.
Post a Comment

Links to this post:

Create a Link

<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool