www.Tobold.com
Tobold@GMail.com
A blog with my thoughts regarding games I am playing and other stuff in life. Please read my Terms of Service
RSS Feed
Thursday, May 28, 2009
How much time do you need to play a game before you can review it?
The Ed Zitron Eurogamer review has become a running gag in the MMO web sphere. But nobody really answered the question of how long exactly you are supposed to play a game before writing a review.
For comparison, when I put up my Luminary review, I had played the game for over 40 hours, and reached level 38 with my main, plus had two level 20 alts to try out other weapons and skills. After that time I certainly understood how the game fundamentals work, what the game is about, but only for the lower to mid-level of the game. Somebody who would review World of Warcraft after playing it for 40 hours would certainly know how the leveling game functioned, and maybe have done a 5-man instance. But he couldn't possibly know anything about raiding or other end game activities. So his review would necessarily be incomplete.
But imagine the other extreme. Imagine I had not up to now written anything about Warhammer Online, but been busy playing it for 1,000 hours, and just killed Emperor Karl Franz, and done all the parts of a successful Altdorf siege. So I'd write my first WAR review now. Who the hell would be reading that? Sure, I might have a far more complete view of how the whole game works from start to end game. But the interest in a review of last year's games would be close to zero.
There is no question that in 2 hours (he claims 9) Ed Zitron couldn't possibly get more than a fleeting impression. But that is actually *more* time than many people would take to judge a Free2Play game, where the barrier to entry is just the time to download. Of course if you paid $50 for a game you probably play a bit longer before deciding you don't like it and giving up on a game. But I doubt anyone takes 40+ hours to make that decision for himself.
So, how long do you think a reviewer should play a MMO before writing a review? And how long do you play a game before forming an opinion about it?
- posted by Tobold @ 6:47 AM Permanent Link
Comments:
<< Home
Newer› ‹Older
MMOs are very subjective in their nature. However if you write a review and the game is not interesting or fun after the first 10 or so hours I would not play it.
MMOs have one chance to make the game fun and pleasing when people do trials before they buy. If it is not fun, engaging or entertainging after the first 3-10 hours I would not play the game.
Like many say MMOs are never done and are always changing. Writing a review for an MMO is like buying the strategy guide and having it be useful 3-4 months after release.
I like early MMO reviews because they give me a preview of the game and what to expect. Never rely on just one, I take tons for multiple sites and look for patters. If they all say that combat is to slow or have performance problems then I know what to expect.
If the review if just a big rant then I do not take that advice.
MMOs have one chance to make the game fun and pleasing when people do trials before they buy. If it is not fun, engaging or entertainging after the first 3-10 hours I would not play the game.
Like many say MMOs are never done and are always changing. Writing a review for an MMO is like buying the strategy guide and having it be useful 3-4 months after release.
I like early MMO reviews because they give me a preview of the game and what to expect. Never rely on just one, I take tons for multiple sites and look for patters. If they all say that combat is to slow or have performance problems then I know what to expect.
If the review if just a big rant then I do not take that advice.
Some companies supply reviewers with multiple characters (or savegames in single-player games) so that they can review multiple parts of the game with minimal time investment. Considering that Eurogamer was supplied with a pair of accounts for review purposes, I wonder why the same was not done here.
They cannot expect Ed Zitron to play the game till he finally finds something positive. To find some hidden pearls at the bottom of the bucket full of crap.
The idea that a bad first impression gets better over time is pretentious, and the phase of decision making is not longer than the 2-9 hours Ed Zitron needed to review the game.
Good writers and reviewers do not need longer to get everything they need to evaluate a game.
Aventurine just missed to point out that Ed Zitron was right on spot with his review. Whatever redeeming qualities might be hidden in the dark of Darkfall, it has glaring issues that he pointed out.
BTW: Is watching Youtube Darkfall vids and reading blogs not enough to judge Darkfall, from a customer perspective? Do I really have to buy the game and play it to decide that I just wasted my money? :>
Scott Jennings is now playing Darkfall, too. He really paid some money to come to... I wonder what conclusions.
The idea that a bad first impression gets better over time is pretentious, and the phase of decision making is not longer than the 2-9 hours Ed Zitron needed to review the game.
Good writers and reviewers do not need longer to get everything they need to evaluate a game.
Aventurine just missed to point out that Ed Zitron was right on spot with his review. Whatever redeeming qualities might be hidden in the dark of Darkfall, it has glaring issues that he pointed out.
BTW: Is watching Youtube Darkfall vids and reading blogs not enough to judge Darkfall, from a customer perspective? Do I really have to buy the game and play it to decide that I just wasted my money? :>
Scott Jennings is now playing Darkfall, too. He really paid some money to come to... I wonder what conclusions.
If DF has a 14- trail or whatnot I would do that to try the game out. If not then having a different MMO experience based on PvP, Dog Eat Dog, FFA, Sand-Box then DF would be the game for you.
As for Youtube I would never trust it. Go read reviews from the players on fan sites or official forums. Let them tell you what problems the game has (many) and see if it your cup of tea.
As for Youtube I would never trust it. Go read reviews from the players on fan sites or official forums. Let them tell you what problems the game has (many) and see if it your cup of tea.
@Longasc
I think that Lum's reports so far are not the DF bashing some were expecting.
He is pointing out the flaws and pointing out the good.
And he's already played more than that "spot on" reviewer did. And I don't mean TIME logged on. I mean things done in game..
As for the glaring issues he pointed out, will that trolling ever stop? It was proven already that most of his review is based on fallacy/exageration.
Anyway, I can understand someone saying "even if he exagerated I won't risk giving my money to Aventurine." What I cannot understand is saying a review is "spot on" based on blogs and youtube clips.
But maybe you were trying to make an altogether different point that went over my head.
I think that Lum's reports so far are not the DF bashing some were expecting.
He is pointing out the flaws and pointing out the good.
And he's already played more than that "spot on" reviewer did. And I don't mean TIME logged on. I mean things done in game..
As for the glaring issues he pointed out, will that trolling ever stop? It was proven already that most of his review is based on fallacy/exageration.
Anyway, I can understand someone saying "even if he exagerated I won't risk giving my money to Aventurine." What I cannot understand is saying a review is "spot on" based on blogs and youtube clips.
But maybe you were trying to make an altogether different point that went over my head.
I believe I remember reading a while back that in EQ2, where they keep incredibly detailed metrics, the average time-played on accounts that did not renew after installation was less than an hour.
An account of first impressions probably do count a lot more than a considered view of the end-game in terms of deciding whether it's worth trying a game out.
An account of first impressions probably do count a lot more than a considered view of the end-game in terms of deciding whether it's worth trying a game out.
Depends on the type of game.
I played Matrix online for 30minutes and knew without a doubt I wouldn't continue.
I played EQ2 to 10th level before I figured it wasn't going to last. (about 4 hours, hmm)
I played WoW for 1 hour and knew I liked it. (got to 10th level as well)
I played KotoR2 for 2 hours and fell in love.
Game companies shouldn't make people play 3-9 HOURS of crap then tell them they need 30 more hours before they REALLY start liking it.
I played Matrix online for 30minutes and knew without a doubt I wouldn't continue.
I played EQ2 to 10th level before I figured it wasn't going to last. (about 4 hours, hmm)
I played WoW for 1 hour and knew I liked it. (got to 10th level as well)
I played KotoR2 for 2 hours and fell in love.
Game companies shouldn't make people play 3-9 HOURS of crap then tell them they need 30 more hours before they REALLY start liking it.
What actually did Lum ENJOY in Darkfall until May 22nd. He is slightly bemused at the moment, but I have yet to read something more positive than that he is excited about the leather bikini.
I wonder if he will find the raw, unwashed hardcore potential, that intangible quality that makes Darkfall special despite the numerous flaws.
I wonder if he will find the raw, unwashed hardcore potential, that intangible quality that makes Darkfall special despite the numerous flaws.
@Longasc
You're absolutely right.
Now I think that reviews should not be made by just simply playing the game as any other player would have.
Ideally the reviewer should have access to codes that allow him to visit most of the levels or, in case of a mmo, spend a couple of days playing different parts of the game at different levels. It would be in the interest of the devs to allow this.
Of course that isn't possible but then you'll have situations where players who would've loved the later game are put off by a review made in the early game. In WoW, playing the game until level 20, where you enter deadmines (more or less) had nothing to do with playing from 50 to 60 and the leveling game had nothing to do with raiding.
I met some raiders that loved the raiding game but had a visceral hate of the leveling part.
You're absolutely right.
Now I think that reviews should not be made by just simply playing the game as any other player would have.
Ideally the reviewer should have access to codes that allow him to visit most of the levels or, in case of a mmo, spend a couple of days playing different parts of the game at different levels. It would be in the interest of the devs to allow this.
Of course that isn't possible but then you'll have situations where players who would've loved the later game are put off by a review made in the early game. In WoW, playing the game until level 20, where you enter deadmines (more or less) had nothing to do with playing from 50 to 60 and the leveling game had nothing to do with raiding.
I met some raiders that loved the raiding game but had a visceral hate of the leveling part.
It's not that a reviewer can't form an opinion after two hours, but it's an opinion of just two hours of play, and the reviewer needs to be up front about that. Then we readers can read the review with that in mind -- it's based on two hours of play. Or nine hours.
I reviewed AoC and Warhammer for PC Gamer. I put in between 40-60 hours on each game, which I honestly felt like it wasn't quite enough because there were so many parts of the game I didn't experience.
And if you look at the arc both of those games took, the first week of play was met with much enthusiasm by players, but as players got deeper into the games, flaws became more evident.
For example, if you based your review of AoC on Tortage only, it could be a glowing review. If you get out of Tortage and continue playing, and see that the PvP endgame wasn't really in there at release, you need to ding the game a bit. Other things like the math behind the stats in the game being screwy weren't evident for some time.
Same with Warhammer, a more complete game, but a game now that has revealed some deep issues with the endgame, etc.
A two hour review strikes me as a review that is only worth a few paragraphs. "I played this game for two hours and quit because I didn't like it. I didn't like it because...."
I reviewed AoC and Warhammer for PC Gamer. I put in between 40-60 hours on each game, which I honestly felt like it wasn't quite enough because there were so many parts of the game I didn't experience.
And if you look at the arc both of those games took, the first week of play was met with much enthusiasm by players, but as players got deeper into the games, flaws became more evident.
For example, if you based your review of AoC on Tortage only, it could be a glowing review. If you get out of Tortage and continue playing, and see that the PvP endgame wasn't really in there at release, you need to ding the game a bit. Other things like the math behind the stats in the game being screwy weren't evident for some time.
Same with Warhammer, a more complete game, but a game now that has revealed some deep issues with the endgame, etc.
A two hour review strikes me as a review that is only worth a few paragraphs. "I played this game for two hours and quit because I didn't like it. I didn't like it because...."
Makes you wonder whether the reason why "unfinished" releases nearly always have the endgame lacking, not the start, is because they don't want to scare off reviewers and potential customers in the first hour.
As Mark points out, the issue is not HOW long a reviewer played the game, it's about being up-front with that. The problem with the EG trash piece was that Zitron lied about his in-game time. All he needed to do was start the review with "Based on 30 minutes in-game and a few hours in character creation, here are my issues with DF". No one would have thought twice about challenging that 'review'.
How long a reviewer wants/needs to spend with a game is up to the reviewer or his employer, and so long as they are honest and up-front with that info, all good. If I want a quick overview of a game, I'll read an IGN 'review', if I want depth, I'll go to a specific site that caters to the genre (GameFAQs, Massively, ect)
How long a reviewer wants/needs to spend with a game is up to the reviewer or his employer, and so long as they are honest and up-front with that info, all good. If I want a quick overview of a game, I'll read an IGN 'review', if I want depth, I'll go to a specific site that caters to the genre (GameFAQs, Massively, ect)
# posted by 
: 28/5/09 15:06
: 28/5/09 15:06
On the topic... MMOGs are still games. Players still care about things like learning curve, core features, and, above all else, whether the game would be fun for the person reading the review.
In this day and age, a game shouldn't expect to 'delay the fun' for hours and hours. It shouldn't take 10, 15, 50 or 100 hours to get to the fun parts. That's just shoddy design. Usability is important to games; if you can't have fun with a game near-immediately, it's a failure.
For reviewers, they should put in a reasonable amount of time to cover all of the salient features of the game. If the game is all about raiding, the reviewer should raid and discuss that. If the game is about pvp, the reviewer should participate in as many aspects of pvp as are readily available. That's what the reviewer's job is. However, it should absolutely be proclaimed to the heavens about a game's initial learning curve and time to fun. If a game's time to fun factor is too long, I'm gonna pass on it because my time is more valuable than that.
In this day and age, a game shouldn't expect to 'delay the fun' for hours and hours. It shouldn't take 10, 15, 50 or 100 hours to get to the fun parts. That's just shoddy design. Usability is important to games; if you can't have fun with a game near-immediately, it's a failure.
For reviewers, they should put in a reasonable amount of time to cover all of the salient features of the game. If the game is all about raiding, the reviewer should raid and discuss that. If the game is about pvp, the reviewer should participate in as many aspects of pvp as are readily available. That's what the reviewer's job is. However, it should absolutely be proclaimed to the heavens about a game's initial learning curve and time to fun. If a game's time to fun factor is too long, I'm gonna pass on it because my time is more valuable than that.
# posted by : 28/5/09 16:20
: 28/5/09 16:20
I've always considered a game to be too short or incomplete it it takes fewer than 10-12 hours of playtime to complete it. With MMOs, the playtime requirement is much more, but a good feel is garnered in maybe 15-20 hours. I'd say that would be about standard to figure out of the game's worth.
Perhaps traditional game reviews don't work on MMOs and the community should move on to a different way of giving game impressions. It's clear to me that it takes so long to play MMOs "fully" that by the time a substantive review of the whole game comes out, no one would care (as Tobold said). Also, there's a demand for up-to-date impressions on MMOs, because the games change as they age from player and developer action.
My dream reviewing system: The reviewer sits down with the game and plays it to figure out the core game mechanics. What do stats do? How do mobs behave? Are all the quests dull? Is the community helpful? etc. The reviewer than writes a factual summary of what the game offers to players and at what levels. It doesn't have to all be from experience as long as the reviewer is transparent about what he heard and what he did.
Once "the facts" are written, the reviewer starts a new section about what his opinions of the game. If he does the factual part right, most seasoned players won't have to read past it in order to know if they'll like the game or not. The opinion area would be where most people new to MMOs would get their fill.
This proposed system would not focus on the experience of one guy (who may or may not be qualified to give his opinions on a game), but instead what is actually in the game regardless of what the reviewer happens to think about it. Reviews in this style will have a higher chance of relating useful information to seasoned players, while still offering the opinion of the reviewer.
My dream reviewing system: The reviewer sits down with the game and plays it to figure out the core game mechanics. What do stats do? How do mobs behave? Are all the quests dull? Is the community helpful? etc. The reviewer than writes a factual summary of what the game offers to players and at what levels. It doesn't have to all be from experience as long as the reviewer is transparent about what he heard and what he did.
Once "the facts" are written, the reviewer starts a new section about what his opinions of the game. If he does the factual part right, most seasoned players won't have to read past it in order to know if they'll like the game or not. The opinion area would be where most people new to MMOs would get their fill.
This proposed system would not focus on the experience of one guy (who may or may not be qualified to give his opinions on a game), but instead what is actually in the game regardless of what the reviewer happens to think about it. Reviews in this style will have a higher chance of relating useful information to seasoned players, while still offering the opinion of the reviewer.
I can personally tell whether I like a game or not within the first hour, and based on that whether to continue to play or not.
However, there is a difference if one is getting paid to write an actual review. I believe that a paid reviewer should spend more time at a game and try their best to dig a little deeper, even if they themselves personally felt that the game just wasn't that fun... for them.
Obviously a lot of people play Darkfall and like it. So the game cannot be a total failure. Any good reviewer, despite their own personal feelings, would have tried to get to the bottom of the issue and find out why those players like it.
However, there is a difference if one is getting paid to write an actual review. I believe that a paid reviewer should spend more time at a game and try their best to dig a little deeper, even if they themselves personally felt that the game just wasn't that fun... for them.
Obviously a lot of people play Darkfall and like it. So the game cannot be a total failure. Any good reviewer, despite their own personal feelings, would have tried to get to the bottom of the issue and find out why those players like it.
A good game needs to be able to grab you in the first fifteen minutes, if not less. You need to know how it works, and whether or not you'll enjoy doing the same sort of thing for dozens of hours. If the game sucks in that initial impression, it's fair to say that something in the game is broken.
Maybe the endgame is fantastic, but if you have to suffer through stupid design to get there, why bother, especially if you're paying a subscription?
I like what Guild Wars did by letting players make a level capped character from the outset, limiting them to certain content. It's a great way for players to sample both the "noob" part of the game, and the high end in a fairly short time.
This is especially important as designers keep obsessing with the incredibly stupid notion that "the game starts at the level cap". Don't make players suffer through grindy design by the numbers if you want them to have fun with the stuff you put at the end of the tunnel. That's just poor design and poor PR, and if the tunnel itself isn't any fun, people have every right to call you out on it.
Maybe the endgame is fantastic, but if you have to suffer through stupid design to get there, why bother, especially if you're paying a subscription?
I like what Guild Wars did by letting players make a level capped character from the outset, limiting them to certain content. It's a great way for players to sample both the "noob" part of the game, and the high end in a fairly short time.
This is especially important as designers keep obsessing with the incredibly stupid notion that "the game starts at the level cap". Don't make players suffer through grindy design by the numbers if you want them to have fun with the stuff you put at the end of the tunnel. That's just poor design and poor PR, and if the tunnel itself isn't any fun, people have every right to call you out on it.
The biggest problem with reviewing MMOs is simply the sheer scale. I could write a neat little review of most games after having played and completed them in 20 or 40 hours, but that amount of time put into an MMO is just the tip of the iceberg. Even if you put the time in to completely level up your character, it's likely that you're playing in a world with several different classes / races / crafting professions and so you still haven't seen everything the game has to offer. It is simply impossible to experience all of the content in an MMO the way that you might be able to do in, for example, a racing game.
As others have mentioned, the solution to this in reviews is being up front with how far the reviewer got in the game, and what paths he or she chose to play. Since the review cannot possibly be about all the content in the game, let's understand what section of content in the game the review is actually about.
As others have mentioned, the solution to this in reviews is being up front with how far the reviewer got in the game, and what paths he or she chose to play. Since the review cannot possibly be about all the content in the game, let's understand what section of content in the game the review is actually about.
# posted by : 29/5/09 05:15
: 29/5/09 05:15
I think the reason there are often endgame issues in newly released games is that these things are so hard to test. A mature endgame really needs a community of max level players who know the game fairly well.
I remember in WAR, we had a blast in tier 3/4 keep takes in the beta, an absolute blast. The flaws didn't become clear until later and part of it is because we were playing beta with a different mindset to a permanent player. I blame Mythic for many of the issues, but I really think they did try to test parts of the endgame at least.
And for time played: as an experienced MMO player I'd expect to get the basics of a game and be able to communicate it after an hour or two. I mean by that, figure out what it's /about/. Plus you can always read forums and blogs to see what people who have played longer have to say.
I remember in WAR, we had a blast in tier 3/4 keep takes in the beta, an absolute blast. The flaws didn't become clear until later and part of it is because we were playing beta with a different mindset to a permanent player. I blame Mythic for many of the issues, but I really think they did try to test parts of the endgame at least.
And for time played: as an experienced MMO player I'd expect to get the basics of a game and be able to communicate it after an hour or two. I mean by that, figure out what it's /about/. Plus you can always read forums and blogs to see what people who have played longer have to say.
# posted by : 29/5/09 07:02
: 29/5/09 07:02
This reminds me of the movie reviewer, Roger Ebert. He walked of a movie only once and wrote a review about it. And even though he mentioned that he walked out in his review he didn't feel very comfortable about it. Reviewing a movie or book you haven't completed is just not done.
With games, it's a lot harder. A magazine has to review recent games and usually only has a few weeks for each review. That's enough to finish most single player games but nowhere near enough for an mmorpg. I think reviewers should mention how far they got. "I've played a horde shaman to level 40". "I've played through about half of Baldur's Gate". That will give the audience at least an idea how far in the game they got. Alternatively you can split your review. Give your first impressions in one review, play it a month and then write an addendum.
With games, it's a lot harder. A magazine has to review recent games and usually only has a few weeks for each review. That's enough to finish most single player games but nowhere near enough for an mmorpg. I think reviewers should mention how far they got. "I've played a horde shaman to level 40". "I've played through about half of Baldur's Gate". That will give the audience at least an idea how far in the game they got. Alternatively you can split your review. Give your first impressions in one review, play it a month and then write an addendum.
Another interesting question is what happens to the author of a bad review if both the designers and fans of the reviewed game disagree... :P
I usually give a game, any game, a weekend before I either like or dislike. Or like a lot and finish it without playing much else in between for offline games...
Some games just hit a nerve and I'm hooked right away. Dawn of War, Lotro, WoW (back when it was my first MMO), Forwa Motorsport, Kotor come to mind...
Whether that lasts is another question, but the first two days well... If I don't like a game after two days, I quit. And the game doesn't get to redeem itself after that. Back when I was young, we didn't have as much choice between games and a mediocre one? You'd finish it anyway. But now we have l'embarras du choix. A game that is only so-so gets dropped. A good game gets played in bursts. And only a real gem, in my opinion, will ever see me reach the end. Or the level cap...
Some games just hit a nerve and I'm hooked right away. Dawn of War, Lotro, WoW (back when it was my first MMO), Forwa Motorsport, Kotor come to mind...
Whether that lasts is another question, but the first two days well... If I don't like a game after two days, I quit. And the game doesn't get to redeem itself after that. Back when I was young, we didn't have as much choice between games and a mediocre one? You'd finish it anyway. But now we have l'embarras du choix. A game that is only so-so gets dropped. A good game gets played in bursts. And only a real gem, in my opinion, will ever see me reach the end. Or the level cap...
And some games, like Darkfall as an excellent example, only take a publicity blurb to be turned down... I KNOW I wouldn't like it. So why bother?
I think it depends on the game - the review should match the typical player experience. I.e., simple & free games are going to be evaluated by the customer in a shorter time frame is my guess. How many people say "I'm going to play EVE Online for 30 minutes and decide if I will continue"?
And I see there being two kinds of reviews - the "here is what you will encounter before you make your go/no-go decision and some pointers to ease the way" and the "here is the end-game, how long to get there, how different it is from leveling, etc."
And I see there being two kinds of reviews - the "here is what you will encounter before you make your go/no-go decision and some pointers to ease the way" and the "here is the end-game, how long to get there, how different it is from leveling, etc."
I'm a day late, dollar short, etc.
One thing I like is when sites review MMOs with a blog of what's going on. I think that's the best way to get a feel for a game, especially since they change over time. The experience someone has during the week of launch may be completely different than what happens a month later.
By doing blogging, you can get the word out there quickly, while still allowing for the reviewer to show how their experience changes. It eliminates the question of "how long is long enough?"
Post a Comment
One thing I like is when sites review MMOs with a blog of what's going on. I think that's the best way to get a feel for a game, especially since they change over time. The experience someone has during the week of launch may be completely different than what happens a month later.
By doing blogging, you can get the word out there quickly, while still allowing for the reviewer to show how their experience changes. It eliminates the question of "how long is long enough?"
<< Home
