www.Tobold.com
Tobold@GMail.com
A blog with my thoughts regarding games I am playing and other stuff in life. Please read my Terms of Service
RSS Feed
Wednesday, March 19, 2008
Beta Leaks
The issues we discuss around MMORPGs are often similar, if smaller in scale, than the issues of the real world. One hotly discussed issue of the real world is leaks and whistle blowing: should people reveal secrets if they think that the revelation is in the best general interest, and should journalists report those leaks? If a chemical company is poisoning the ground water, Enron is fiddling their accounts, or the CIA is falsifying data on WMD or torturing people, would it be better to reveal that, or should they be allowed to keep those things secret? The equivalent subject in MMORPGs is leaking information from beta tests: If a game sucks, should beta testers leak that information, or should they keep mum and let the company make money by selling lots of games on the day of release before people realize how bad it is? Vanguard apparently sold over 200,000 copies before crashing to under 50,000 subscribers, should beta testers have warned potential players earlier?
It is easy to claim that "you signed an NDA, you shouldn't talk about that beta". Obviously the people who alerted the authorities and press about the Enron account fiddling also had a contract with Enron forbidding them to pass on secret information about the company. Making a bad game is not as bad a crime as ruining your investors, but then an NDA is not as strong a contract as an employment contract. Both Age of Conan and Warhammer Online have been announced for May, just two months from now, and unless they are postponed there isn't much time left before release, and neither game has even announced an open beta yet. Either there will be no open beta at all, or a rather short one, with very little opportunity to spread the word about possible flaws in these games. But as especially in MMORPGs the drive to be in the game right from the start is strong, that can lead to many people buying the game without being able to test it before, and then being disappointed, having wasted $50+ on a flawed or incomplete game.
A reader alerted me to a site called Beta Leaks, which does exactly what it says in the title: it is a forum for leaking information from MMORPG betas. And I was reading a thread called "AoC Beta Leak FAQ" there which gave me a lot of information I'm sure Funcom doesn't want me to have, and especially not write about. Now I never signed an NDA with Funcom, but that information is obviously tainted by coming from somebody else's breach of his NDA. So I'm not going to report it. But although I'm sure I'll get another one of those idiot comments that me linking to that beta leaks site is already breaking a taboo, I would at least like to start a discussion here about how legitimate beta leaks are. The people who leak obviously think that it is in the better interest of the players to have that sort of information, while the game companies obviously think this stuff shouldn't be public knowledge. Where do you stand in that? Could you imagine to ever leak information from a beta, if you thought the game was really, really unacceptable? With companies buying favorable reviews from bigger game sites, at which point does revealing information become a legitimate guerilla tactic instead of just a breach of contract?
- posted by Tobold @ 8:53 AM Permanent Link
Comments:
Could you imagine to ever leak information from a beta, if you thought the game was really, really unacceptable? With companies buying favorable reviews from bigger game sites, at which point does revealing information become a legitimate guerilla tactic instead of just a breach of contract?
Personally, I could never leak actual information about a game that I was participating in a beta for having signed an NDA.
On whether it can ever become a legitimate 'guerilla advertising' tactic, I would say no. When you electronically sign an NDA, you are agreeing to abide by its terms. You are essentially employed as a volunteer by the company, and are bound by contract. Yes, the worst thing they can do is likely cancel your beta access, and refuse you further access to their beta. Is that so bad? I suppose it depends on the individual.
One thought I've had beyond this though is stance some companines have taken on not allowing bloggers into their betas. I beleive WAR has taken this stance, as well as AoC. The massive amounts of leaks through blogs drive the NDAs, and the hushed releases that we're seeing now. Too many times has a game been burned by loud mouthed individuals who can't tell the difference between a finalized product and the beta.
It is common sense now that you cant really do a beta without proper process. This process needs to deliver the promise of quality to the testers, even if it is not currently present.
This, IMO, is garbage. The purpose of a beta test is not to provide quality. It is a process to ensure quality in the final product. If you go into a beta test expecting anything other than bugs and issues, then you're looking for a free trial, not a beta.
Blizzard's high standards of quality(Yes, I'm using Blizzard. I've yet to test a WoW build that's been released to public for test that was extremely buggy and completely unplayable) have set the bar far above what people should be expecting. I don't know if this is good or bad, but I do know that they've done a good job getting stuff out the door, 90% of the time with minimal problems that have actual impact.
If companies don't want beta leaks, then they should spend the money to have a good game before beta starts. i hate going into beta and seeing crap while the devs are in articles talking about how good their stuff is working.
Then don't go into a beta expecting a finished product. Beta isn't a free trial, and it's not a demo. Beta is for testing so that the game can become a finished product.
Yes, the developers are going to do nothing but talk up the positive. Wouldn't you? Would you honestly make a game, and go to the news outlets and say "Right now, our game sucks. The combat engine is broken, the marketplace is broken, and you can't play more than 15 minutes at time without the OS crashing down around you." Seriously here, you need some more realistic expectations if you think that ANY game company is going to say that in public.
<< Home
Newer› ‹Older
Unless I get into a Beta, I hardly ever take heed of previewed information. It's all marketing or hype from people in the beta who like one aspect or the other of the game.
I remember when Dungeons & Dragons Online was released. I read the previews but was swayed neither here nor there by any online info. Had I played the beta though, I definitely would not have bought the game (Turbine bah!) or at least not straight away.
I do think that online games should have 15 day trials as standard. It's silly that there are online games that have been going on for years now where you cannot test the game out before purchasing a serial key in a box on a gamestore's shelf.
Leaking beta information is not the same as whistleblowing though - if it was, those people who did leak beta information would never be able to subscribe to any online game ever again (the cost of whistleblowing in an industry).
I remember when Dungeons & Dragons Online was released. I read the previews but was swayed neither here nor there by any online info. Had I played the beta though, I definitely would not have bought the game (Turbine bah!) or at least not straight away.
I do think that online games should have 15 day trials as standard. It's silly that there are online games that have been going on for years now where you cannot test the game out before purchasing a serial key in a box on a gamestore's shelf.
Leaking beta information is not the same as whistleblowing though - if it was, those people who did leak beta information would never be able to subscribe to any online game ever again (the cost of whistleblowing in an industry).
I don't want to make a comment about the beta leaks but rather on the comment that WAR will be released in May, I think that caught a lot of peoples eyes.
Is this really true?
Is this really true?
# posted by 
: 19/3/08 11:22
: 19/3/08 11:22
I don't want to make a comment about the beta leaks but rather on the comment that WAR will be released in May, I think that caught a lot of peoples eyes. Is this really true?
It is still the tentative official release date, we just don't know if they'll actually make it. Latest news from this month were, quote: "Jacobs once again reiterated Mythic's Q2 target, but added that the game's release date "is really set in mud, not stone, due to the complexity of these games.""
And I'm not slinging mud at WAR here. :)
It is still the tentative official release date, we just don't know if they'll actually make it. Latest news from this month were, quote: "Jacobs once again reiterated Mythic's Q2 target, but added that the game's release date "is really set in mud, not stone, due to the complexity of these games.""
And I'm not slinging mud at WAR here. :)
are you supporting NDA breaker by posting this link ?
i like the new tobold , more angry more realistic and not pretending to be a 'nice' blogger. its about time this blog getting a bit realistic.
i like the new tobold , more angry more realistic and not pretending to be a 'nice' blogger. its about time this blog getting a bit realistic.
# posted by : 19/3/08 14:16
: 19/3/08 14:16
I know that when the mega leak for AoC first surfaced it was mainly negative. Then that guy got cease and desisted like a newb and I guess he released more. The newer stuff he released is mostly all positive and his only gripe really is that he gets ganked in open pvp, lols.
Here's a link if you wanna check it out (or haven't seen it yet)
http://betaleaks.com/forums/forumdisplay.php?f=2
Here's a link if you wanna check it out (or haven't seen it yet)
http://betaleaks.com/forums/forumdisplay.php?f=2
First open gold farmer support, and now breaking NDAs? Whats next, 'Tobolds guide to cooking crack'?
On a serious note, that only real issue with breaking an NDA is that often the players are given an incomplete or limited version of the game, so many reported issues are somewhat intentional in a beta. Open beta is clearly different, but often closed beta will have certain features removed to focus testing. All it takes is one bad apple to make a long post about how so-and-so is broken to start spreading rumors, and soon enough those rumors get takes as fact.
That said, a game that is 1-2 months from release is not likely to undergo massive changes, so if its a wreck a month from release, clearly it has issues.
On a serious note, that only real issue with breaking an NDA is that often the players are given an incomplete or limited version of the game, so many reported issues are somewhat intentional in a beta. Open beta is clearly different, but often closed beta will have certain features removed to focus testing. All it takes is one bad apple to make a long post about how so-and-so is broken to start spreading rumors, and soon enough those rumors get takes as fact.
That said, a game that is 1-2 months from release is not likely to undergo massive changes, so if its a wreck a month from release, clearly it has issues.
# posted by : 19/3/08 14:29
: 19/3/08 14:29
I don't think that I would publicly tell anyone if a game was a wreck a month from launch. I don't feel like I'm responsible for making sure other people make wise purchases.
However, I would certainly tell my family/friends over dinner the ways that the game was broken.
However, I would certainly tell my family/friends over dinner the ways that the game was broken.
First open gold farmer support, and now breaking NDAs? Whats next, 'Tobolds guide to cooking crack'?
Isn't 'Tobold's guide to playing WoW' the same thing? :)
But more seriously, when does constituting posting a link somewhere "support"? The alternative would be reporting about a website without posting a link, and even then you would have to hide the actual name of the site so people can't google it. Shouldn't I be able to trust my readers to not follow links if they find the content objectionable? I am very much against the method of "let's solve these problems by not talking about them".
Isn't 'Tobold's guide to playing WoW' the same thing? :)
But more seriously, when does constituting posting a link somewhere "support"? The alternative would be reporting about a website without posting a link, and even then you would have to hide the actual name of the site so people can't google it. Shouldn't I be able to trust my readers to not follow links if they find the content objectionable? I am very much against the method of "let's solve these problems by not talking about them".
As far as AoC goes, almost anyone could of gotten in on the technical betas they've had. That is all I needed to know that AoC is going to be crap for a PC game, but may be OK for the console crowd.
Some companies don't want beta testers talking about their game because it sucks, but most of the time they probably just don't want some kid whose only MMO experience is WoW during its later release years seeing an early edition of their game and posting that it is "totally ghey" because it simply isn't a finished product.
As a player you also have to keep in mind that the player, and subsequently the tester, base has expanded, probably to include people you totally don't agree with. I was playing WoW and heard about how much EVE sucked, but then I tried it, and liked it. Now everyone in that game says WoW sucks. So I don't decide to play or to avoid a game based on anything short of trying it out myself, along with the opinions of a select few who I know have similar tastes.
Open betas are the new trial period for a game. Companies really shouldn't go to open beta with a product that they wouldn't be happy releasing without some minor changes. Example, Tabula Rasa. I heard it might be fun now. Oh well, I'm not going to pay money to find out because the risk isn't worth it based on what I saw in beta.
As a player you also have to keep in mind that the player, and subsequently the tester, base has expanded, probably to include people you totally don't agree with. I was playing WoW and heard about how much EVE sucked, but then I tried it, and liked it. Now everyone in that game says WoW sucks. So I don't decide to play or to avoid a game based on anything short of trying it out myself, along with the opinions of a select few who I know have similar tastes.
Open betas are the new trial period for a game. Companies really shouldn't go to open beta with a product that they wouldn't be happy releasing without some minor changes. Example, Tabula Rasa. I heard it might be fun now. Oh well, I'm not going to pay money to find out because the risk isn't worth it based on what I saw in beta.
# posted by : 19/3/08 15:05
: 19/3/08 15:05
I find the comparison of beta leaks to whistleblowing a bit overblown, although I guess I get the analogy. It's just, an NDA is something the beta testers have agreed to in order to test a game. The stakes for whistleblowing are much higher and involve information about an organization's illegal activities. Revealing information about a game is not remotely on the same level.
That was a bit disjointed, so what I'm saying is: whistleblowers should whistleblow when someone is breaking a law, as the depth of that public service justifies breaking their word to their organization. Beta leakers should not break their NDAs, unless perhaps the game company is just completely lying about their product. The public service a beta leaker provides is generally paltry and not worth breaking their word over.
That was a bit disjointed, so what I'm saying is: whistleblowers should whistleblow when someone is breaking a law, as the depth of that public service justifies breaking their word to their organization. Beta leakers should not break their NDAs, unless perhaps the game company is just completely lying about their product. The public service a beta leaker provides is generally paltry and not worth breaking their word over.
# posted by : 19/3/08 15:53
: 19/3/08 15:53
But more seriously, when does constituting posting a link somewhere "support"? The alternative would be reporting about a website without posting a link, and even then you would have to hide the actual name of the site so people can't google it. Shouldn't I be able to trust my readers to not follow links if they find the content objectionable? I am very much against the method of "let's solve these problems by not talking about them".
Well for one thing, actually providing the link helps to drive traffic more than simply mentioning the name. Even if 99% of that traffic has no interest in actually making a purchase, if that 1% gets sold after looking at the site, you contributed 1% to that gold farmer, even if it is only in a small way.
Which is not to say I think it was a mistake to do the interview, I found it overall interesting, but I wonder why you had to provide a link in addition to mentioning the name. If someone goes to the trouble of googling the sites name, they are far more serious about buying gold than someone who has perhaps had limited exposure to it, and only got the real details after following the link from your blog.
But in the end, it's your blog, so you could have a huge banner ad at the top saying "Tobold says buy gold now!", and it would still not be 'wrong'. The blog is your opinion, and your readers come here because they find your opinion interesting. The only way they can voice their objection to gold farmer support is to stop viewing the blog. And like you and most other bloggers have said (myself included), we blog for us, and if people read it, great. We don't make money one way or the other, and we certainly don't make posts to please anyone but ourselves.
Re-reading that last sentence, I basically just said blogging is internet masturbation...
Well for one thing, actually providing the link helps to drive traffic more than simply mentioning the name. Even if 99% of that traffic has no interest in actually making a purchase, if that 1% gets sold after looking at the site, you contributed 1% to that gold farmer, even if it is only in a small way.
Which is not to say I think it was a mistake to do the interview, I found it overall interesting, but I wonder why you had to provide a link in addition to mentioning the name. If someone goes to the trouble of googling the sites name, they are far more serious about buying gold than someone who has perhaps had limited exposure to it, and only got the real details after following the link from your blog.
But in the end, it's your blog, so you could have a huge banner ad at the top saying "Tobold says buy gold now!", and it would still not be 'wrong'. The blog is your opinion, and your readers come here because they find your opinion interesting. The only way they can voice their objection to gold farmer support is to stop viewing the blog. And like you and most other bloggers have said (myself included), we blog for us, and if people read it, great. We don't make money one way or the other, and we certainly don't make posts to please anyone but ourselves.
Re-reading that last sentence, I basically just said blogging is internet masturbation...
# posted by : 19/3/08 15:59
: 19/3/08 15:59
It is common sense now that you cant really do a beta without proper process. This process needs to deliver the promise of quality to the testers, even if it is not currently present.
If you make an mmorpg and cant hande a positive beta process then you might be in the wrong industry.
If you make an mmorpg and cant hande a positive beta process then you might be in the wrong industry.
# posted by : 19/3/08 16:24
: 19/3/08 16:24
Beta leaks are only serving a demand for it and that service changed over the last 10 years. Right now 80% of all beta "leaks" are written to get attention. Many are semi-leaks, written from summarizing information collected elsewhere, rather than playing the actual beta. NDA breaks changed just the way as companies changed the way they used betas. What sounds stupid actually makes sense. When you have studios publicly advertising for beta spots, you know this is pure fuel for the hype train, rather than involving actual testing. People break more and more NDAs cause their tools of the process. If you want quality testing, even large scale, you do not open up a freaking online signup for it. This way you only lure in the low quality testing material.
Back then, when the whole term blogosphere wasn't even invented, you knew your spots for quality beta leakage: IRC channels. Those leaks were quality material for an audience, that cared about the product, rather than trying to get attention just for leaking information. Information stayed inside those channels. Right now NDAs are futile. If you want to get information, your will find a way, no matter how closed the beta is.
I blame the most recent leaks aka those dozens of WAR and AoC blow ups, to companies that tried to play with fire, but got their hands burned. You can get more attention right now writing a phony WAR "leak" than writing about the millioth class balance topic for WoW or something. You have tons of WoW fanboys that come up with this stuff and even if those contain some truth and those games do suck, the studios damaged their game even more than without an NDA, cause there isn't a leak that praises those examples, only negative ones spreading the bad word of mouth for the product. One of many WoW lessons is to drop the NDA early.mk
Back then, when the whole term blogosphere wasn't even invented, you knew your spots for quality beta leakage: IRC channels. Those leaks were quality material for an audience, that cared about the product, rather than trying to get attention just for leaking information. Information stayed inside those channels. Right now NDAs are futile. If you want to get information, your will find a way, no matter how closed the beta is.
I blame the most recent leaks aka those dozens of WAR and AoC blow ups, to companies that tried to play with fire, but got their hands burned. You can get more attention right now writing a phony WAR "leak" than writing about the millioth class balance topic for WoW or something. You have tons of WoW fanboys that come up with this stuff and even if those contain some truth and those games do suck, the studios damaged their game even more than without an NDA, cause there isn't a leak that praises those examples, only negative ones spreading the bad word of mouth for the product. One of many WoW lessons is to drop the NDA early.mk
# posted by : 19/3/08 16:37
: 19/3/08 16:37
If companies don't want beta leaks, then they should spend the money to have a good game before beta starts. i hate going into beta and seeing crap while the devs are in articles talking about how good their stuff is working.
I don't mind leaks that speak to that but I do not want to read leaks of people just complaining because they don't like the game. Therefore I'm always careful as to how much I believe any leak.
I don't mind leaks that speak to that but I do not want to read leaks of people just complaining because they don't like the game. Therefore I'm always careful as to how much I believe any leak.
# posted by : 19/3/08 16:43
: 19/3/08 16:43Could you imagine to ever leak information from a beta, if you thought the game was really, really unacceptable? With companies buying favorable reviews from bigger game sites, at which point does revealing information become a legitimate guerilla tactic instead of just a breach of contract?
Personally, I could never leak actual information about a game that I was participating in a beta for having signed an NDA.
On whether it can ever become a legitimate 'guerilla advertising' tactic, I would say no. When you electronically sign an NDA, you are agreeing to abide by its terms. You are essentially employed as a volunteer by the company, and are bound by contract. Yes, the worst thing they can do is likely cancel your beta access, and refuse you further access to their beta. Is that so bad? I suppose it depends on the individual.
One thought I've had beyond this though is stance some companines have taken on not allowing bloggers into their betas. I beleive WAR has taken this stance, as well as AoC. The massive amounts of leaks through blogs drive the NDAs, and the hushed releases that we're seeing now. Too many times has a game been burned by loud mouthed individuals who can't tell the difference between a finalized product and the beta.
It is common sense now that you cant really do a beta without proper process. This process needs to deliver the promise of quality to the testers, even if it is not currently present.
This, IMO, is garbage. The purpose of a beta test is not to provide quality. It is a process to ensure quality in the final product. If you go into a beta test expecting anything other than bugs and issues, then you're looking for a free trial, not a beta.
Blizzard's high standards of quality(Yes, I'm using Blizzard. I've yet to test a WoW build that's been released to public for test that was extremely buggy and completely unplayable) have set the bar far above what people should be expecting. I don't know if this is good or bad, but I do know that they've done a good job getting stuff out the door, 90% of the time with minimal problems that have actual impact.
If companies don't want beta leaks, then they should spend the money to have a good game before beta starts. i hate going into beta and seeing crap while the devs are in articles talking about how good their stuff is working.
Then don't go into a beta expecting a finished product. Beta isn't a free trial, and it's not a demo. Beta is for testing so that the game can become a finished product.
Yes, the developers are going to do nothing but talk up the positive. Wouldn't you? Would you honestly make a game, and go to the news outlets and say "Right now, our game sucks. The combat engine is broken, the marketplace is broken, and you can't play more than 15 minutes at time without the OS crashing down around you." Seriously here, you need some more realistic expectations if you think that ANY game company is going to say that in public.
So last week you advertise an RMT vendor that encourages breaking EULAs, and this week you advertise a forum that encourages posting content in violations of NDAs?
Whether this is a pattern or happenstance can be an exercise for the reader.
Whether this is a pattern or happenstance can be an exercise for the reader.
# posted by : 19/3/08 21:13
: 19/3/08 21:13
tobold
dont support NDA breaker, just remove the link from your blog. its not polite and game companies will look at your blog and see you supporting NDA breaker by spreading the link. just like providing link to porn site.. even if you dont watch porn you still support porn by linking em
reconsider please
dont support NDA breaker, just remove the link from your blog. its not polite and game companies will look at your blog and see you supporting NDA breaker by spreading the link. just like providing link to porn site.. even if you dont watch porn you still support porn by linking em
reconsider please
# posted by : 20/3/08 04:51
: 20/3/08 04:51
"its not polite and game companies will look at your blog and see you supporting NDA breaker by spreading the link. just like providing link to porn site.. even if you dont watch porn you still support porn by linking em"
You have got to be retarded or something.
You have got to be retarded or something.
# posted by : 20/3/08 08:14
: 20/3/08 08:14
Do you really think Tobold linking the website's URL is any big deal?
Protip: If someone is interested in looking at Beta Leaks, the first listing in a google search is, omg, http://betaleaks.com/.
Protip: If someone is interested in looking at Beta Leaks, the first listing in a google search is, omg, http://betaleaks.com/.
Hi Tobold,
Long time reader, long time lurker.
Summary for the tldr crowd: Game companies know that their product will be held up for scrutiny as soon as the first person is let into the open beta. Therefore, whether or not breaking the NDA is bad or not is largely irrelevant.
I think most companies are very, very aware that as soon as they start adding non-handpicked people (those who are not professional testers and/or related to the game in some way), the realities of the game will be in the public eye. Randomly selected beta tests are a bell that cannot be unrung.
This being the case, a smart company doesn't even invite random beta testers until they have a product that they are mainly happy and comfortable with. The first closed beta that is still open to any person with an email address is very much a public one, regardless of the presence or absence of an NDA. This is not Mythic's first MMO, so they are highly aware that their game came under NDA-breaking scruity as soon as they let in John Doe, Jane Doe, and other people they have never met or seen and have no control over.
Anyone who frequents gaming sites, even if they actively try to avoid leaks, at least hears rumblings and musings of NDA-wrapped information.
Game companies are aware that:
1.) Someone will leak something, regardless of whatever NDA the company puts on it, and
2.) All it takes is one leak and things can no longer be hidden.
So therefore, MMO developers don't/shouldn't allow ANYONE into an open beta unless they are OK with the game as it stands being held up for scruitiny. Yes, I know that I sidestepped the original question, but I don't think it really matters in the long run if it is right or wrong on an individual basis.
Long time reader, long time lurker.
Summary for the tldr crowd: Game companies know that their product will be held up for scrutiny as soon as the first person is let into the open beta. Therefore, whether or not breaking the NDA is bad or not is largely irrelevant.
I think most companies are very, very aware that as soon as they start adding non-handpicked people (those who are not professional testers and/or related to the game in some way), the realities of the game will be in the public eye. Randomly selected beta tests are a bell that cannot be unrung.
This being the case, a smart company doesn't even invite random beta testers until they have a product that they are mainly happy and comfortable with. The first closed beta that is still open to any person with an email address is very much a public one, regardless of the presence or absence of an NDA. This is not Mythic's first MMO, so they are highly aware that their game came under NDA-breaking scruity as soon as they let in John Doe, Jane Doe, and other people they have never met or seen and have no control over.
Anyone who frequents gaming sites, even if they actively try to avoid leaks, at least hears rumblings and musings of NDA-wrapped information.
Game companies are aware that:
1.) Someone will leak something, regardless of whatever NDA the company puts on it, and
2.) All it takes is one leak and things can no longer be hidden.
So therefore, MMO developers don't/shouldn't allow ANYONE into an open beta unless they are OK with the game as it stands being held up for scruitiny. Yes, I know that I sidestepped the original question, but I don't think it really matters in the long run if it is right or wrong on an individual basis.
Open Beta's are not covered by NDAs so that kinda makes everything you just typed pointless.
NDA's cover closed beta's because the game isnt finished and the developers dont want information about broken stuff or stuff that may not even make it into the game leaked.
If developers waited until the game was working before they released it to beta as somone above suggested, then it would defeat the whole point of closed beta.
Post a Comment
NDA's cover closed beta's because the game isnt finished and the developers dont want information about broken stuff or stuff that may not even make it into the game leaked.
If developers waited until the game was working before they released it to beta as somone above suggested, then it would defeat the whole point of closed beta.
# posted by : 12/8/08 21:31
: 12/8/08 21:31<< Home
