Tobold's Blog
Wednesday, November 16, 2011
Beta isn't what it used to be

I am not in the Star Wars: The Old Republic beta. But I did get a mail telling me to not get a beta code from sites like IGN, because I would get into the beta at the same time as the people who got a beta key from those sites anyway. And now Bioware revealed that everybody who signed up for the beta before 11/11/11 would get in, which is millions and millions of people.

Gazimoff from Mana Obscura writes about how beta testing has changed, moving away from bug reports and using heat maps and data mining instead. If a large percentage of beta testers don't finish a specific quest, that tells you more than sifting through bug reports complaining about quests, many of which might be perfectly working as intended. Furthermore I can totally see the interest of Bioware to want to test if their servers can withstand millions of people, seeing how this is their first MMORPG and problems in that area are to be expected.

What is a bit weird is keeping up the NDA while having so many testers. Chris from Levelcapped points out that if you tell a secret to millions, it isn't a secret any more. Thus keeping information away from possible competitors can't be the reason for the SWTOR NDA. The general assumption is that Bioware marketing is using the NDA to better control either the hype or the negative opinions on the game. I'd say that strategy is backfiring. Some people simply don't care about the NDA, because they think that nothing bad will happen to them if they break it. Others use the fact that the NDA is still up to cast suspicion upon the game.

One thing which is certain is that the meaning of the word "beta" has changed. Google's GMail was in beta for 5 years. Minecraft is being "released" this week. Allods Online was in beta until version 2.0, with people already paying for items from the item shop long before "release". And many other MMORPGs obviously used betas as both marketing tool and server stress test tool. I've been in betas which didn't even *have* a bug reporting tool, bug reporting in betas is so last century.

If everyone that wants a beta code can get one straight from Bioware, what's the point of IGN giving them out as prizes?
These are what were called "marketing betas."

You have alphas and internal Betas and Betas. They are about finding bugs.

Then you let a lot of people have access to the code when it is basically too late to change any design and only fix crash sort of bugs. They feel special since they got into the Beta. They got to see the software before the hop polloi.

The NDA allows Bioware to control the volume of the free press. I.e., they got a large amount of free press for last Fansite Summit. Then when the NDA lifts for the rest of what they were told, there will be another round of posts. Whereas without the NDA, all that would have been covered is the new stuff.
I was going to request TOR beta access, until I read the terms of the beta (fairly standard fare). I read about the NDA and immediately started looking for the 'delete account' button.
Six weeks before release and the only thing you can do is acknowledge that there is a beta and that you're in it? Incredibly suspicious to me.
This wasn't a bug-beta, it was a sneak peek aimed at people like me who are considering whether or not to subscribe. In my case, I was very on-the-fence before the weekend and I preordered it on Monday.

Still playing WoT while I wait. :)
Like Hagu indicates, it seems this relates to control of the information flow.

If the BioWare confidentiality clause is worded anything like they usually are, any information that is already legitimately in the public domain is excluded from the secrecy commitment. In other words, if Palpateen and LurkSkywalker reveal everything on their blogs that are read by two people and their mother, then IGN and Eurogamer are free to do so as well.

I think BioWare couldn't care less about small fry publishing their experiences on the net, but I expect that they do care if the big guns start writing about it "prematurely".
It's good marketing and it effectively allows Bioware to do some stress testing on what will certainly be the largest day one mmorpg launch ever.

I was on the fence about SW:ToR prior to last weekend but no longer. This will be a day one purchase for me.
Looks like a good opportunity to try it out and see if it's a possible buy.

Although I'm afraid that the client will be a 20 gig download for something I can play for a few days.
I'm currently in the Beta for SWTOR and thats all I can say related to that, though I'm framing it this way so people know that I'm not just pulling something out of nowhere.

I feel that a major part of the NDA is the core component of the game - Story. Whilst, sure, theres lots of information you'd like to know (which can be found anyway, regardless, but you have to look for it of your own accord), theres a huge part of the game based around telling a story.

Seriously, some parts of it are so awesome and have such interesting outcomes that can even change depending on what you do etc etc, even a small review about something could potentially ruin an entire story arc for you.

I mean, I dont support a ridiculously draconian NDA still in place till release, but it would become nigh on impossible to avoid pretty "game ruining" - if you're playing SWTOR for the story - spoilers otherwise.
I'm not sure how inviting millions can really stress test anything but the beta testers' patience. The waiting queues were massive for almost every server last weekend (at least on Friday night). They may increase it slightly but my suggestion is not to even try to get into the first day of the upcoming "full" beta test. Wait for day 2 when things have calmed down. You'll save yourself a lot of grief.
Mekias - queues were almost completely gone by Saturday night, and eliminated on Sunday. All night Saturday, there was a message about once an hour about a new server coming online.

Xay nailed it. That's why the NDA is still in place.

Carra - not only is it a 20g download, it's another 8g or so of patches.
The NDA for SW:TOR is mostly to protect against story spoilers. This is far and away the most story centric MMO ever released. While I can't speak about my beta experience, I was a doubter before playing. Since playing, I've ordered the Digital Deluxe edition (still can't justify spending $150 for any computer game)
I think that a lot of people are overthinking the NDA when it comes to SWTOR and forget that there have been plenty of game releases where the NDA covered everything right up to launch, including "open" betas.

The point of the article holds true. The term beta does not mean what it once meant, even in parts of the industry that don't deal with millions of players. Publishers have done a lot of that by changing what they deem beta to mean and forcing developers to work around their ideas.
It's quite curious, you're right. The only people who won't know before release what SW:toR is actually like are those like me, who have not only have no intention of buying the game but who are so uninterested they haven't even bothered to register for the beta.

But then, by definition, those are the people who care the least, so I don't suppose it makes any commercial difference one way or the other.
Yes Beta releases have changed. But this is due to the modern realities of massive software development.

In times gone by you would code a product. Get it to a 90% alpha complete state and then let your testers bang on it to get it to 95% beta. THEN you would release the Beta version to your valued and friendly customers and see what happened in the "real world".

Now with the multitude of OS/platform/graphic/processor/multimedia PC combinations you can't have alpha and beta cycles like yesteryear.

In house testing has to be expanded and lengthened to accommodate these realities or suffer in quality. This is why we see the divergence between a RIFT launch vs other lesser MMOs. RIFT was probably tested to death before release. But many other MMOs went the route of launch 1st fix later.

I'm impressed by the quality of SWTOR but I don't know what to make of this movement to deeper time intensive testing...

It's funny I was talking to one of our engineers in my professional capacity today and he bemoaned TWO distinct build cycles. One for software compile and another for software configuration and security...

More complex software environments are here to stay my friends.
NDA dropped a few hours ago.

At the very least it was a courtesy to the 13 sites at the FS Summit. They got to get back and have a week to get their site updates and other stuff ready. E.g., the site purchased by CURSE already has skill trees, quests, locations and companions up.

By not allowing it be discussed earlier, they got more attention to the lower zone stuff and higher quality on the discussions that got an extra week. I am old and cynical enough to remember when companies had to hire PR flacks and advertising instead of using unpaid bloggers and fan sites.

But they did get rid of NDA before the million Beta testers got in.
Post a Comment

<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool