The many meanings of beta

Recently Richard Garriot revived one of the dead horses of MMORPG discussion by stating that it was a mistake to invite so many people to the Tabula Rasa beta. Cameron from Random Battle has an excellect article on beta testing up. To a certain extent I do agree with him: *If* MMORPGs would have the option for a free trial from day 1 of the release, beta tests could be a lot smaller and shorter.

There are several games I never bought because I disliked them in the beta, Vanguard, Tabula Rasa, and Hellgate London for example just for this year. And while I do like Pirates of the Burning Sea, I'm currently not participating in the open beta, because I already played too much of it in the closed beta, and rather wait for the release version instead of experiencing another character wipe. And I only played the PotBS beta since August, other players were in that beta for even longer, I heard the first beta test started in 2005. In my opinion beta tests should be a lot shorter, otherwise you risk that the beta testers are burned out before the game is even released. And if they are burned out when the NDA is dropped, that has a negative effect on the reviews as well.

On the other hand stress test weekends are a bit too short to give players a good impression of a game. Often the very fact that it is a stress test results in lots of server problems, which aren't necessarily there in the real game. And for more complicated game features a weekend isn't sufficient to experience them. For example on the PotBS stress test weekends, the economy features couldn't really be explored, because it takes days to build something up.

The PotBS devs are a pleasure to read on the official forums, because they are so disarmingly honest. About the open PotBS beta (STARTS TODAY!) they quite honestly said that it was more of a marketing device than for quality control and testing. Games having an open beta of two to four weeks just before release can serve both for marketing and for still finding some bugs. It is not because not every beta tester reports every bug that a beta test is useless. Some people take beta testing more serious than others. I must admit that I was very pleased when I reported a possible exploit in the PotBS beta which then ended up gettting fixed in the next patch, or when I got reply mails from the PotBS devs saying "good catch". But even somebody not reporting a single bug can serve as tester, because the simple fact that he is playing causes an effect on the game. Some exploits are caught not because the people who find them report them, but because players observe other players using them and complain. Or if a player manages to "break" the economy, that should become pretty obvious without anyone having to write a bug report. I would even go so far as to say that if all beta testers were just looking for bugs instead of trying to play the game, many faults would be missed. So I would have to say that it is more the length of a beta that I see as problematic than the fact that there is an open beta in which not everybody is an eager bug reporter.

But the bad press of Vanguard, Tabula Rasa, Hellgate London, and to some extent even Pirates of the Burning Sea, is also due to the game companies releasing games when they are just about playable. I read lots of reports that Vanguard is getting better, that Tabula Rasa improved a lot from beta to release, that Hellgate London is slowly adding things that would actually make it interesting to subscribe, or read about all the wonderful things the PotBS devs want to add after release. But sorry, you can't release an unfinished game and then ask of all the game reviewers to not write about it until a year later when the game is better. If you ask your paying customers to pay for a game, you must live with the reviews you get for the version you released. If you need better reviews, do it like Warhammer Online, cancel your beta and delay the game. You only get one chance to make a good first impression.
- posted by Tobold @ 11:44 AM Permanent Link