Thursday, September 06, 2007
What are beta tests for?
Bloggers read other people's blogs. And unsurprisingly, subjects that are discussed one one blog thus often jump to the next blog. Although this custom is described as "circle jerking", I think it isn't all bad. As long as every blogger also makes some posts with his own ideas, and the same subjects don't circle forever until they are stale, crosslinking to other blogs creates the fabled "blogosphere". This weeks subject in the MMO blogosphere is beta tests, as discussed at Kill Ten Rats, based on a rant at Wife Aggro.
Pvthudson from Wife Aggro thinks that people aren't playing betas right, and should stop complaining about what's wrong with the game on that game's beta forums. He says: "The beta is not for you to form a guild, get all your friends in, camp mobs, farm money and then complain that the game isn’t complete yet." Well, I'd say that if you would form a guild, get all your friends in, camp mobs and farm money in the real game, then this is exactly what you should do in the beta as well. It is good if some beta testers try out exotic stuff to find obscure bugs. But if the large majority of beta testers plays the beta exactly as if it was the real thing, that is very helpful for the developers too.
Finding bugs and exploits and reporting them can be very rewarding. In one of the betas I'm currently in I found an exploit, reported it, and saw it fixed in the patch notes a week later. I also had non-canned "thank you for finding this bug, good catch" e-mails from devs. Things like this really make you feel you are contributing to the game improving towards release.
But people just playing and then complaining that "the combat system sucks" on the beta forums are important too. Age of Conan was delayed by half a year based on exactly that sort of comment. Whether a feature is too complicated, working well, or just plain boring is not something you're likely to find in a bug report. Even a heated flame war on the forums with some people hating feature X and other defending it can give ideas to developers on how to improve it. Because if the flame war doesn't happen on the beta forums, it will sure break out later on the real forums, and then it's much harder to change things.
Open betas, or the kind of betas you can get into by just having a Fileplanet subscription, aren't even intended for everybody to write bug reports. If everybody did that, the devs would get thousands of bug reports on the same bug, with a huge workload to sift through all of them. Open betas are a joint project of the hardware guys and the marketing department, the guys developing game mechanics do barely figure at that point. "Open beta" just sounds more attractive than "free trial", because everybody is aware that it won't last, and you better get in before the release. So the hardware guys get their stress test, and the marketing guys get their free advertising. I think the way betas are going now is working as intended, there is no need to admonish players that they are doing something wrong in the betas.
The only advice I would give game companies is to keep the closed beta phase restricted to a small number of players, and to keep the open beta phase short. One of the most frequent problems of new MMORPGs is that due to having to release the game on a schedule and not having unlimited amounts of cash, games often are released with a lack of content. The last thing you want is a large number of beta players already playing through all of your content and being bored of your game before it is even released. You want to give the players of the open beta just enough of a hook to make them buy the game, not enough time to finish it for free.