Numeracy

100% of my readers are literate, that is they can understand the meaning of written words and phrases, and can write words and phrases to make arguments themselves. The percentage of people who are numerate, that is they fully understand the meaning of numbers and basic mathematics, and can make a coherent argument with numbers, is much lower. And as it is well known that many people are bad even at basic math, numbers and graphs are often used to mislead people, because there are countless ways to manipulate numbers and "support" false arguments. This misleading can be deliberate, or it can be that the person making the argument doesn't understand the numbers himself.

In the context of MMORPGs, the most false arguments are based on player numbers. For example yesterday Nils made arguments about Wrath of the Lich King based on this graph, or more specifically the 2009 - 2010 player numbers for World of Warcraft. His argument, that a visible kink in the curve tells us something happens, was cogent. Unfortunately the numbers displayed on the graph are certainly completely wrong, and made up by somebody. We simply don't know how many players World of Warcraft has worldwide right now, it could be any number from 7 to 14 million. Somebody took the last available data and used the generally held belief that WoW subscriptions have remained constant to invent the last data points. As it is absolutely certain and confirmed that during that period WoW for several month lost all Chinese players, and we can assume that the Chinese servers have suffered from the shutdown, and the fact that they are still running Burning Crusade, the straight line on the display can't possibly be true. Blizzard simply stopped giving out numbers, and people are just guessing.

A case where numbers are probably deliberately used to mislead people is the common practice of Free2Play games to release the total number of accounts ever created, instead of the number of active players, or the number of players who paid at least $1 to them in the last month. Thus in the past weeks a lot of games like Wizard 101 or Free Realms proudly reported having 10 or more million players. In fact they have 10 or more million accounts, most of which haven't been used for months, and 95% of which aren't paying anything (according to industry estimates). If World of Warcraft would report player numbers like that, they would count all previous players and trial accounts, and get up to some fantastic number of 50 to 100 million players.

Especially journalists are often bad with numbers, and thus come to completely wrong conclusions. That way they often conclude that Farmville is more successful than World of Warcraft, because 80 million is more than 11 million, or that Free2Play games are as successful as WoW, because they have about the same "number of players". Thus PC Gamer seriously asked Blizzard whether they planned on going Free2Play. And then turned the obvious diplomatic answer of "not anytime soon, maybe in the far future at the end of WoW's life cycle" into a sensationalist headline.

My recommendation on getting meaningful numbers is getting financial reports and SEC filings, instead of press releases and game magazine articles. Thanks to Sarbanes–Oxley threatening CEOs with up to 20 years in prison for lying on that sort of financial documents, the numbers tend to be as accurate as possible. And by looking how much money players spent on a game, you can more accurately determine how much they liked that game than you can by the total number of accounts created.

And even that only gives you information about mass market success, and not whether a game is "good" or "bad". For example I would say that A Tale in the Desert is a very good MMORPG, but as it targets the tiny niche of players who want actualy meaningful social interaction and immersion instead of artificial monster fights and ganking, ATITD will never make millions. It is good to be able to read numbers, but they can't tell you everything.
- posted by Tobold @ 2:00 PM Permanent Link