Tobold's Blog
Thursday, July 01, 2010
 
Numeracy

100% of my readers are literate, that is they can understand the meaning of written words and phrases, and can write words and phrases to make arguments themselves. The percentage of people who are numerate, that is they fully understand the meaning of numbers and basic mathematics, and can make a coherent argument with numbers, is much lower. And as it is well known that many people are bad even at basic math, numbers and graphs are often used to mislead people, because there are countless ways to manipulate numbers and "support" false arguments. This misleading can be deliberate, or it can be that the person making the argument doesn't understand the numbers himself.

In the context of MMORPGs, the most false arguments are based on player numbers. For example yesterday Nils made arguments about Wrath of the Lich King based on this graph, or more specifically the 2009 - 2010 player numbers for World of Warcraft. His argument, that a visible kink in the curve tells us something happens, was cogent. Unfortunately the numbers displayed on the graph are certainly completely wrong, and made up by somebody. We simply don't know how many players World of Warcraft has worldwide right now, it could be any number from 7 to 14 million. Somebody took the last available data and used the generally held belief that WoW subscriptions have remained constant to invent the last data points. As it is absolutely certain and confirmed that during that period WoW for several month lost all Chinese players, and we can assume that the Chinese servers have suffered from the shutdown, and the fact that they are still running Burning Crusade, the straight line on the display can't possibly be true. Blizzard simply stopped giving out numbers, and people are just guessing.

A case where numbers are probably deliberately used to mislead people is the common practice of Free2Play games to release the total number of accounts ever created, instead of the number of active players, or the number of players who paid at least $1 to them in the last month. Thus in the past weeks a lot of games like Wizard 101 or Free Realms proudly reported having 10 or more million players. In fact they have 10 or more million accounts, most of which haven't been used for months, and 95% of which aren't paying anything (according to industry estimates). If World of Warcraft would report player numbers like that, they would count all previous players and trial accounts, and get up to some fantastic number of 50 to 100 million players.

Especially journalists are often bad with numbers, and thus come to completely wrong conclusions. That way they often conclude that Farmville is more successful than World of Warcraft, because 80 million is more than 11 million, or that Free2Play games are as successful as WoW, because they have about the same "number of players". Thus PC Gamer seriously asked Blizzard whether they planned on going Free2Play. And then turned the obvious diplomatic answer of "not anytime soon, maybe in the far future at the end of WoW's life cycle" into a sensationalist headline.

My recommendation on getting meaningful numbers is getting financial reports and SEC filings, instead of press releases and game magazine articles. Thanks to Sarbanes–Oxley threatening CEOs with up to 20 years in prison for lying on that sort of financial documents, the numbers tend to be as accurate as possible. And by looking how much money players spent on a game, you can more accurately determine how much they liked that game than you can by the total number of accounts created.

And even that only gives you information about mass market success, and not whether a game is "good" or "bad". For example I would say that A Tale in the Desert is a very good MMORPG, but as it targets the tiny niche of players who want actualy meaningful social interaction and immersion instead of artificial monster fights and ganking, ATITD will never make millions. It is good to be able to read numbers, but they can't tell you everything.
Comments:
At least you respond to the graph :)

It comes from
http://mmodata.net/

This page uses an accuracy rating of A-D for the numbers. 'A' being the highest. WoW numbers are marked with A.


A – Confident : Most or all of the data points come from official or other reliable sources and are in line with other numbers and information.
[..]
World of Warcraft
Abbreviation = WoW
Accuracy Rating A


More explanation for the numbers can be found here.
 
You can also have a look here.

Blizzard always proudly presented new subscriber numbers.

Last time they did it was 21.11.2008.
 
I look on this with much interest, mostly because I'm studying statistics this semester and my mathematical BS meter has been tuned somewhat as a result.

:)
 
You can also find the WoW Subscriber Definition in the above links:

World of Warcraft's Subscriber Definition:
World of Warcraft subscribers include individuals who have paid a subscription fee or have an active prepaid card to play World of Warcraft, as well as those who have purchased the game and are within their free month of access. Internet Game Room players who have accessed the game over the last thirty days are also counted as subscribers. The above definition excludes all players under free promotional subscriptions, expired or cancelled subscriptions, and expired prepaid cards. Subscribers in licensees’ territories are defined along the same rules.

 
You know you just contradicted yourself in your two posts, do you, Nils?

And you are actually confirming what I said: We don't know the subscription numbers after November 2008, thus the MMODATA points are fake. Do you really believe the "accuracy rating" bullshit if no source is given?

If, however, you believe the MMODATA numbers to be correct, and we *know* that Blizzard lost millions of players in China due to the servers being down for months and the expansion never having been released, your only conclusion can be that WotLK led to several million new US & European players, and would thus be the most successful expansion ever.
 
For all (German) statistic freaks: this page is mandatory:

... should have put all this information in one comment. Looks like spam, again. Sorry.
 
@ Tobold:

The last official statement of Blizzard was in November 2008.

First: Do you think that WoW reached 13 mio subscriber and Blizzard didn't bother to tell us? Just using this argument, the graph is a very optimistic guess.

Second: The information of the other datapoints comes from other trustworthy links (Like statements by Blizzard):


here, here and here
 
and we *know* that Blizzard lost millions of players in China due to the servers being down for months and the expansion never having been released, your only conclusion can be that WotLK led to several million new US & European players, and would thus be the most successful expansion ever.

Tis is also something I added when if irst posted the graph here, yesterday.

It is absolutely impossible that Blizzard didn't lose subscribers in China or did lose those subscribers and made up for it in the rest of the world.

Still, Blizzard still talks about some 11-12mio subscribers ...

Either way. Non of the this makes WoW appear to have hit 13mio in the meantime. And this would have been necessary to fallow the growth path of the past.

The graph is an overly optimistic guess at worst!
 
Nils, is that a deliberate smokescreen tactic, where you spam us with completely irrelevant links and quotes that have nothing whatsoever to do with the argument? Or are you very confused?

I'm saying that the MMODATA points saying that WoW had exactly 10.6 million subscribers in 2009 and 10.5 million subscribers in 2008 are completely spurious inventions. Are you saying these numbers are true?
 
Tobold, it isn't. Have you actually read though all of the links within the last few minutes?

I am not saying these datapoints are 100% true.

What I do say is that if anything the number of subscribers stayed roughly constant over WotLK.
 
What I do say is that if anything the number of subscribers stayed roughly constant over WotLK.

Aeh, yes, and?

I never said otherwise.

I stated that the number of subscribers is between 7 and 14 million, and we don't know where exactly. I'd agree that *under* 11 million is more likely than *over* 11 million.

But as we *don't* know the number, we can't really just take a fake data point and make an argument with it. Your argument was that the kink in the curve shows that players deserted WoW because they didn't like WotLK. I'm saying that WoW certainly lost several million players in China during the time period covered by those two fake data points, and that if WoW lost players in US/EU, we should be down to 7 million or so now, while if we really are at 10.5 million now, WotLK actually attracted a *lot* of new US/EU players.
 
Great then, we almost agree ;)

14 mio and 13 mio is impossible in my opinion, because Blizzard would have made a press release as they always did.

12 mio and less means that the growth abruptly stopped.

WoW might have 7mio subscribers by now. But the links I posted make this look overly pessimistic.

Your argument was that the kink in the curve shows that players deserted WoW because they didn't like WotLK

Hell, no!
The kink in the curve is just a kink in the curve! If it is true, the subscriber numbers stopped growing some months after WotLK released.


The China-problem remains:
It seems, Blizzard isn't really honest about these Chinese players. Otherwise, why should Reuters write this?
 
Nils,

I'm confused whether you're:

1) Stating hypotheses and using a bogus graph to support them

or

2) Showing a bogus graph and trying the explain the fake-trends with your hypotheses


In your comments^^, it seems to be a bit of both!
Either way, the OP is right: the internet is rife with guess-work. Let's not pretend there's some science behind it.
 
Have a look at mmodata.net first, please. Looks like a good page to me that gives even more links than I just gave many, many MMOs.

If you don't study the links before you comment, I cannot help you.

I sent a mail to the operator of the page and asked for his sources for the last few data points. let's see if he respondes.

Even without the graph, you can say that WoW has less than 13 mio subscriber now. That means that a steep growth path has been left. It happend in the first half of 2009. I say that because Blizzard stopped stating Sub numbers in Nov 2008.

What can be said is that WoW sub number stagnates or decreased since early 2009. Maybe the Chinese play a role here. Would be interesting to hear more about that.
 
I know it's not on the topic and I'm awfully sorry to interrupt but is it only me that get's a weird feeling reading this comment section recently? It feels as if I'm sort of prying as I'm watching a more or less private eye-to-eye dinner conversation between Nils and Tobold. I know your comments are intended to be public but yet... it looks a little bit strange.
Sorry for interrupting. I think it was Tobold's turn to say something now.
 
Ok, blogger ate my first comment which was long winded and wordy, so I'll keep this one short.

Those links you posted all say one thing: Blizzard hasn't posted any official numbers since November 2008. Therefore all these extra data points after that are, as Tobold said, made up.

Doesn't matter if the site is "trusted" (whatever that means?). The numbers are not official numbers. It doesn't matter if the guess is a very good guess based on the evidence at hand. It's still a guess.
 
I know it's not on the topic and I'm awfully sorry to interrupt but is it only me that get's a weird feeling reading this comment section recently? It feels as if I'm sort of prying as I'm watching a more or less private eye-to-eye dinner conversation between Nils and Tobold. I know your comments are intended to be public but yet... it looks a little bit strange.
Sorry for interrupting. I think it was Tobold's turn to say something now.


Well, what can I do? I already asked Nils repeatedly in comments and by e-mail to post less here, and maybe limit himself to one comment per post, but he won't listen. Should I ban him for spamming?
 
At Mon Mar 2, 2009 10:13am EST Chief Executive Jean-Bernard Levy was reported by Reuters to have said this during a press conference:
"We expect video games to continue to show a nice growth. We started the year with 12 million subscribers for World of Warcraft, which is a good base,"


Yes - it is not an official press release by Blizzard. But it is a trusted source, isn't it?

Would you (all) please stop commenting my links before you read them?

Now - somebody could have set up that Reuters news page to mislead me. It is not a official number by Blizzard, I agree.
 
I shouldn't have used the comments section to discuss your opinion about the graph I posted.

Sorry.
 
"is it only me that get's a weird feeling reading this comment section recently?"

No, Laríssa. We all feel the same. This sexual tension between Nils and Tobold will result in something very destructive to everyone. I've seen it before.
 
I don't think WoW lost 7 million subscribers just because WotLK is not released there. They play BC with just as much addiction as everyone else plays WotLK.
 
I think the "Free 2 Play Shot Heard Round The World" was not so much Farmville as Turbine going F2P on LoTRO -- and so quickly after going F2P on DDO.

LoTRO seemed like a fairly popular MMO, and it's reasonable to believe it was a success as a subscription game. For Turbine to go Free 2 Play on LoTRO means the DDO F2P experiment went very, very well.
 
This comment has been removed by a blog administrator.
 
Noo, don't delete the last guy, Tobold! It was a perfect set up for "Then go read his blog instead. Oh wait, oops!"
 
I did some research to fiond out more about the last 5 datapoints on the graph.

The 11mio one and the following 11.5 mio one come directly from Blizzard press releases.

The 12 mio one in the beginning of 2009 comes from the Reuters article.

The 11.6 mio one comes from the Guinness World Record Awards in June 2009.

The 11.5 mio one from 2010 comes from this
Activision Blizzard Q1 2010 Earnings Call Transcript"

A shorter version can be found here .

There have been several articles about the last one:
1
2
3
4
5
6
 

Links to this post:

Create a Link



<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool