Tobold's Blog
Friday, June 24, 2011
 
Numeracy and Democracy

Somehow it seems I'm writing this post at least once a year. But numeracy is actually something I am passionate about, and thus it hurts to see the huge amount of writing in which numbers are used in completely wrong and misleading ways. And disregard for the majority is one of eternal pet peeves of mine.

As this is a MMORPG blog, the relevant numbers most often represented in a completely wrong way are player numbers. There is actually not a single MMORPG in which player numbers are represented in a meaningful way. World of Warcraft famously lists total number of paying customers, thus bunching together the US/EU customer paying around $15 per month with the Chinese customers who actually pay by the hour, and normally much less. Rift reports the 1st million players, and you need to look in the small print to see that this is 1 million boxes sold, not 1 million players currently subscribed. EVE reports number of accounts, which in a game where most serious players have multiple accounts gives a much larger number than counting players. Free2Play games count accounts too, so if you open an account, play for 5 minutes, hate the game, and uninstall it, you will be forever counted as a player, as most of the time there isn't even a delete account functionality.

Now some people respond to this misleading bunch of numbers by claiming that numbers have no meaning at all. Nils loves to quote "Eat shit! Billions of flies can't be wrong." But if that would be a meaningful comment, we would also have to abandon democracy, which is built on the idea that the majority is right. Once you look closer at it, you quickly realize that this saying only means that what is good for one population (flies) isn't necessarily good for another population (humans). Now of course some people want to express that the players of WoW are shit-eating flies, while they are superior being with more refined tastes, but that is just the usual elitism. There is not one autocratic regime in the world which wouldn't argue that they know better what is for the best of the people than the people themselves would know in a democratic system.

MMORPGs are extremely democratic. There is very little other than personal tastes which would keep the player of one game from playing another game. Thus it matters very much which games people actually end up choosing. If out of their free will people vote with their feet and their wallets for one game, and not another, that really tells you something about how "good" that game is. With "good" of course being synonym with "fun", "entertaining", or "popular", not necessarily "acclaimed by some highbrow critics". Just like certain books or movies can catered towards very narrow tastes and end up getting great reviews and lousy sales, one can always find somebody giving a great review to a game only 10,000 people play. That game is "good" for its 10,000 players, but "bad" for everybody else. To be really worthy to be called a good game, a significant number of people need to spend a lot of time and money playing it, for a significant time, and out of their own free will.

What one has too look at to make these numbers meaningful is what are the scarce resources. By looking at how players spend their scarce resources, one can accurately say something about how good a game is. In the case of MMORPGs the scarce resources are money and time. A billion flies can be wrong as culinary advisors for humans, because flies have different food selection criteria than humans. A billion dollars can't be wrong, because money has value for all of us (albeit to different degrees, depending on your disposable income). You simply can't make a billion dollars a year with a bad game, without twisting the definition of what a "bad game" is to something irrelevant and not recognizable.

For time, the number of players or accounts is a bad measure, because you don't know how much time each of these players spends. Ultra-casual free games and "virtual spaces" quickly get millions of players, but most of them neither spend money, nor more than a few minutes per day on the game. Services like XFire or Raptr can give some idea of activity in a game, but usually there is only a small percentage of players who have thus tools installed, and they aren't necessarily representative of the whole player base. One better number one can see sometimes is maximum concurrent users, which at least after correcting for multi-boxing gives you some idea of how many people are actually playing. Server-based games usually have technical limitations of how many concurrent users they can support, and thus how many servers a game has, and whether they are opening up new servers or merging and closing down old ones is a valuable information here.

While there isn't much price differentiation between subscription games, these appear to be a dying breed, with every week bringing new announcements of some game going Free2Play (this week City of Heroes and Lego Universe). A Free2Play game will always have more players than a game of equal quality that charges a subscription. Hey, I'd actually might try CoH again now that it went free, although that is more about a lack of committment than a lack of money. Thus I'd argue that a game's revenue is a better indicator of how good it is than the number of players, unless you compare two games with the same business model and similar pricing.

If a game is very popular, one can use that fact for predictions and recommendations. If somebody of whose gaming habits I know absolutely nothing asks be recommend him a MMORPG, chances are that he will like World of Warcraft. That is not the same as saying that "everybody likes World of Warcraft", which obviously isn't true. If you don't know what dessert people like, ice-cream is usually a safe bet, while chocolate seaweed isn't. But once in a while you get somebody who is lactose-intolerant, and ice-cream would not be the best dessert choice. Some people are WoW-intolerant for various reasons, although the largest number of people actually disliking WoW are those who played it for hundreds or thousands of hours and burned out. Then you get into that strange area of discussion where people feel the urge to defend their game choices as if they were lifestyle decisions, so if somebody played a game for thousands of hours the game was obviously good at that point, and the miracuously turned bad just at the time that person was quitting.

So, player numbers are not totally meaningless, especially not if it is a subscription-based game, and there is a good correlation between player numbers and money spent. When Star Wars: The Old Republic comes out, the number of players it will attract (if actual subscription numbers are released) is a good indicator of whether that game sucks or is any good. That doesn't mean certain people can't have a very different definition of "good", and go totally against the majority opinion. Klepsacovic tried to persuade me to play Starcraft 2. For all I hear that is a good game, but I simply don't like real-time strategy games as much as I like turn-based strategy games. There is nothing wrong with not agreeing with the majority, and having different tastes. It is wrong to claim that the minority is right and the majority is wrong. As long as we can choose freely which games to play, that choice is like a democratic vote. Question the vote of the majority (if counted right), and you question democracy itself. Or as Churchill said: "Democracy is the worst form of government except all the others that have been tried". If you argue that your exotic choice is better than the majority vote, then why yours, and not that of somebody else having a very different minority taste? A single person can only answer the question which game is best for *him*, the majority says which game is best for most.
Comments:
There is very little other than personal tastes which would keep the player of one game from playing another game.

Good post, but you need to be careful about ignoring network effects. For example, D&D is the most popular tabletop RPG, but it's not necessarily "better" than its competitors. It's just the one that everyone has heard of. I suspect the same thing applies to games like Everquest and WoW.
 
One word: Farmville.

I agree with your overall point, but it is a pretty dark road to travel down when you agree to base the goodness of a thing on the number of people doing/consuming it.
 
Farmville is exactly the example how people misuse numbers to make a point. The number of "players" of Farmville isn't relevant at all. What is relevant is how much time they spend, and how much money they spend. And if you see that 80 million Farmville players make Zynga 200 million dollars per year, that ends up being just a few cents per user per month. And these users spend only a few minutes per day with the game. Furthermore even the number of players went downhill after half a year.

Thus if you look at the RIGHT numbers, you will see that better games than Farmville attract players spending more time and money for a longer time. Which is how it should be.

but you need to be careful about ignoring network effects

I'd include network effects in the overall quality of a game. There is an added value to a game that all of your friends play and where you can play with them.
 
There is one and only one meaningful number to measure the success of the game: the profit of the publisher company.
 
Gevlon just said it. The most relevant, widely agreed upon yardstick to measure succes in any activity today is financial. It makes the world go round. Accepting that, it is indeed impossible to make a bad game and make a billion a year. However this doesnt necessarily say anything about the intrinsic values of that game (or movie etc). It says something about the quality of mainstream targeted (viral) marketing and hype-management. Btw, the internet is of course a godsend tool for exactly this.

I would argue that REAL intrinsic quality is usually found beyond mainstream, where no concessions need to be made.

Sure, when giving advice to a fellow gamer or more generally, consumer, a mainstream choice is the safe bet, but when you know him or her well, you can really wow them with niche.
 
"MMORPGs are extremely democratic. There is very little other than personal tastes which would keep the player of one game from playing another game."

That's false.

There are a great many other variables that determine whether or not someone plays an MMO: cost, friends/relatives playing the game, knowledge of other games actually being out there and simply emotional attachment to the game.
 
I would argue that REAL intrinsic quality is usually found beyond mainstream, where no concessions need to be made.

But who determines what REAL intrinsic quality is? Does it even exist? If you get a single person or small group to select what they think is intrinsic quality, you only end up with a niche opinion. What would make one person a better judge of intrinsic quality than another?

Thus I would argue that even if you are beyond mainstream, lets say with indie games, a very good indie game like Minecraft also attracts a lot of players and money.
 
3 commenters in (not counting Tobold) and already we're running into a problem: successful vs. good.

A better game could have lower profits, since development costs were higher, but did not deliver sufficient popularity to offset the higher cost.

Success (as measured by profit, which is an excessively narrow measure itself), popularity. and goodness will be related in some ways, but there's no way to simply link each one to the next with no major complicating factors.
 
@Tobold
Agreed, for some products it is probably easier to precisely describe the qualities it should possess, in order to determine how well it fulfills its reason for being. In my perception, when designing mainstream products those responsible not only consider these practical issues but also take into account that the final product *must* appeal to the masses or else it will be considered a failure. Mass appeal may even in importance supersede the aspects the product was designed for in the first place. Compromise will probably be the result. Niche products can be made with more pure focus on function. But i agree, they can also unintentionally (?) appeal to masses.
 
I guess it all boils down to how we measure success. Products with niche markets are hard to compare with mass-appeal ones. Kind of like coke and fine wine.

The only measure for companies is revenues and profits. For players, it is in subjective terms like "fun".

Critics and fanboys alike with rally to whatever numbers seemingly support their argument. More subs, but less fun vs more fun (their type of it at least) but fewer subs. In every case, when someone says the glass is half-full, another will say it is half-empty.

Like Sir Charles Wentworth Dilke said, "There are three kinds of lies: lies, damned lies, and statistics."
 
Well written post but wrong in some levels...

"But if that would be a meaningful comment, we would also have to abandon democracy, which is built on the idea that the majority is right."

You can't compare political systems with media entertainment. If most people chose their rulers the same way they choose the game they will play (and some do) we would be in deeper crap than we are now.

"although the largest number of people actually disliking WoW are those who played it for hundreds or thousands of hours and burned out"

Where exactly did you pulled this from?

Is it inadmissible for you that many people liked Vanilla and TBC and disliked Wrath and Cata? Even I only got back to Cata to fly in Azeroth but I actually enjoyed the dungeons until the first nerf.

"is a good indicator of whether that game sucks or is any good"

That actually depends on what you mean by "good". Some people want hard tasks that require lots of hand/eye coordination and reflexes. Other people require brainless tasks with lots of shining. So that point is moot. Cinema is a good example as most blockbusters are nevertheless poor films as often the story is derivative, cliche ridden and there is an effort to make the plot easy to follow. But to some people (and me sometimes) that is exactly what they need/want/think is the very definition of "good".

How would you think a person who liked Bergman, Wenders or Kurosawa would rate the latest Michael Bay movie? Yet Transformers has probably brought much more profit and is wildly more popular than "The Seventh Seal".

So numbers will tell you about the popularity of a game, profits will tell you about the success of a game but none of them will tell you about how "good" or "bad" a game is.

Speaking for myself, I think that WoW became a bland RPG where no decision you make has any impact whatsoever since you can change everything. it doesn't mean I didn't had fun on Cata, but it means that if not for the whole "Fly in Azeroth" thing I would never resubscribe to the game. And what does this mean? It means that nostalgia for the great fun I had in Vanilla was the only thing that made me fork over abou 80€ to Blizzard this time around.

In short, if I want to have Macdonalds I'm not going around pretending I'm having a good meal because I'm not. I'm having a mac and sometimes is just what I need.
 
This comment has been removed by the author.
 
Sorry about the somewhat confusing post. I wrote it throughout the day...
 
But who says that "The Seventh Seal" is actually a good movie? I'd say it isn't. It is a niche product full of obscure references which only a very small number of people are able to understand. That these very small number of people then say that this is a great movie, and use the fact that they get it and others don't to brag about their intellectual superiority is just elitist crap. And with games it's just the same.

What exactly makes you a better judge of quality than the majority vote?
 
Nils loves to quote "Eat shit! Billions of flies can't be wrong."

I prefer Stalin's "Quantity has a quality of its own."

Or Depeche Mode's "Everything counts in large amounts."
 
There are countless example of 'bad' product gaining mass popularity and driving profits; quality is not always (usually?) the driving force (unless you'd care to argue that reality TV, pop music, and summer blockbusters are, in fact, high quality?) There is a reason marketing people get paid what they get paid, and marketing often has nothing to do with the product itself.

You combine the hype machine factor with the social hooks of an MMO, and hey, Zynga is crazy profitable (for now).

Retention to me is the must useful number to judge an MMO by if you believe in the original design of an endless virtual world. A high churn rate that gets filled, to me, shows more about the marketing for the products than it's actual quality. Take this approach, and the 'value' of F2P is pretty clear.

Sadly retention data is very hard to come by, at least on a large scale.
 
I do not see how numbers are an accurate way to judge art and taste at this time. Maybe someday math will figure it out like they have with baseball, but trying to judge art and taste by number of viewers is like judging a baseball player on RBI’s, it is a statistic that tells very little about the overall product. Nearly every Network Primetime Show has a higher Nielsen rating than Mad Men or Breaking Bad, but Mad Men and Breaking Bad win more awards and are more loved by critics. The fact that Two and a Half Men had higher ratings than Breaking Bad had more to do with Breaking Bad being on AMC, a cable channel, and Two and a Half Men being on CBS. Numbers might not lie, but that does not mean we always understand what the numbers are telling us.

Oh and, “the best argument against democracy is a five minute conversation with the average voter.”-Churchill
 
You can use numeracy to objectively demonstrate "successful" or "popular", but those numbers, as you noted, can easily be manipulated.

But can you use numeracy to demonstrate "good"?

"Good" not only implies subjectivity, but also approaching some as yet undefined ideal state, or even "truth".

An old saying I recall from college; (paraphrasing) the number of people believing something to be true has no bearing on whether or not it really is true.

There may be correlation, but not causality.
 
"actually a good movie? I'd say it isn't. It is a niche product full of obscure references which only a very small number of people are able to understand. That these very small number of people then say that this is a great movie, and use the fact that they get it and others don't to brag about their intellectual superiority is just elitist crap."

Is this you avoiding debate and saying something controversial with the objective of turning the conversation sour so you can delete some posts and complain about the elitist jerks?

The Seventh Seal is a great work full of beautiful imagery, great acting and basically is, for me, a meditation on finitude and mortality. I don't think is full of obscure references at all, just thought provoking dialogues, images and interactions between characters.

But yeah, I can totally see how feel that Transformers is a much better film.

"What exactly makes you a better judge of quality than the majority vote?"

Again, what do you mean by quality? If you mean quality as in customer satisfaction, absolutely nothing.

But I will always believe that complex, thought provoking and challenging works, be it music, films or games, will always be better than products built with the sole intention of profit (nothing wrong with that mind you) that dilute the content in order to appeal to the greatest number of individuals possible.

Beauty is in the eyes of the beholder and every work has it's fans but claiming that numbers are by themselves a sign of a good or bad work is not the most smart thing to state.
 
But I will always believe that complex, thought provoking and challenging works, be it music, films or games, will always be better than products built with the sole intention of profit (nothing wrong with that mind you) that dilute the content in order to appeal to the greatest number of individuals possible.


This. Mainstream's prime objective is not to make to best, purest product possible, but to appeal to the masses and sell lots of units. Here accountants are in charge of the creative process. Niche players focus solely on the intrinsic qualities of the product.
 
We know that it is possible for the majority of people to be wrong about something, and even for the majority to be wrong about what the majority thinks.

For nearly anyone who is starting a new online game, a major factor in their choice will be whether they know anyone playing that game. Popularity can be self-reinforcing. But it is actually possible that the majority of people who play a game are not that impressed with it.

I recall once someone making an analogy to a diner that you always go to with your friends. You never really liked the food that much, but its where your friends go. When you friends move away you stop going there, but whether you go there or not is no comment on whether you like the food. It could easily be, in fact, that none of your friends particularly liked it either, and that everyone was there simply because everyone else was there. Presumably at some point it had something going for it (one of you liked it, it was in a convenient location) but that could have long since faded.

Even if you think that the only way to judge quality is by majority opinion, you can't determine majority opinion from majority action. Popularity creates popularity and age diminishes popularity. Both of those factors are going to be as significant as whether the people who play the game actually think it is a good game.

You are also ignoring "good enough." It's hard to argue that 10 million people playing a game doesn't mean the game is good enough for a lot of people. But good enough and good are two different things and we all know it.

A study showed that people who did not drink a lot of wine actually enjoyed cheap wine more than expensive wine. Most people drink cheap wine more than expensive wine, and most people enjoy cheap wine more than expensive wine. But saying that the cheap wine is better would be odd. It actually makes sense to let the connoisseurs judge the "objective" quality. (http://www.xkcd.com/915/)

They reason why arguments about which games are good degenerate is because we are really arguing about is who are the video game connoisseurs.
 
The Seventh Seal is a great work full of beautiful imagery, great acting and basically is, for me, a meditation on finitude and mortality.

World of Warcraft is a great work full of beautiful imagery, great stories and basically is, for me, a meditation on finitude and mortality. I sure died often enough for that.

The only thing you CAN argue is that quality is not correlated with quantity. It does NOT follow that something popular automatically needs to be bad. There are a million criteria where you can without sarcasm say that lets say World of Warcraft is a much better game than Darkfall.
 
"The only thing you CAN argue is that quality is not correlated with quantity. It does NOT follow that something popular automatically needs to be bad. There are a million criteria where you can without sarcasm say that lets say World of Warcraft is a much better game than Darkfall."

But I'm not saying that popularity equals bad, it was you who was stating the opposite.

Quoting myself from the last comment:

"Beauty is in the eyes of the beholder and every work has it's fans but claiming that numbers are by themselves a sign of a good or bad work is not the most smart thing to state."

So again you are answering someone but not me.

And Darkfall is the better game if you want World PvP. WoW is the better game if you want dungeon crawls and EVE is the better economy game.

There even are a million criteria where you can without sarcasm say that lets say Farmville is a much better game than World of Warcraft. :)
 
Exactly. And at that point we are back to where "good" and "bad" become meaningless, because we defined them on an individual level which has absolutely no relevance for somebody else. Thus "popular" tells you a lot more about a game, and "popular" becomes the new "good".
 
Or the new "average."
 
Post a Comment

<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool