Tobold's Blog
Thursday, March 05, 2009
Data mining in MMOs

You probably heard the story by now that SOE handed over 60 Terabyte of Everquest 2 data to the University of Minnesota for a scientific study. Apparently no chat logs, for privacy reasons, but detailed records of what everyone was doing all week long. 300,000 current subscribers with an average of 26 hours logged on per week, and the computer is tracking their every movement. Big Brother is watching you! :)

I do think those data contain useful information, but maybe less for scientists than for the game developers themselves. Simple example, since we've been talking class balance a lot all week: World of Warcraft has 10 classes, so if they were perfectly balanced and equally attractive, every class would represent 10% of the characters at every single level. So if you find that at level 19 there are more than 10% of rogues, and at level 80 there are far less than 10%, that tells you something about the rogue class, and possible imbalances. If, as the result of a patch, one class becomes significantly more popular, and another class significantly less popular, it tells the developers whether their patch went in the right or wrong direction. Another example is death knights: Are most of them leveled to 80, or are half of them just leveled to 60 and used as bank alts? Or adding up the data of everyone who visited Naxxramas in a given week, are there classes that are more represented in raids than in the general population? Even simple numbers like server load for WoW probably paint an interesting picture of spikes after expansions and content patches, and can tell the developers how long such a spike can last. How fast would they have to add new content to not lose X % of their players between patches / expansions?

Whether these data are useful for behavioral scientists is less sure. The environment of a virtual world is highly artificial, and the behavior of players is very much dictated by the incentive structure of the game. It would be easy to look at level 1 to 79 in World of Warcraft and conclude that people prefer soloing to grouping by a huge margin. But if you looked at similar data from the original Everquest, you would conclude that people prefer grouping over soloing. The reality is somewhere in the middle, but unless somebody makes a game in which soloing and grouping rewards are perfectly balanced, there is no way to find out the exact numbers. When given triple xp for grouping in the context of the WoW recruit-a-friend program, players change their behavior. Even something as simple as an achievement with a title can incite players to start grinding grey low-level quests A friend proudly showed me the tabard you get if you did nearly all the quests in the game with the same character: It has a yellow exclamation mark on it. :) Before that achievement was introduced, nobody would have dreamed of doing level 1 quests with a high-level character.

Thus I would say that data mining player behavior tells you more about how balanced game incentives are than it tells you about how what people prefer. You can't simply look at what people are doing and say A% of players prefer PvP, B% prefer raiding, and C% prefer doing solo quests. Because everyone who played World of Warcraft for years knows that the number of players doing PvP skyrocketed with patch 2.0, and the number of players raiding went up considerably with patch 3.0. The only insight about human psychology that gives you is that players tend to run after the rewards, and few of them have one preferred activity they'll do regardless of the incentive structure.
whats with the old post?
You've posted this stuff before.
I did? Link please. A reader wrote me a mail with the link in this post, and I thought I hadn't written about it yet. Maybe Alzheimer? :)
I think you're right, but there's probably some interesting data in there for economists wanting to study how people respond to different incentives.
could just be deja vu. but im sure i remember reading about the 10% over 10 classes would mean they are all equal (as if that could ever happen). I think the statistics would be interesting.
Your last paragraph made me think that all WoW players are doing is chasing the carrot. I suppose with achievements blizzard are trying to make people chase their own objectives which is one of the types from the Bartle Test (the killer, explorer, gamer, achiever, and the other one). Its a problem for other games - can they please all those categories?
Statistical analysis that follows often shows `subliminal' trends that may indicate important results. Correlations, factor analysis and other tools can `cluster' information depending on relevance and thus give further indications. For example, as you noted, on would find strong correlation between solo tasks and levels <79, where as raiding activity will be strongly correlated to >80. Groups that exhibit strong in involvement with PvP quite possibly exhibit less involvement with raids but possibly relatively high with 5-mans. Or indeed VoA where PvP rewards can be obtained.

@DracoChapel: Satisfying the Bartle test can be done based on assumptions (of what elements of the game depict in relation to Bartle's categories) and probabilities. No absolute results can be obtained but high relevance with certain activities could be indicative. For example a factor analysis could result in 5 factors that possibly could be `assigned' to Bartle's categories. The degree any game satisfies these categories ofc varies and possibly the numbers can depict that. Maybe Blizzard has managed to classify players in those categories - the extent of that might appear in the analysis.

What I think is essential is the knowledge of the virtual world in order to be able to `read' the results. Some results may seem different once you are aware of such mechanics (such as changes among patches, ease of certain activities, post-expansion weeks etc). For example the 10% comment needs to take into consideration the appeal a certain class might have due to factors other than viablility/effectiveness etc, such as lore importance, looks, race-relation or in the past faction availability. For example NE were ever so popular in the past to some extent because they looked good.(for some :p)

As Einstein implied... its all relative.
Perhaps an even more interesting study would be to compare the data of WoW to say EVE, or some other more open-ended MMO. We all suspect that in WoW, more than any other MMO, players play for the reward and not the experience. Would the data support that? If you gave away the most powerful epic for swimming around Azeroth 50 times, how many players would we see spending hours doing nothing but swimming rather than raiding or grinding BGs?
One also has to be careful with sampling here. For example, game with a reputation for easy soloing will attract and retain customers who enjoy doing that, thus skewing the figures still further in that direction. It doesn't necessarily tell you anything about what MMO players as a whole like.

"Players who like soloing tend to pick a solo-friendly game and ... err ... solo in it" is hardly a revelation.
"We all suspect that in WoW, more than any other MMO, players play for the reward and not the experience."
I'd argue it was the other way around. The reason WOW is so much more successful than other MMOs is because the ratio of fun to labour is so much better. An EQ1 player had to do far more pointless boring tasks to get their rewards than in WOW.
One of the things I, as an economics student, would be most interested in is the economic data. Unfortunately, I haven't played (and probably will never play) EQ or EQ2 and don't know how the system is organized or what structures and institutions exist. One thing that a similar data dump from Blizzard would be fantastic for is semi-real microeconomics simulations. Given that there are multiple servers, and multiple sets of servers (US/EU/Oceania) embedded in different cultural spaces, all working with the same primary set of economic constants, it would be interesting to attempt to trace back differences in economic outcomes as a function of initial variables. The holy grail would be if Blizzard would allow a researcher to "tweak" certain constants in the game (vendor prices, AH commission rates, etc...) on different servers. Screw random attribute agent-based computation, with WoW players you'd get real human decision making in all it's irrational glory driving the microeconomic decisions.
Post a Comment

Links to this post:

Create a Link

<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool