Data mining in MMOs

You probably heard the story by now that SOE handed over 60 Terabyte of Everquest 2 data to the University of Minnesota for a scientific study. Apparently no chat logs, for privacy reasons, but detailed records of what everyone was doing all week long. 300,000 current subscribers with an average of 26 hours logged on per week, and the computer is tracking their every movement. Big Brother is watching you! :)

I do think those data contain useful information, but maybe less for scientists than for the game developers themselves. Simple example, since we've been talking class balance a lot all week: World of Warcraft has 10 classes, so if they were perfectly balanced and equally attractive, every class would represent 10% of the characters at every single level. So if you find that at level 19 there are more than 10% of rogues, and at level 80 there are far less than 10%, that tells you something about the rogue class, and possible imbalances. If, as the result of a patch, one class becomes significantly more popular, and another class significantly less popular, it tells the developers whether their patch went in the right or wrong direction. Another example is death knights: Are most of them leveled to 80, or are half of them just leveled to 60 and used as bank alts? Or adding up the data of everyone who visited Naxxramas in a given week, are there classes that are more represented in raids than in the general population? Even simple numbers like server load for WoW probably paint an interesting picture of spikes after expansions and content patches, and can tell the developers how long such a spike can last. How fast would they have to add new content to not lose X % of their players between patches / expansions?

Whether these data are useful for behavioral scientists is less sure. The environment of a virtual world is highly artificial, and the behavior of players is very much dictated by the incentive structure of the game. It would be easy to look at level 1 to 79 in World of Warcraft and conclude that people prefer soloing to grouping by a huge margin. But if you looked at similar data from the original Everquest, you would conclude that people prefer grouping over soloing. The reality is somewhere in the middle, but unless somebody makes a game in which soloing and grouping rewards are perfectly balanced, there is no way to find out the exact numbers. When given triple xp for grouping in the context of the WoW recruit-a-friend program, players change their behavior. Even something as simple as an achievement with a title can incite players to start grinding grey low-level quests A friend proudly showed me the tabard you get if you did nearly all the quests in the game with the same character: It has a yellow exclamation mark on it. :) Before that achievement was introduced, nobody would have dreamed of doing level 1 quests with a high-level character.

Thus I would say that data mining player behavior tells you more about how balanced game incentives are than it tells you about how what people prefer. You can't simply look at what people are doing and say A% of players prefer PvP, B% prefer raiding, and C% prefer doing solo quests. Because everyone who played World of Warcraft for years knows that the number of players doing PvP skyrocketed with patch 2.0, and the number of players raiding went up considerably with patch 3.0. The only insight about human psychology that gives you is that players tend to run after the rewards, and few of them have one preferred activity they'll do regardless of the incentive structure.
- posted by Tobold @ 8:36 AM Permanent Link