Tobold's Blog
Saturday, January 26, 2008
 
How to interpret WoWJutsu numbers

WoWJutsu is a third party website that uses World of Warcraft Armory data to rank guilds. On it's front page it proudly proclaims 2.4 million total players ranked, of which 98% are said to have visited Karazhan. This has been widely interpreted as 2.4 million players having seen Karazhan, which is about half of the total population of 4.5 million US and Euro players. Other people said that this number seems to high, and various arguments went back and forward on who to interpret the numbers. The Wowjutsu FAQ isn't all that precise on who exactly is counted. I had started using guarded phrases like "according to the most optimistic interpretation of Wowjutsu data", because I wasn't sure which interpretation was correct, and constantly got attacked for my numbers being wrong. Enough of that! Lets find out how Wowjutsu REALLY counts raiders.

The trick to find out the truth is to drill down to a level of which you are absolutely certain. On the Wowjutsu site you can click on "Realms", choose US or Euro, choose your realm, and finally choose your own guild to see detailed stats about your guild. And then you can go to WoW and compare the Wowjutsu numbers with the WoW numbers. To be very exact in my case I even submitted a "manual update request" to Wowjutsu, so I have scanned Wowjutsu data of today from my guild.

The result: In WoW my guild has 394 members (characters, not players), due to most of us suffering from altitis. Of these 394 characters, 164 are level 70. On the Wowjutsu site my guild is considered to have 164 raiders!

Wowjutsu considers every level 70 character in a ranked guild as being a raider!

A ranked guild is a guild in which at least one person has Karazhan loot. But that doesn't make every level 70 character in that guild a raider! In the case of my guild I went through the rooster on Wowjutsu, which has detailed information on who has loot from what raid dungeon. I counted 73 players with at least one piece of raid loot. Due to my guild having Karazhan "on farm" and being generous with raid epics towards newbies, I do not think we have many raiders with no epic loot. Thus in the specific case of my guild, Wowjutsu overestimates our number of raiders by a factor of over 2. Wowjutsu counts us as 164 raiders, when in reality we only have 73 people who ever visited Karazhan. It even counts my warrior as a raider, of whom I am 100% certain that he never set a foot into Karazhan.

There are far less than 2.4 million players who visited Karazhan. Far less than 50% of the player base ever got the chance to raid.

By counting every level 70 character in a guild with raiders as a raider, Wowjutsu seriously overestimates the number of raiders in WoW. There might be tightly organized guilds in which every character is a raider. But in general more guilds are like mine, having a mix of raiders and non-raiders, and Wowjutsu counts them all as raiders. By extension that means that not only the assumption that 50% of players already visited Karazhan is too high, but also there are less people who have visited the Black Temple. Wowjutsu counts every level 70 character in a guild in which at least one person has Black Temple loot as having visited the Black Temple, and thus lists 7% of their 2.4 million "raiders" as having gone there. But there are certainly guilds in which people are at different stages of the raid circuit, and only the top guys have visited the Black Temple. Raiding remains far too inaccessible to WoW players, and the top level raid dungeons have only been visited by a far too tiny number of players.
Comments:
Instance statistics are on a per-guild basis, not a per-person basis.

7% of all guilds that raid have visited the Black Temple.

It's actually more interesting to see where guilds are having trouble, rather than just looking at the numbers. 53% of raiding guilds have downed Gruul, which is a pretty decent accomplishment.

To be honest, I'm not sure why there's such a drop-off from Gruul to SSC. Hydross/Lurker aren't that hard in comparison.
 
hmm tobold arent you running the risk of getting bad/vicious comment from WOW raiders who get riled by your post ?

i agree though, raiding is not for everybody. i just curious why blizzard focus too much on raid when there is ppl who love to see other avenue for endgame..

off course making a raid instance is way easier than expanding the WOW program.
 
Interesting conclusions, since WoW Jutsu does track loot on a per-character basis.

If you look at the WoW Jutsu page, you'll see that it doesn't claim that every player it tracks is a raider. This is your own misconception. If you look at the FAQ, you'll see that WoW Jutsu starts tracking a guild as soon as you have more than 10 level 70ies (ergo, theoretically enough people for Karazhan) and got roughly halfway through Kara, one could assume that the percentage of characters who have seen Karazhan is 100%. But as you can see from the page itself, it's not. It's 98.12%. So a more accurate claim would be that there's almost 2.4 million characters in raiding guilds, not that there's 2.4 million raiders.

My guild has both Black Temple raiders and people who have never entered Karazhan, and WoW Jutsu has no problem separating the two. If anything, it partially underestimates the real figures. For example, it doesn't know that I've ever been to Serpentshrine Cavern or The Eye, since I've never worn loot that I've gotten from there long enough for WoW Jutsu to detect it. It also skips all of the bosses that I've never gotten loot from, even if I must have killed them to proceed to later bosses. For example, I must have killed Vashj, A'lar, Kael'thas and Rage Winterchill to be able to get loot from the Illidari Council.

I hope this doesn't count as "bad/vicious comment from WoW raiders who get riled by your post". But if you draw bad conclusions from false assumptions, you have to expect that people might get a bit annoyed. Of course, that's no reason to self-censor yourself.
 
Interesting conclusions, since WoW Jutsu does track loot on a per-character basis.

I said so. Wowjutsu tracks loot on a per-character basis. But the 2.4 million people it lists as raiders on its front page is the sum of all level 70 characters in raid guilds, not the sum of all characters owning raid loot. Or as I specifically explained with the example of my guild, it counts 164 characters as raiders in my guild, when in reality we only have 73.
 
i find it interesting that wowjutsu tells me i got 7 raiding chars even though none of them has ever set foot in a raid instance or even got a kara key.
 
Per guild estimations are also over-estimating.

On my serve for example, only one raid group uses the guild-structure. The rest are "alliances" where people from many guilds join together to form a raid-group. Someone on Bosskiller has important WoWjutsu data and the number of supposed raid groups on my server has exploded since.

If one knows ones local raid group structure, one can get fairly good guesses at the actual numbers but it takes a lot of extra information that WoWjutsu can't automatically extract.

It doesn't really matter though. It's quite obvious that raiding isn't as accessible as it could be, independent of whether Kara is seen by 25% or 50%, or if 1% or 3% of raiders has killed Illidan.
 
I said so. Wowjutsu tracks loot on a per-character basis. But the 2.4 million people it lists as raiders on its front page is the sum of all level 70 characters in raid guilds, not the sum of all characters owning raid loot. Or as I specifically explained with the example of my guild, it counts 164 characters as raiders in my guild, when in reality we only have 73.

And I said that it's a misconception that WoW Jutsu tracks raiders. It doesn't. It tracks raiding guilds and their members. One can be a member of a raiding guild and not be a raider, and WoW Jutsu does take that distinction into account.

i find it interesting that wowjutsu tells me i got 7 raiding chars even though none of them has ever set foot in a raid instance or even got a kara key.
Are members of your guild raiding? If they are, then WoW Jutsu will take your characters into account, but it does not claim that those characters are raiders. If WoW Jutsu was tracking raiders and that the criteria for being a raider was that you have visited Karazhan, then the statistics should show that 100% of the people have visited Karazhan. This is not the case.
 
To recap: If you are writing an article on how to interpret WoW Jutsu numbers, it is a good idea to check your own assumptions on the proverbial door. Being in a guild that raids does not mean that you are a raider.
 
tobold you really are too focussed on the numbers. numbers are numbers and do not carry any weight when you don´t exactly know what the circumstances are.
if you were a blizzard officer and you had to decide if raiding is too inaccessible or not, you would have to know if the people, that infact never visited kara, didn´t go there either because they were not interested or because the entry barrier was too high.
in my guild, for example, we have black temple raiders that downed illidan some weeks ago, we have pure pvp players that fight for the top spots in arena ladder and we do have "relaxed" people that just don´t care for both and just play around "inefficient" with no goal ingame just for fun.
in this guild zul aman, kara, and even ssc is like :"hey who wants to do X , im bored ? " so everyone in our guild could just jump into kara and collect loot with ease.
anyhow 20-30 people didn´t see kara
till now .
is it that the accessibility is too low ? hell no they simply don´t care for raiding in general.

this numbers are as significant as:
only 1% of the worlds population make good car drivers ! (while not listing that only 1,5% even possesses a car :P)
 
Raiders are very territorial. They've been babied for years and are use to getting most of the development time. They hate casual players with a passion, because they're worried that developers might focus on other parts of the game, instead of creating new raid instances for them.

More solo content please!
 
To recap: If you are writing an article on how to interpret WoW Jutsu numbers, it is a good idea to check your own assumptions on the proverbial door. Being in a guild that raids does not mean that you are a raider.

Shalkis, I have honestly no idea what your point is. You say exactly what I am saying, that not all members of a raiding guild are raiders. And thus that people who claim that there are 2.4 million raiders in US/Euro WoW are wrong.

Of course it is possible that *some* of the non-raiders simply don't raid because they don't like raiding. But the difficulty of raiding certainly poses a huge barrier to entry. I can't believe that only 1% of players *want* to see BT. I'd say many people want to go there, but they *can't*.
 
Looks like shalkis is counting Wowjutsu's "Total Number of Raiders". (i.e. 2.4 million).

Meanwhile, the point Tobold is making is that much fewer than 2.4 million toons (maybe 1 million?) have gone to Kara.

My question - how can you be considered a "raider" if you haven't even been to Kara. What are you raiding? MC or AQ? Lol!
 
Look at it this way: WoWjutsu can count you as a raider if you are in a 'raiding guild' (however it determines that), but will never class you as a non-raider if you have raid loot (unless you've sharded all of that gear already).

WOWJutsu will always overestimate the number of raiders, QED.
 
Let me rephrase then: You spend the whole article criticizing WoW Jutsu for incorrectly labeling characters raiders.
For example:
But in general more guilds are like mine, having a mix of raiders and non-raiders, and Wowjutsu counts them all as raiders.
WoW Jutsu makes no such claim. You do. It's fair and proper to criticize them for not factoring out alts (although I can't see how they could do that), but you can't criticize them for improper labeling if it's you doing the labeling.
 
I didn't read the article as criticism of WoWJutsu, but as criticism of uncritical and false interpretation of WoWJutsu data.

It is well known that WoWjutsu overestimates, it should be fine to state that.

WoWJutsu could actlly do better, by providing individual data, which it actually scans, before merging it on a guild level. That data is less prone to false assumptions that guild-labels reflect raid-groups.

That too is no criticism but if anything a suggestion for improvement!
 
It would be good to have a count of how many people have at least one piece of raid loot. But that would be *under*estimating the total number of raiders. Because you can well visit a raid dungeon and kill a boss without getting any loot.

The post was not at all about criticizing Wowjutsu (they are doing a great job in what they are trying to attempt, which just isn't about counting total number of raiders). But many people misinterpreted these data as being the total number of raiders, even if Wowjutsu never called them that. Wasn't me who was doing the misrepresenting.
 
tobold you really are too focussed on the numbers. numbers are numbers and do not carry any weight when you don´t exactly know what the circumstances are.
if you were a blizzard officer and you had to decide if raiding is too inaccessible or not, you would have to know if the people, that infact never visited kara, didn´t go there either because they were not interested or because the entry barrier was too high.


however if you make a product and most people don't use or experience it you do have to decide if the work to reward ratio is really worth it. content is there to give players something to do. If most can't or won't get to the content then is it worth producing in it's current format?
 
Tobold I am curious of drew your conclusions from just your guild page or from the guys behind the website? I have not received a reply from them to get a more definitive answer on the front page information.

Anyways, we can cut the number of players tracked in half and STILL end up with a much higher percentage of player participation in raiding from pre-TBC data.

I've been a trumpeter of these numbers fairly hard on this blog and elsewhere, because I think WoWJutsu has some hard data we can work with. We just need to nail down some specifics.

Just to counteract your guild's numbers, my guild has 139 level 70's. 46 have no recorded raid loot, leaving 93 with at least Kara loot. On average those 93 have gained raid loots each.

I'll admit that originally I made the mistake of not understanding the ranked member status.

Hopefully we can get some better analysis of the WoWJutsu data. Specifically number of characters with > 1 raiding item (which would eliminate some of the one-time raiders).

Still, I stand by my analysis that TBC raiding participation is at a significantly higher level than pre-TBC.

I also believe that Blizzard can still do more to bridge the gap that is evident between the entry-level raiding and the next steps.

In my opinion, Blizzard has figured out the entry-level raiding with Karazhan.
 
And comments are getting truncated again on blogger... woo woo. 93 members in my guild have received an average of 10 raid loots each.
 
"however if you make a product and most people don't use or experience it you do have to decide if the work to reward ratio is really worth it. content is there to give players something to do. If most can't or won't get to the content then is it worth producing in it's current format?"

well that is not 100% correct.
in wow you have choices,so lets say 50% of the gamers choose to not raid but to pvp or just level some more alts, because it is more fun to them; does that mean the raid content is not accessible or bad designed ?
i don´t want to say that out of that 50% there isn´t a single player that wants to raid but can´t get into it. it is 100% certain that there are people like that, but i really think that those are in the minority. i believe that the entry barrier is just an amount of effort you have to put into that part of the game. and everybody estimates for himself if the effort is worth it compared to spending the effort in other ares of the game, like pvp.

as long as you have choices, people will neglect some parts of the game.
in my opinion it is dangerous to change the raidcontent to a degree, so it suits everyone, cause you will end up with raidcontent that pleases noone.
 

Are members of your guild raiding? If they are, then WoW Jutsu will take your characters into account, but it does not claim that those characters are raiders. If WoW Jutsu was tracking raiders and that the criteria for being a raider was that you have visited Karazhan, then the statistics should show that 100% of the people have visited Karazhan. This is not the case.


The criteria for being a raider is certainly not visiting Karazhan, it's visiting any raid instance the site tracks and getting loot from there. 1.82% sounds small enough to be from those fairly rare cases where people have loot from somewhere else but not from Karazhan. It does not sound small enough to account for non-raiding guild members in all guilds to me but your opinion may differ.

Then again, someone claimed the instance statistics only are on a per-guild basis. That fits the numbers rather well too and it does sound more sensible to count how many guilds have cleared instances/bosses instead of players. WoWJutsu doesn't explicitly tell which one it is counting though.

In any case Tobold's main point was about how the character totals seemingly include each and every member of a guild that has at least one member who has a piece of loot from some raid instance. Counting it that way means this number isn't probably even close to the actual number of EU/US WoW-players that raid.
 
Why all these irate comment from raiders ? why defend blizzard's raid-only endgame focus ? to tell the truth , not everyone loves raiding and thats the fact, and ppl who like to raid must have a good guild or friends that wants them.

The barrier of raiding is high and it is not for everyone. and the tedium of REPEATING over and over for gear drop get old fast with lot of ppl. Many player will just get bored by the tedium and look for other raid instance. I think GEAR-BASED raid check should be replace with SKILL-BASED raid check.

Just look at ppl afk-ing in BG just to farm honor point. They see easy path to get nice gear from pvp. The whole END-GAME of WOW is just an endless treadmill of GEAR advancement.

i waited for long long time (2yrs) and still dont see new feature added to WOW, there is nothing like GUILD HALL, housing, more area , more horizontal / fluff advancement. My patience with blizzard developer already runs out. endless raiding just for gear isnt my cup of tea..

for info, i dont hate raid (MC,ZG,BWL,AQ all done) but doing them repeatedly for gear update burns me out already and i dont want to do that again.
 
well that is not 100% correct.
in wow you have choices,so lets say 50% of the gamers choose to not raid but to pvp or just level some more alts, because it is more fun to them; does that mean the raid content is not accessible or bad designed ?
i don´t want to say that out of that 50% there isn´t a single player that wants to raid but can´t get into it. it is 100% certain that there are people like that, but i really think that those are in the minority. i believe that the entry barrier is just an amount of effort you have to put into that part of the game. and everybody estimates for himself if the effort is worth it compared to spending the effort in other ares of the game, like pvp.

as long as you have choices, people will neglect some parts of the game.
in my opinion it is dangerous to change the raidcontent to a degree, so it suits everyone, cause you will end up with raidcontent that pleases noone.


but the problem here is that the number of raiders is pretty low. And if you take out the people that raided till they got one drop or just the ones that raided but never got any gear the number gets smaller. I've had characters in hard core raiding guilds and you begin to feel like everyone raids because everyone you deal with raids. But that just isn't the case. There are a lot of people that just don't because of time constraints, lack of desire, the fact that pvp is a more efficient path to gear. I'd bet at some point if blizzard released the numbers they'd be pretty pathetic. And I base that on the fact that anytime the numbers are good blizzard shoves them in your face and trumpets to the heavens.
 
yeah,it's not 100% correct,but the number will be not huge difference.
 
Not sure if I should wade into this or not, since I have no dog in this hunt... :)

I didn't see anyone specifically point out that it is well-known that wowjutsu has no way to separate toons by account. Since there can be multiple toons in one account, that will skew the numbers in a way that cannot be tracked by wowjutsu's methods.

I have a raiding priest and a non-raiding warrior in the same guild on my single account. I know people in the guild who have multiple raiding toons on one account. I know a person who within one account has three level-70 raiding toons over three guilds and two servers.

My point is that there is really no way to map WoW's accounts to wowjutsu's raiding numbers to arrive at anything near to a percentage-of-raiders number. Given that multiple raiding toons within one account does not seem to be unusual, it appears likely to me if a person mistakenly assumes wowjutsu's is one-toon-to-one-account, they will greatly overestimate the percentage of raiders in WoW.

===

Two things that might lead to an underestimate of raiders:
1) Until recently, was that the WoW Armory was not properly updating.
2) D/E-ed gear is not tracked.

===

An entirely separate question (in terms of raiding accessibility) is:
"What is a raider?"

For example, one might make the argument that Tobold's incursion with a well-organized guild who has Kara on farm was not really raiding, but a farming run. He saw the content, and got the loot, and was in a raid instance, wowjutsu will track him as a Kara raider -- but as other raiders pointed out, the cutting-edge raiders are learning new bosses and experiencing the challenges of figuring it all out together (in other words, wiping a lot :) ). Of course, wowjutsu doesn't make any attempt to track separate farming from raiding (assuming one even thinks there is a difference).

This question becomes more pointed when one notices the 'raid access' numbers drop off dramatically: On my realm (both factions included), only one guild has "finished" (the guild has about 130 level-70 toons), six guilds are working on MH/BT, and then it drops off to TK/SC where there are about 25 guilds who have downed more than one boss in TK/SC.

The realm is well-populated -- twice in the past two weeks I've had a weekend log-on queue. The warcraftrealms census reports over 7,800 level 70 toons (but I have no idea how accurate that is :) ).

So my realm has one guild that has "beaten the game" after a year, six that are in MH/BT, and about 30 that are seriously into 25-man content.
You be the judge of "raiding accessibility" in WoW 2.0... :)
 
Shalkis, it is mind boggling how dumb you are.
 
Why all these irate comment from raiders ? why defend blizzard's raid-only endgame focus ? to tell the truth , not everyone loves raiding and thats the fact, and ppl who like to raid must have a good guild or friends that wants them.
Believe me, the feeling is quite mutual, especially on roleplay servers. Not that I was defending Blizzard's "raid-only endgame focus", mind you. My "irate comments" were not directed at non-raiders, those were directed towards people who misrepresent statistics in an attempt to justify their own viewpoint, no matter what that viewpoint might be.

Until WoW Jutsu, discussions on the relative importance of raiders vs non-raiders for Blizzard were always somewhat.. nebulous. There simply weren't any representative statistics available. But now there is, and Blizzard's "raid focus" might or might not start making sense to people.

Personally, I think it's less about concrete numbers than it's about early adopters and feedback loops between raiders and developers. Raiding is one of the few parts of the game where it's relatively easy to gather hard, reproducible data and form an informed opinion on that. And when you turn over that data to developers, they are highly likely to listen. It's much harder to form a piece of compelling and constructive criticism on other areas of the game. Which is a pity, really. On those occassions when someone succeeded in that elusive task, the results were generally good and benefited a lot of players, raiders included.

Shalkis, it is mind boggling how dumb you are.
Welcome to my fanclub. I'll be sure to remember you the next time I find myself holding aggro in battlegrounds.
 
I came too late, Shalkis had already quoted and commented on the comment from "anonymous". Otherwise I would have just deleted it. Read my terms of services, insults like that are not permitted, especially not from people who are too yellow to put their name.
 
Another big problem is guilds that are ignored by WoWJutsu.

My guild has been raiding Karazhan for 4 months straight and was listed with WoWJutsu 25 dec 2007.

Not one single member of the guild has been updated since december despite having manualy requested an update.

I know for sure that our maintank use his kara loot always as I do not think he has another gearset and I know I have been using my loot on logout everyday for several weeks.

Check our jutsu page and then compare to armory, it has not even updated our memberlist.

http://www.wowjutsu.com/eu/darkspear/Swedish%20Alliance/roster

And I have written to the site maintainer and recieved a response that he would manualy see to it that we get updated with no result.

I have not recieved any reply the last 3 months thou.

We have cleared all but netherspite multiple times and at least 30+ members or so have kara loot.
 
Post a Comment

Links to this post:

Create a Link



<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool