www.Tobold.com
Tobold@GMail.com
A blog with my thoughts regarding games I am playing and other stuff in life. Please read my Terms of Service
RSS Feed
Tuesday, January 21, 2014
Guest Post: How do MMOs reach $200m budgets?
Guest post by Hugh Hancock:
---
In 2009, I started making a World of Warcraft fanfilm, Death Knight Love Story, which I estimated would take 6 months to make. I just released the first part of it this morning. Total development time: 5 years. The budget for the project has gone the same way: from a few thousand dollars (perfectly reasonable for a project aimed at learning how to use motion capture) to ... let's just say "more".
Why would you be interested in this? Because one of the topics we end up debating a lot in the games blogging world is the way in which AAA games' budgets spiral out of control.
Just a week or so ago we heard that The Elder Scrolls Online may have spiralled up to $200m. Tobold has spent a lot of time looking at the economics of game business plans in the past, and how - and if - they'll make their money back. But one thing that I've rarely seen discussed is just how the hell budgets get that high in the first place.
The Death Knight Love Story experience has given me a solid insight into exactly how that happens. Death Knight Love Story is, in fact, very similar to an AAA game in many ways. he level of art quality is the same - including both the WoW art but also the original art I've had made for it. We worked with Hollywood actors to produce the voice for it, just like an AAA title. And Death Knight Love Story is such a huge opportunity for me and my movie-making that I couldn't afford to screw it up. Just like, after spending even $100m, the developers of TESO must have known they couldn't afford to screw it up.
Imagine you're the executive producer of TESO. You've got an early alpha version of the game, and you've sat your focus groups down to play through it. And the result comes in: it's not very fun. It's, you know, OK.
What do you do?
In the case of Death Knight Love Story, we actually threw nearly-finished versions of the film away three times over. If the opportunity hadn't been so big, I would have released a far earlier version and lived with the rough edges. But because I had managed to land such high-profile actors - Brian Blessed, Jack Davenport, Joanna Lumley and Anna Chancellor - I was very aware that I had to get the film just right.
For the TESO team, and particularly the heads of development, the pressure is even more intense. Games development works on an iterative process right from the start, so there's probably nothing in the game that hasn't been thrown away or at least heavily revised three times over. And the problem is that every time you do this, the stakes get higher. The game has now spent more money, so it's even more important that it be a hit. So the focus-testing becomes even more stringent and there's an even higher bar of fun to hit. And so on.
If you're trying anything sufficiently ambitious in the game or film world, this sort of scope creep is almost impossible to avoid. Why? Because if you're aiming to create a detailed 3D world of any kind, you are by definition attempting something hugely ambitious and massively complex - and worse, something that's fractal. The deeper in you go, the more complex it becomes.
Let's go back to our poor TESO developers.
The focus group results come in. The design team sits down to look at them. And the design lead says "Hmm, what we need here is a new settlement in Morrowind where the levelling curve can flatten out a bit.". Problem solved! The quests team go away, and they design the settlement in rough. It turns out that this will be a really key part of the game's progression, so they're going to need to introduce new art elements. So the concept artists come in and create drawings for the characters and buildings of the settlements. And the quest teams come in, and then the 3D art guys start adding things up.
Have you ever looked closely at the number of models in any World of Warcraft location? For example, much of Death Knight Love Story is set in the Ebon Hold in World of Warcraft, so I have the model to hand. The Ebon Hold is a reasonably simple two-floor structure, made of 6 main model parts. Six models for the building - not too bad. But that's just the main structure - walls, floor and so on. If you just use that model, it'll look very bare: in a 2001 or so game it might have been fine, but in 2014 we demand a lot more detail. So the Hold needs anvils, sacks of grain, forges, and so on.
The Ebon Hold model has 426 models for anvils, grain sacks, runeforges, etc, etc, etc. Each of which has at least one 3D texture - probably more. Each of those textures had to be drawn by an artist, taking a couple of hours, then each model had to be assembled, for another couple of hours, then it had to be reviewed to check that it fits with the art style. Some models had to be revised, maybe a couple of times. Then the models had to be processed into a form that the game engine can understand - a surprisingly non-trivial problem. Some of them will throw errors, and have to be fixed. Then a level designer came in and placed the models for each area. They probably realise they need a couple of models they don't have, so they submit a request for a new model, and that has to be approved, and concept art created, and so on. Then the gameplay designers come in, and design the gameplay for the area, necessitating a few thousand lines of scripting code.
Then the area is tested, and it turns out that some of the models break the AI pathing, or get in the way of the player's movement, so they have to go right back to the start, and...
Even a single building in a game like WoW is a massive work of art.
And how many of those are there? Well, let's put it this way: in Wrath of the Lich King, for example (an expansion, not a full game), the continent of Northrend alone is a grid of squares, 49 squares wide by 37 across. Each of those squares has been designed by hand. Almost all of those squares have at least some structures and some placed buildings in, as well as character spawning points, quest NPCs, special effects, and so on. So that makes a total of 1131 squares, within each of which there are probably a few hundred 3D models and a few thousand lines of code. That makes a grand total of somewhere in the order of 350,000 individually placed models, many of them unique, and literally millions of lines of quest scripting.
And that was just part of one expansion pack.
When I was making Death Knight Love Story, we'd frequently experience the fractal nature of 3D design. We'd come up with a simple-sounding change, like adding an additional small scene, and suddenly that would turn into weeks or months of work. We needed to create new characters, export a new location, update the textures, then total up all the animation needed by all the characters in the scene, organise a motion capture session, process the motion capture data, insert dozens of motion capture files into the scene... And all of this from one phrase casually spoken in one development meeting.
Of course, everyone on a AAA team will be aware that 3D products are complex. But when the pressure's on, and the game or film is clearly not as successful as it needs to be, it's very easy to say "well, if we just add this one thing...". And sometimes - often - you actually need to say that in order to make a great product. And it's impossible to tell whether the months of work you're authorising will be the months that make the project incredibly successful, or if they'll get to the next focus group round and then get discarded. I have entire scenes which took weeks or months to create which are just sitting on my hard drive now - the cost of making a great project.
In the course of revision-driven, throw-it-away-and-start-again hell, you genuinely have no idea whether what you're making will be a hit or a flop. All you know is that you're not at "good" yet, and the only way to get there is to keep going. The first Half-Life famously took years because Valve threw it away half-way through and started again. World of Warcraft's development started in 1999, with the game being released in 2005 - it's reasonable to assume that there were quite a few iterations and dark times during the process.
Unfortunately, AAA game development, just like movie development, is a very high-stakes game. The studios can spend $200m and they might still end up with a dud - but if they didn't do that, and released the game when their testers were saying it was terrible, they'd have even worse odds of making their money back.
As for my own experience with development hell - also 5 years, coincidentally enough - I'll leave it up to you to decide if it finally got to "good enough"! You can check it out at www.deathknightlovestory.com .
---
In 2009, I started making a World of Warcraft fanfilm, Death Knight Love Story, which I estimated would take 6 months to make. I just released the first part of it this morning. Total development time: 5 years. The budget for the project has gone the same way: from a few thousand dollars (perfectly reasonable for a project aimed at learning how to use motion capture) to ... let's just say "more".
Why would you be interested in this? Because one of the topics we end up debating a lot in the games blogging world is the way in which AAA games' budgets spiral out of control.
Just a week or so ago we heard that The Elder Scrolls Online may have spiralled up to $200m. Tobold has spent a lot of time looking at the economics of game business plans in the past, and how - and if - they'll make their money back. But one thing that I've rarely seen discussed is just how the hell budgets get that high in the first place.
The Death Knight Love Story experience has given me a solid insight into exactly how that happens. Death Knight Love Story is, in fact, very similar to an AAA game in many ways. he level of art quality is the same - including both the WoW art but also the original art I've had made for it. We worked with Hollywood actors to produce the voice for it, just like an AAA title. And Death Knight Love Story is such a huge opportunity for me and my movie-making that I couldn't afford to screw it up. Just like, after spending even $100m, the developers of TESO must have known they couldn't afford to screw it up.
Imagine you're the executive producer of TESO. You've got an early alpha version of the game, and you've sat your focus groups down to play through it. And the result comes in: it's not very fun. It's, you know, OK.
What do you do?
In the case of Death Knight Love Story, we actually threw nearly-finished versions of the film away three times over. If the opportunity hadn't been so big, I would have released a far earlier version and lived with the rough edges. But because I had managed to land such high-profile actors - Brian Blessed, Jack Davenport, Joanna Lumley and Anna Chancellor - I was very aware that I had to get the film just right.
For the TESO team, and particularly the heads of development, the pressure is even more intense. Games development works on an iterative process right from the start, so there's probably nothing in the game that hasn't been thrown away or at least heavily revised three times over. And the problem is that every time you do this, the stakes get higher. The game has now spent more money, so it's even more important that it be a hit. So the focus-testing becomes even more stringent and there's an even higher bar of fun to hit. And so on.
If you're trying anything sufficiently ambitious in the game or film world, this sort of scope creep is almost impossible to avoid. Why? Because if you're aiming to create a detailed 3D world of any kind, you are by definition attempting something hugely ambitious and massively complex - and worse, something that's fractal. The deeper in you go, the more complex it becomes.
Let's go back to our poor TESO developers.
The focus group results come in. The design team sits down to look at them. And the design lead says "Hmm, what we need here is a new settlement in Morrowind where the levelling curve can flatten out a bit.". Problem solved! The quests team go away, and they design the settlement in rough. It turns out that this will be a really key part of the game's progression, so they're going to need to introduce new art elements. So the concept artists come in and create drawings for the characters and buildings of the settlements. And the quest teams come in, and then the 3D art guys start adding things up.
Have you ever looked closely at the number of models in any World of Warcraft location? For example, much of Death Knight Love Story is set in the Ebon Hold in World of Warcraft, so I have the model to hand. The Ebon Hold is a reasonably simple two-floor structure, made of 6 main model parts. Six models for the building - not too bad. But that's just the main structure - walls, floor and so on. If you just use that model, it'll look very bare: in a 2001 or so game it might have been fine, but in 2014 we demand a lot more detail. So the Hold needs anvils, sacks of grain, forges, and so on.
The Ebon Hold model has 426 models for anvils, grain sacks, runeforges, etc, etc, etc. Each of which has at least one 3D texture - probably more. Each of those textures had to be drawn by an artist, taking a couple of hours, then each model had to be assembled, for another couple of hours, then it had to be reviewed to check that it fits with the art style. Some models had to be revised, maybe a couple of times. Then the models had to be processed into a form that the game engine can understand - a surprisingly non-trivial problem. Some of them will throw errors, and have to be fixed. Then a level designer came in and placed the models for each area. They probably realise they need a couple of models they don't have, so they submit a request for a new model, and that has to be approved, and concept art created, and so on. Then the gameplay designers come in, and design the gameplay for the area, necessitating a few thousand lines of scripting code.
Then the area is tested, and it turns out that some of the models break the AI pathing, or get in the way of the player's movement, so they have to go right back to the start, and...
Even a single building in a game like WoW is a massive work of art.
And how many of those are there? Well, let's put it this way: in Wrath of the Lich King, for example (an expansion, not a full game), the continent of Northrend alone is a grid of squares, 49 squares wide by 37 across. Each of those squares has been designed by hand. Almost all of those squares have at least some structures and some placed buildings in, as well as character spawning points, quest NPCs, special effects, and so on. So that makes a total of 1131 squares, within each of which there are probably a few hundred 3D models and a few thousand lines of code. That makes a grand total of somewhere in the order of 350,000 individually placed models, many of them unique, and literally millions of lines of quest scripting.
And that was just part of one expansion pack.
When I was making Death Knight Love Story, we'd frequently experience the fractal nature of 3D design. We'd come up with a simple-sounding change, like adding an additional small scene, and suddenly that would turn into weeks or months of work. We needed to create new characters, export a new location, update the textures, then total up all the animation needed by all the characters in the scene, organise a motion capture session, process the motion capture data, insert dozens of motion capture files into the scene... And all of this from one phrase casually spoken in one development meeting.
Of course, everyone on a AAA team will be aware that 3D products are complex. But when the pressure's on, and the game or film is clearly not as successful as it needs to be, it's very easy to say "well, if we just add this one thing...". And sometimes - often - you actually need to say that in order to make a great product. And it's impossible to tell whether the months of work you're authorising will be the months that make the project incredibly successful, or if they'll get to the next focus group round and then get discarded. I have entire scenes which took weeks or months to create which are just sitting on my hard drive now - the cost of making a great project.
In the course of revision-driven, throw-it-away-and-start-again hell, you genuinely have no idea whether what you're making will be a hit or a flop. All you know is that you're not at "good" yet, and the only way to get there is to keep going. The first Half-Life famously took years because Valve threw it away half-way through and started again. World of Warcraft's development started in 1999, with the game being released in 2005 - it's reasonable to assume that there were quite a few iterations and dark times during the process.
Unfortunately, AAA game development, just like movie development, is a very high-stakes game. The studios can spend $200m and they might still end up with a dud - but if they didn't do that, and released the game when their testers were saying it was terrible, they'd have even worse odds of making their money back.
As for my own experience with development hell - also 5 years, coincidentally enough - I'll leave it up to you to decide if it finally got to "good enough"! You can check it out at www.deathknightlovestory.com .
- posted by Tobold @ 6:14 PM Permanent Link
Comments:
<< Home
Newer› ‹Older
I'll come back and praise DKLS after I have seen it; this is a surprise!
I have read some console devs commenting on HD being more expensive. Similarly, more polygons are expected these days. I.e. the costs per "square meter" / toon / second are going up.
It not only balloons costs but raising the estimating risk. Each of ten components could be done on the first go, or each could take 3 but we hope the average is 1.5. So do you budget/plan for 10, 15, 30? And the difference between 100 and 300 million is non-trivial money. The stress is when the devs are told they can only have 100mm and they could make it for that so they start and then reality hits.
And a game that takes $200mm to develop is actually at least a $250mm game since if you spent $200 on it you are going to want to do some PR and marketing.
And these cost inevitably lead to: You can have a AAA or a niche game but not both.
P.S.: there are also time risks. If you were looking for this film to fund your retirement vineyard in Provence, then it would have been preferable if it came out when LK was the current expansion and there were 13m players. Something I think the just-started-shooting WoW movie may feel.
I have read some console devs commenting on HD being more expensive. Similarly, more polygons are expected these days. I.e. the costs per "square meter" / toon / second are going up.
It not only balloons costs but raising the estimating risk. Each of ten components could be done on the first go, or each could take 3 but we hope the average is 1.5. So do you budget/plan for 10, 15, 30? And the difference between 100 and 300 million is non-trivial money. The stress is when the devs are told they can only have 100mm and they could make it for that so they start and then reality hits.
And a game that takes $200mm to develop is actually at least a $250mm game since if you spent $200 on it you are going to want to do some PR and marketing.
And these cost inevitably lead to: You can have a AAA or a niche game but not both.
P.S.: there are also time risks. If you were looking for this film to fund your retirement vineyard in Provence, then it would have been preferable if it came out when LK was the current expansion and there were 13m players. Something I think the just-started-shooting WoW movie may feel.
@Hagu - Yes, I definitely agree on ALL those points.
In fact, I wanted to touch on the marketing point in the post, but it was already starting to resemble a short book more than a long blog post...
Time risks: I very much agree. I think Death Knight Love Story will definitely have suffered from being finished such a long time after Wrath - and games can often go the same way.
In DKLS' case, I think the improved quality was worth it, but it's a hell of a risk calculation to make.
In fact, I wanted to touch on the marketing point in the post, but it was already starting to resemble a short book more than a long blog post...
Time risks: I very much agree. I think Death Knight Love Story will definitely have suffered from being finished such a long time after Wrath - and games can often go the same way.
In DKLS' case, I think the improved quality was worth it, but it's a hell of a risk calculation to make.
There's an old quote I like. I don't remember the exact words or who said it but it is something like "art isn't finished until you are done cutting away." You start with an ambitious idea, slowly figure out what the best parts are, and remove the bad parts. For a movie or a game, it might mean it is much shorter than you originally planned, but at least it will be good.
# posted by 
: 22/1/14 01:11
: 22/1/14 01:11
After they have created xxxx objects, can't they just copy those in a ratio that will not become easily noticable from the average player? I am mostly referring to the game world size...Does it really need to be 100% handcrafted with new objects in every square?
I mean if you design 30 different trees, does it matter if you have 4 zones with them or 20?(same goes to other 3d objects). So having a beautiful and huge virtual world is the thing that cost the most in MMOs?Or the part "huge" can somehow be less costy because we can just copy objects?
I mean if you design 30 different trees, does it matter if you have 4 zones with them or 20?(same goes to other 3d objects). So having a beautiful and huge virtual world is the thing that cost the most in MMOs?Or the part "huge" can somehow be less costy because we can just copy objects?
@giannis: procedural worlds are a neat solution, the only problem is that they look like..... Procedurally generated worlds..... :)
I sometimes wonder if project "resets" and non-product specific expenditure is included in this total.
For example, TESO is a first time mmo. How much of that $200m is actually non-TESO specific spend such as the creation of generic online services. That sort of stuff would have a realisable value obtainable outside the TESO product but is probably being included in the program costs.
If you see a product is rubbish and throw it away, do you keep the incurred costs allocated to a program the only thing that remains is a name? What about if you retain a small fraction of the assets? These sunk costs can easily lead to a situation where you've spent $100m on a product that is so bad it is worthless, but spending a further $100m to redo everything and produce a product that will take $125m. Net loss of $75m but probably the right call on a financial basis.
For example, TESO is a first time mmo. How much of that $200m is actually non-TESO specific spend such as the creation of generic online services. That sort of stuff would have a realisable value obtainable outside the TESO product but is probably being included in the program costs.
If you see a product is rubbish and throw it away, do you keep the incurred costs allocated to a program the only thing that remains is a name? What about if you retain a small fraction of the assets? These sunk costs can easily lead to a situation where you've spent $100m on a product that is so bad it is worthless, but spending a further $100m to redo everything and produce a product that will take $125m. Net loss of $75m but probably the right call on a financial basis.
# posted by : 23/1/14 09:56
: 23/1/14 09:56
One thing that people need to separate in their heads concerning game development is that the computer programmers --who generate the bones used by the art/quest/integration people-- are a completely separate charge. Sure, they have to tweak things and make incremental improvements from time to time, but if you've got an engine that works (like WoW) you don't have to rebuild it completely from expac to expac. The water and the seamless integration in the original non-fly vanilla zones probably represented the biggest code development in WoW since Vanilla's incremental additions. (Anybody remember when rain showed up finally?)
Art can get very expensive, especially when the tools/bones used to create the art hasn't changed much over the years. You have to get creative in using those tools so that the objects --trees and whatnot, for example-- don't look like the same trees found in Vanilla. Look closely at the trees in Mists and you'll find they use the same bones as in prior editions, they're just used in a clever fashion to make it seem different.
Also, in the case of big corporations, never underestimate the ability of middle management to suck up dollars.
Hugh, thanks for the insights on your project. I'm looking forward to seeing the finished product!
Art can get very expensive, especially when the tools/bones used to create the art hasn't changed much over the years. You have to get creative in using those tools so that the objects --trees and whatnot, for example-- don't look like the same trees found in Vanilla. Look closely at the trees in Mists and you'll find they use the same bones as in prior editions, they're just used in a clever fashion to make it seem different.
Also, in the case of big corporations, never underestimate the ability of middle management to suck up dollars.
Hugh, thanks for the insights on your project. I'm looking forward to seeing the finished product!
"Art is never finished: it is only abandoned."
In the world of roguelikes they really take this seriously - a lot of people describe them as abandoned if they are not under current development. They have no room for the concept of "finished" at all!
Post a Comment
In the world of roguelikes they really take this seriously - a lot of people describe them as abandoned if they are not under current development. They have no room for the concept of "finished" at all!
<< Home
