www.Tobold.com
Tobold@GMail.com
A blog with my thoughts regarding games I am playing and other stuff in life. Please read my Terms of Service
Wednesday, June 25, 2008
Servers and critical mass
Eric from Elder Gamer says that an MMO dies when its population no longer reaches critical mass, and wants the EQ2 servers to be merged. The problem is that most games even in bad times still have "critical mass", only it is badly distributed over many servers, each of which is below critical mass. Or there is critical mass only for the most popular content. There are still enough players on any one WoW servers to get a group to Karazhan together. But there aren't enough to get a Gnomeregan run together. And with the typical summer slump coming up, on less populated WoW servers on the less popular faction it will be hard to find groups for many dungeons. The specific problem for WoW is that server mergers are probably not a good idea. Most of the absentees will come back for the Wrath of the Lich King, and then servers would suffer from overpopulation and queues if they were merged.
I'm not a tech guy. I have enough tech knowledge to install a new graphics card or RAM, or to network a printer, but I know nothing about servers and data protocols and stuff. But from a simple user perspective I have to ask: why are most MMORPGs divided into several servers? Why can't we all play together on one big single server?
Apparently it is *not* a problem of total server population. EVE Online has only one single server, with a quarter of a million players on it, and most of the time it is running fine (although the Empyrean Age seems to have caused connection problems). The problem in general appears to be server instability and lag if too many people gather at the same point. Even in EVE there are hubs like Jita, where bad things happen if too many people gather there. In World of Warcraft the opening of the gates of Ahn'qiraj caused too many players to gather in Silithus, which then caused server crashes and enormous lag. In Age of Conan siege warfare, where by definition many players gather at one place, are a buggy lagfest. Apparently when in a virtual world you go somewhere where there are many other players, data traffic goes up exponentially, because every one of those players needs to be informed of the movements of every other player, and the servers can't handle that traffic.
So I'm wondering if developers can't come up with a better system. I remember in Final Fantasy XI the data traffic simply was automatically throttled when you came to a crowded area. You stepped onto the square in front of the auction house and at first saw nobody, and then people plopped into view slowly, one after one. Of course that can't be a solution for massive PvP battles; but who says that I need to be instantly informed of every other player's action in Orgrimmar when I just want to use the bank and auction house? I think clever programming of the way data traffic is handled could seriously improve matters.
Another possible solution would be instances. World of Warcraft should have cross-server dungeons working the same way as cross-server battlegrounds already do. The majority of group content in WoW is instanced, so as long as you put in the same restrictions to trade that already exist in cross-server battlegrounds, why shouldn't people be allowed to find a cross-server group for dungeons that are rarely visited nowadays? In Age of Conan all zones are already instanced, I don't know why they also need to separate people into different servers.
The final hope is technological advances. Future servers might simply become more powerful and be able to handle more people. Right now the maximum population a server can handle isn't a large multiple of the "critical mass" Eric is talking about, so even normal daily or seasonal variations of the number of concurrent users mean that at some times there aren't enough people online for groups to certain destinations. If a WoW server could handle 10,000 players, there would be less problems when only half of that are online, because that would still be more than a full server now.
And maybe there are other ideas which could guarantee that a MMO always has more than critical mass for people to play together. If you have an idea, tell us!
- posted by Tobold @ 8:29 AM Permanent Link
Comments:
<< Home
Newer› ‹Older
I don't think more powerful servers will solve this problem in the future. With more power come more features. Sure, if avatars stay the way they are now, better hardware will solve the problem. But imagine the data load if you're not only informed about the movement of every other character in Orgrimmar, but also about if his face looks happy/sad usw at the moment and a million of other tiny things like that.
# posted by 
: 25/6/08 09:59
: 25/6/08 09:59
Its a simple case of scalability. Working myself in a multiserver enviroment where data has to be transactional and propagated to every serwer I find it easy to understand that its hard to implement such a system.
There is one company though and I mean Google which made such a system with large number of machines and it works nicely.
The problem is programming not equipment. I believe EVE is working on only one machine and they upgrade it constantly but it is not the way to go. Some larger company and I hope Blizzard will be up to task, should make a multiserver system for processing player data. The premise can be really simple, for ex. serwers can handle some player transparent zones and zones can even change in scale as players tend to gather in some areas. Its the way mobile phone systems work, and if devs put some work into it, they would be able to write such an engine.
Origrimmar is overcrowded? Add a server and split it into two zones each processed by different server. If you written your engine correctly this should nicely scale into almost 50% of load for each server.
I am really dissapointed that no company is trying to make anything new. Friggin AoC is just WoW 2.0, the same with WaR. The only game that is planned to make any progress in this dump of same ideas polished with new graphics is Darkfall and hell it can not even be released this century.
There is one company though and I mean Google which made such a system with large number of machines and it works nicely.
The problem is programming not equipment. I believe EVE is working on only one machine and they upgrade it constantly but it is not the way to go. Some larger company and I hope Blizzard will be up to task, should make a multiserver system for processing player data. The premise can be really simple, for ex. serwers can handle some player transparent zones and zones can even change in scale as players tend to gather in some areas. Its the way mobile phone systems work, and if devs put some work into it, they would be able to write such an engine.
Origrimmar is overcrowded? Add a server and split it into two zones each processed by different server. If you written your engine correctly this should nicely scale into almost 50% of load for each server.
I am really dissapointed that no company is trying to make anything new. Friggin AoC is just WoW 2.0, the same with WaR. The only game that is planned to make any progress in this dump of same ideas polished with new graphics is Darkfall and hell it can not even be released this century.
# posted by : 25/6/08 10:24
: 25/6/08 10:24
Every time you get a loading screen, chances are high that you're now connected to another machine (server).
The reason EVE manages to hold 30k people, is that they have a loading screen between each jump, so they can have a whole system on 1 dedicated server (which isn't always enough, like in Jita).
In Wow, it's much harder: you have 2 continents (with no loading screens while running from zone to zone), and the dungeons.
The reason EVE manages to hold 30k people, is that they have a loading screen between each jump, so they can have a whole system on 1 dedicated server (which isn't always enough, like in Jita).
In Wow, it's much harder: you have 2 continents (with no loading screens while running from zone to zone), and the dungeons.
# posted by : 25/6/08 10:26
: 25/6/08 10:26
If I'm not mistaken, the server that EVE uses is pretty amazingly powerful and constantly being added to. They claim to have the largest supercomputer in the gaming industry, for only 200K or so players. I doubt you could scale that up in to the millions without going to Google scale server capacity.
# posted by : 25/6/08 10:33
: 25/6/08 10:33
the problem lies in the fact that for 98% of the time there is no problem at all, except when an event occurs that makes people to go to 1 place at a certain time.
this will cause lag no matter how good your servers are or how it is scaled. and tbh, every sensible company doesn't want a server-park that is lagfree just because of the 2%. it will cost them more than they want.
the biggest problem is ofcourse the "unlimited" amount of people that can visit a place. you never can say that only 100people will visit point X.
therefor scaling doesn't work as well, since the set of people is infinite and the equipment or programming is not.
perhaps it's an idea to make a combined p2p and client-server architecture, where the useless data is transferred via p2p and via the server and the critical important data is going via the client-server way
this will cause lag no matter how good your servers are or how it is scaled. and tbh, every sensible company doesn't want a server-park that is lagfree just because of the 2%. it will cost them more than they want.
the biggest problem is ofcourse the "unlimited" amount of people that can visit a place. you never can say that only 100people will visit point X.
therefor scaling doesn't work as well, since the set of people is infinite and the equipment or programming is not.
perhaps it's an idea to make a combined p2p and client-server architecture, where the useless data is transferred via p2p and via the server and the critical important data is going via the client-server way
Eve's Server isn't a single machine, it's a server cluster. (EVE PR http://www.eve-online.com/pressreleases/default.asp?pressReleaseID=25 )
Eve hides this pretty good (no visible loading screens) but there are major slowdowns and lags when there are too many people in the same area.
The main problem is programming, and not just related to MMOs or other multi-server programms. Dual, quad and other multicore systems are everywhere but no one knows how to really use them. Programmer logic is mostly linear: You have this problem, to solve it I do this step, then the next and so on... programming multicore processors needs parrallel problem solving strategies and there is still a lot research and experience needed.
Don't forget the other problems:
Internet connections can't transfer infinite ammounts of information, clients can't handle infinite ammounts of polygons.
And then there is still the question if the idea of a one-server-MMO is a good idea for every game.
In EVE there is huge space, travelling is kept simple (from sector to sector, from station to station via autopilot/navigation) but try to imagine 2 million players meeting in Ironforge or 200.000 trying to climb a hill. You could counter theese problems with _really_ huge worlds, but how to design them so that people will use the complete space but still interact. And how many people really want to meet eachother... servers create communities and communities dont scale infinitely.
I agree wow is too small in scale but I don't think merging everything on single server-systems is a good solution.
- Ben
Eve hides this pretty good (no visible loading screens) but there are major slowdowns and lags when there are too many people in the same area.
The main problem is programming, and not just related to MMOs or other multi-server programms. Dual, quad and other multicore systems are everywhere but no one knows how to really use them. Programmer logic is mostly linear: You have this problem, to solve it I do this step, then the next and so on... programming multicore processors needs parrallel problem solving strategies and there is still a lot research and experience needed.
Don't forget the other problems:
Internet connections can't transfer infinite ammounts of information, clients can't handle infinite ammounts of polygons.
And then there is still the question if the idea of a one-server-MMO is a good idea for every game.
In EVE there is huge space, travelling is kept simple (from sector to sector, from station to station via autopilot/navigation) but try to imagine 2 million players meeting in Ironforge or 200.000 trying to climb a hill. You could counter theese problems with _really_ huge worlds, but how to design them so that people will use the complete space but still interact. And how many people really want to meet eachother... servers create communities and communities dont scale infinitely.
I agree wow is too small in scale but I don't think merging everything on single server-systems is a good solution.
- Ben
# posted by : 25/6/08 11:09
: 25/6/08 11:09
The technical aspect is just the tip of the iceberg. Directly tied with the population you want to have at peak times are parameters like mob and resource spawn rates, and the server's economy.
And that's the real limit, not the tech. With Oracle, on which WoW is reportedly built, you can create large server farms which could host a huge amount of players without any visible drop in performance on the user's end. But when there's 50 people lined up on their epic flyers doing essentially the same circle in Nagrand in 30 seconds intervals in the hope of getting lucky on a node respawn, you have an issue.
And that's the real limit, not the tech. With Oracle, on which WoW is reportedly built, you can create large server farms which could host a huge amount of players without any visible drop in performance on the user's end. But when there's 50 people lined up on their epic flyers doing essentially the same circle in Nagrand in 30 seconds intervals in the hope of getting lucky on a node respawn, you have an issue.
I dont care about loading screens. Lets say there is no loading screens.
Each request for some action on the server side can be load balanced for a palticular server. Heck there can be servers that just do the balancing and decide which server should process every request.
Zones that each server handles can be dynamic and change with player travels. Heck there can be no zones even. The only problem is how do you keep the same data on each server and propagate data as it changes. People can stand pixel to pixel and be handled by completely different servers only requirement is that the servers have the same data.
There are already systems that do much more complicated tasks than needed for an MMO seemless world. Just read some on Map Reducre created by Google and what servers they use.
Companies like Blizzard should be able to create such a world, but they dont want to spend money because people have never tasted how it feels to have a seemless world for all the players so they dont whine about it. Its just a simple incom math. They wont spend as much on developement and revenue will be the same.
Each request for some action on the server side can be load balanced for a palticular server. Heck there can be servers that just do the balancing and decide which server should process every request.
Zones that each server handles can be dynamic and change with player travels. Heck there can be no zones even. The only problem is how do you keep the same data on each server and propagate data as it changes. People can stand pixel to pixel and be handled by completely different servers only requirement is that the servers have the same data.
There are already systems that do much more complicated tasks than needed for an MMO seemless world. Just read some on Map Reducre created by Google and what servers they use.
Companies like Blizzard should be able to create such a world, but they dont want to spend money because people have never tasted how it feels to have a seemless world for all the players so they dont whine about it. Its just a simple incom math. They wont spend as much on developement and revenue will be the same.
# posted by : 25/6/08 12:10
: 25/6/08 12:10
WoW also does throttling when it comes to player models. Unfortunately, this can result in getting ganked by a raid that you didn't see because their models hadn't loaded on your computer.
Likewise, culling the number of updates that need to be sent can result in similar problems. One type of culling that WoW does is to not tell the client about a stealthed character until you can actually see that stealthed character. Unfortunately, this can result in a situation where you can literally be dead before you know it.
Kristofmr's idea about servers transparently distributing load among themselves is a good one, but it isn't miracle cure. The worst-case scenario in such a setup is when something happens on the border between two servers. If players can be affected by someone who's on an another zone, the two zone servers also need to synchronize their statuses. The problem gets exponentially worse when the area of interest needs to be split between more and more servers and all need to be kept "in the loop". But even in that scenario, it probably handles better than the current system. At least from the players' point of view.
From the developer's point of view it could be worse. They would be looking at the amount of investment required to provide acceptable service to a certain number of players. If a load-redistributing server cluster costs as much as two separate server clusters and can provide acceptable performance to less players in total.. then there will be two separate server clusters.
Likewise, culling the number of updates that need to be sent can result in similar problems. One type of culling that WoW does is to not tell the client about a stealthed character until you can actually see that stealthed character. Unfortunately, this can result in a situation where you can literally be dead before you know it.
Kristofmr's idea about servers transparently distributing load among themselves is a good one, but it isn't miracle cure. The worst-case scenario in such a setup is when something happens on the border between two servers. If players can be affected by someone who's on an another zone, the two zone servers also need to synchronize their statuses. The problem gets exponentially worse when the area of interest needs to be split between more and more servers and all need to be kept "in the loop". But even in that scenario, it probably handles better than the current system. At least from the players' point of view.
From the developer's point of view it could be worse. They would be looking at the amount of investment required to provide acceptable service to a certain number of players. If a load-redistributing server cluster costs as much as two separate server clusters and can provide acceptable performance to less players in total.. then there will be two separate server clusters.
There is one very important thing to be aware of. I was considering potential problems created by many players standing in the same area from a server side point of view with a buddy of mine and I just couldnt find a single thing that may be problematic.
The only problem is still coherent data amongst all nodes. If you can keep a scalable solution with a multiple number of nodes (servers) there is none.
Of course if you have one data source for ex. a database each server will have to connect to it and you will have a scalability problem, but there can be database nodes too and there are other solutions as well.
The only problem is still coherent data amongst all nodes. If you can keep a scalable solution with a multiple number of nodes (servers) there is none.
Of course if you have one data source for ex. a database each server will have to connect to it and you will have a scalability problem, but there can be database nodes too and there are other solutions as well.
# posted by : 25/6/08 12:30
: 25/6/08 12:30
I'm getting a serial WTF effect in this thread...
I dont care about loading screens. Lets say there is no loading screens.
If you're still talking about WoW, the only loading screens you have are transcontinental and instances. For all practical purposes, the world is seamless. I don't get what you're talking about.
The only problem is how do you keep the same data on each server and propagate data as it changes.
Unless a MMO dev is trying to reinvent the wheel and waste a huge amount of money creating its own back-end, the problem has been moot for a good decade on intel-based servers, and about three in the rest of computing. When you are talking about a server or shard or whatever on a modern MMO, chances are that the reference is actually to a virtual server instance, not a physical piece of hardware.
The worst-case scenario in such a setup is when something happens on the border between two servers. If players can be affected by someone who's on an another zone, the two zone servers also need to synchronize their statuses.
If you were meaning that a player should be able to cast a spell in Orgrimmar which should affect players in Stormwind in real time, you would have a point, but in purely pragmatical terms, the only areas where this kind of interaction could be considered is between two players on opposite sides of an instance door. You don't really need to have seamless transition / interaction between two continents...
But that wasn't what you were meaning. You were laboring under the assumption that load-balancing would place two players physically at the same in-game coordinates while being handled by two different physical servers. That doesn't happen. Several physical application servers are being tied together into one single virtual server instance, and that's eg Kalimdor - Bronze Dragonflight. All players in Kalimdor are in the same container. Updates get passed from that Virtual server into the database server instance - another bunch of physical servers tied together into a single virtual instance, and this virtual DB server itself handles reading and writing of data to a most probably also virtual storage pool.
There's plenty of tools on the market which handle the nitty-gritty aspect of it, and talking from a former technology consultant's point of view (for whatever it's worth), any MMO dev who wants to rebuild this functionality manually from scratch is a fool.
The only problem is still coherent data amongst all nodes. If you can keep a scalable solution with a multiple number of nodes (servers) there is none.
Of course if you have one data source for ex. a database each server will have to connect to it and you will have a scalability problem, but there can be database nodes too and there are other solutions as well.
You seem to be focusing on network load balancing from a web server farm's point of view. That's definitely going to run into the kind of issues you're describing, but approaching it from that angle would be foolish for any MMO.
I dont care about loading screens. Lets say there is no loading screens.
If you're still talking about WoW, the only loading screens you have are transcontinental and instances. For all practical purposes, the world is seamless. I don't get what you're talking about.
The only problem is how do you keep the same data on each server and propagate data as it changes.
Unless a MMO dev is trying to reinvent the wheel and waste a huge amount of money creating its own back-end, the problem has been moot for a good decade on intel-based servers, and about three in the rest of computing. When you are talking about a server or shard or whatever on a modern MMO, chances are that the reference is actually to a virtual server instance, not a physical piece of hardware.
The worst-case scenario in such a setup is when something happens on the border between two servers. If players can be affected by someone who's on an another zone, the two zone servers also need to synchronize their statuses.
If you were meaning that a player should be able to cast a spell in Orgrimmar which should affect players in Stormwind in real time, you would have a point, but in purely pragmatical terms, the only areas where this kind of interaction could be considered is between two players on opposite sides of an instance door. You don't really need to have seamless transition / interaction between two continents...
But that wasn't what you were meaning. You were laboring under the assumption that load-balancing would place two players physically at the same in-game coordinates while being handled by two different physical servers. That doesn't happen. Several physical application servers are being tied together into one single virtual server instance, and that's eg Kalimdor - Bronze Dragonflight. All players in Kalimdor are in the same container. Updates get passed from that Virtual server into the database server instance - another bunch of physical servers tied together into a single virtual instance, and this virtual DB server itself handles reading and writing of data to a most probably also virtual storage pool.
There's plenty of tools on the market which handle the nitty-gritty aspect of it, and talking from a former technology consultant's point of view (for whatever it's worth), any MMO dev who wants to rebuild this functionality manually from scratch is a fool.
The only problem is still coherent data amongst all nodes. If you can keep a scalable solution with a multiple number of nodes (servers) there is none.
Of course if you have one data source for ex. a database each server will have to connect to it and you will have a scalability problem, but there can be database nodes too and there are other solutions as well.
You seem to be focusing on network load balancing from a web server farm's point of view. That's definitely going to run into the kind of issues you're describing, but approaching it from that angle would be foolish for any MMO.
Just on the point that with WotLK merged WoW servers would be a bad idea, has anyone logged into City of Heroes when a new issue hits? Or a freeplay or double xp weekend?
Usually all servers are green, with Virtue and Freedom being yellow as they've more folks.
Last DXP Freedom was locked several times, Virtue a good few times and most other servers in the yellow if not red.
Sometimes to bork a server all you need is an event as opposed to a popular location.
Usually all servers are green, with Virtue and Freedom being yellow as they've more folks.
Last DXP Freedom was locked several times, Virtue a good few times and most other servers in the yellow if not red.
Sometimes to bork a server all you need is an event as opposed to a popular location.
cross server lower level instances would make leveling a lot more fun in WoW. Deadmines is a great instance and is lots of fun to run at level. When was the last time any of you ran it like that? Cross server instances would allow for a lot more of that. I think WoW would be well served to add a couple of the better old-school dungeons but it is not needed for all instances.
# posted by : 25/6/08 16:07
: 25/6/08 16:07
NO INSTANCING OF THE WORLD.
I hate that. It makes you feel detached as you talk to people and realize you are in the same area but he's in instance 4004 and you are in instance 3.
Any game like that would have to be perfect by every other standard to keep me.
I hate that. It makes you feel detached as you talk to people and realize you are in the same area but he's in instance 4004 and you are in instance 3.
Any game like that would have to be perfect by every other standard to keep me.
I talked about this topic a couple of years ago with Raph Koster and some Turbine programmers. They all agreed that the hardware can handle 1 million people on one server cluster easily, but the software can't.
If you solved your upper program layer suddenly problems occur on another deeper layer etc. Its harder than you think. And comparing traditional clustering for databases or web based machines wit MMOG's is your first step into doom.
There are systems out there which actually does everythign MMOG's need, but they have larger budgets for their servers: Banks and Share trading systems. They can afford a shitload of hardware though while MMOG's today run on compromise machines most of the time unless you are called Blizzard
If you solved your upper program layer suddenly problems occur on another deeper layer etc. Its harder than you think. And comparing traditional clustering for databases or web based machines wit MMOG's is your first step into doom.
There are systems out there which actually does everythign MMOG's need, but they have larger budgets for their servers: Banks and Share trading systems. They can afford a shitload of hardware though while MMOG's today run on compromise machines most of the time unless you are called Blizzard
But that wasn't what you were meaning. You were laboring under the assumption that load-balancing would place two players physically at the same in-game coordinates while being handled by two different physical servers.
Actually, if you reread my comment you'll see that I was talking about servers on a logical level.
Several physical application servers are being tied together into one single virtual server instance, and that's eg Kalimdor - Bronze Dragonflight.
I know. And I wasn't describing the current server architecture in WoW. That architecture seems to break down when you have around 300 players fighting in the same area and causing the traffic to spike exponentially. One big battle in the Shimmering Flats can cause everyone on Kalimdor to time out. I'm interested in a server architecture that would scale beyond that.
Actually, if you reread my comment you'll see that I was talking about servers on a logical level.
Several physical application servers are being tied together into one single virtual server instance, and that's eg Kalimdor - Bronze Dragonflight.
I know. And I wasn't describing the current server architecture in WoW. That architecture seems to break down when you have around 300 players fighting in the same area and causing the traffic to spike exponentially. One big battle in the Shimmering Flats can cause everyone on Kalimdor to time out. I'm interested in a server architecture that would scale beyond that.
In regards to single versus multiple
Think of it this way: You can have a very large one running or lots of little ones. Both ways have challenges, but the “lots of little ones” has been the traditional approach because up until a few years ago the other approach was very cost prohibitive.
The “lots of little ones” problem is that if you want them to solve a common problem, then they all need to talk to each other. From a server perspective, it’s easy to assign different servers for different tasks and then just have them talk to each other. You can take this approach and do things like separate servers based for authentication, instancing, zones, mail, character data, chat messages and so forth in order to distribute the load. This distributed approach relies very much on server balancing to avoid a potential bottleneck in a particular area.
The issue with a single “very large” one is that it needs to be able to scale as big as needed. If you try to do everything on a single computer, then eventually you would reach a point at which it got so big that it lagged and eventually crashed. Back in the old UNIX and Mainframe days, you scaled by adding processors, memory and ports for both networking and storage. However, the cost was VERY prohibitive.
Within the last six or seven years, new technology that allows the “clustering” of lower cost Intel and AMD x86 processors has allowed software to run across multiple computers at the same time. The key difference is that it’s “as-if” one software program is running on all the computers at the same time. In effect, the combined might of all the computers is acting like one computer instead of a bunch of computers running their own thing and just talking to each other. From a scalability standpoint, you just need to add computers to the cluster in order to make it “bigger” and support more people.
This is a very simplistic comparison, but a good explanation of what Eve is doing differently than what WoW is doing. The actual reality is that most companies (and likely WoW) use a mix of both approaches. From a technical standpoint, either is very feasible and the exact mix largely a function of your overall game design.
The simple reality of WoW is that there is not enough virtual space in the game world to support more than a certain number of players. Regardless of any technical limitations, you simply couldn’t find 10,000,000 people in Azeroth at the same time.
In regards to lots of lag when you are in over-populated areas:
This issue is mostly a client-side problem caused by too many player models, animations and graphical effects happening in the same area. The software loaded on to your computer is simply having a hell of a hard time digesting it all. Every visible model, animation, sound effect or other visual (like spells) takes up a chunk of your CPU and memory. This is a graphical display problem and a limitation caused by the client, not the server.
Perhaps the bigger issue is that the server simply has a lot more information to communicate to you, so bandwidth becomes the other bottleneck. The server has to update you on each object that exists in your surroundings. When you are off in the middle of nowhere, that’s not many objects. When there are 200 people all doing different things that you need to know about – that’s a LOT of information.
Think of it this way: You can have a very large one running or lots of little ones. Both ways have challenges, but the “lots of little ones” has been the traditional approach because up until a few years ago the other approach was very cost prohibitive.
The “lots of little ones” problem is that if you want them to solve a common problem, then they all need to talk to each other. From a server perspective, it’s easy to assign different servers for different tasks and then just have them talk to each other. You can take this approach and do things like separate servers based for authentication, instancing, zones, mail, character data, chat messages and so forth in order to distribute the load. This distributed approach relies very much on server balancing to avoid a potential bottleneck in a particular area.
The issue with a single “very large” one is that it needs to be able to scale as big as needed. If you try to do everything on a single computer, then eventually you would reach a point at which it got so big that it lagged and eventually crashed. Back in the old UNIX and Mainframe days, you scaled by adding processors, memory and ports for both networking and storage. However, the cost was VERY prohibitive.
Within the last six or seven years, new technology that allows the “clustering” of lower cost Intel and AMD x86 processors has allowed software to run across multiple computers at the same time. The key difference is that it’s “as-if” one software program is running on all the computers at the same time. In effect, the combined might of all the computers is acting like one computer instead of a bunch of computers running their own thing and just talking to each other. From a scalability standpoint, you just need to add computers to the cluster in order to make it “bigger” and support more people.
This is a very simplistic comparison, but a good explanation of what Eve is doing differently than what WoW is doing. The actual reality is that most companies (and likely WoW) use a mix of both approaches. From a technical standpoint, either is very feasible and the exact mix largely a function of your overall game design.
The simple reality of WoW is that there is not enough virtual space in the game world to support more than a certain number of players. Regardless of any technical limitations, you simply couldn’t find 10,000,000 people in Azeroth at the same time.
In regards to lots of lag when you are in over-populated areas:
This issue is mostly a client-side problem caused by too many player models, animations and graphical effects happening in the same area. The software loaded on to your computer is simply having a hell of a hard time digesting it all. Every visible model, animation, sound effect or other visual (like spells) takes up a chunk of your CPU and memory. This is a graphical display problem and a limitation caused by the client, not the server.
Perhaps the bigger issue is that the server simply has a lot more information to communicate to you, so bandwidth becomes the other bottleneck. The server has to update you on each object that exists in your surroundings. When you are off in the middle of nowhere, that’s not many objects. When there are 200 people all doing different things that you need to know about – that’s a LOT of information.
There's many factors to consider with an MMO and increasing simultaneous users logged in on same server.
Before discussing, let's say that is common practice to consider 1 realm (1 server) as an unit. But in reality one server is made by many single servers arranged as a cluster.
So one server is made by many servers for computation speed, reliability and avaibility. For example in the cluster you could have one server that is idle and ready to take over of another in case of necessity.
Now, more servers in the realm, more issues can arise due to comunications, mesages and syncronization between machines.
This can be partly solved by having single machines dealing with specific zones instead of equally dividing players.
If a player A and a player B are geographically near, they will reside on same machine (server side) so that computation and messages of what each player do is simplified...
In an MMO regional division is simple and effective as only interaction between players distant each other is made by chat. By chat is less time sensitive than movement and combat.. and eventually chat could be taken by a single server alone as isn't computationally intensive.
Movement, combat, and ideally anything around a player is computationally intensive as everything need to be checked for distance, LoS and whatever with anything in the area. A chat is simple a list of clients to send a message.
Said that.....
Problems to make a bigger realm are:
1) Bandwidth in case of many players gathering on same place
2) Computation as every player action need to be checked against more players, more messages, more decisions, collisions (if game support them), LoS and whatever
3) computation client side
4) graphic power to display many more poligons than usual
5) gameplay
Now, many aspect are just tech aspects. With more work by engineers/developers all of those can be solved. If not even already by having more money to put into development and hardware machines.
But the last one.... GAMEPLAY....
well... you need a game that support more players without having all those players getting in the way each other.
And making a world simply bigger, probably won't be a solution as development cost rise a lot and many will stick anyway on the same places.... places that in a few days will prove to be the best for exp/farm or whatever players seek
So simplest solution had been sharding.... creating replicas of the same world, to divide players.
Now, as of I know....
Only 2 MMO's aren't sharded.
Guild Wars, EVE Online
GW has only one server per region (europe, north america, etc).
But even if you login virtually always on the same only one server avaible... in reality you are routed into an istance and a main server just try to keep istances with about a costant number of players. When players get fewer, istances are closed....
So at any time you get into a random istance from the ones that are open. And feel to be with many others.
EVE Online is more of a single server as is a cluster but has no istances. When you jump or un/dock you switch server inside the cluster. So as you say... if many are very near (example Jita), they are on the very same machine and start to lag.
Before discussing, let's say that is common practice to consider 1 realm (1 server) as an unit. But in reality one server is made by many single servers arranged as a cluster.
So one server is made by many servers for computation speed, reliability and avaibility. For example in the cluster you could have one server that is idle and ready to take over of another in case of necessity.
Now, more servers in the realm, more issues can arise due to comunications, mesages and syncronization between machines.
This can be partly solved by having single machines dealing with specific zones instead of equally dividing players.
If a player A and a player B are geographically near, they will reside on same machine (server side) so that computation and messages of what each player do is simplified...
In an MMO regional division is simple and effective as only interaction between players distant each other is made by chat. By chat is less time sensitive than movement and combat.. and eventually chat could be taken by a single server alone as isn't computationally intensive.
Movement, combat, and ideally anything around a player is computationally intensive as everything need to be checked for distance, LoS and whatever with anything in the area. A chat is simple a list of clients to send a message.
Said that.....
Problems to make a bigger realm are:
1) Bandwidth in case of many players gathering on same place
2) Computation as every player action need to be checked against more players, more messages, more decisions, collisions (if game support them), LoS and whatever
3) computation client side
4) graphic power to display many more poligons than usual
5) gameplay
Now, many aspect are just tech aspects. With more work by engineers/developers all of those can be solved. If not even already by having more money to put into development and hardware machines.
But the last one.... GAMEPLAY....
well... you need a game that support more players without having all those players getting in the way each other.
And making a world simply bigger, probably won't be a solution as development cost rise a lot and many will stick anyway on the same places.... places that in a few days will prove to be the best for exp/farm or whatever players seek
So simplest solution had been sharding.... creating replicas of the same world, to divide players.
Now, as of I know....
Only 2 MMO's aren't sharded.
Guild Wars, EVE Online
GW has only one server per region (europe, north america, etc).
But even if you login virtually always on the same only one server avaible... in reality you are routed into an istance and a main server just try to keep istances with about a costant number of players. When players get fewer, istances are closed....
So at any time you get into a random istance from the ones that are open. And feel to be with many others.
EVE Online is more of a single server as is a cluster but has no istances. When you jump or un/dock you switch server inside the cluster. So as you say... if many are very near (example Jita), they are on the very same machine and start to lag.
if many are very near (example Jita), they are on the very same machine and start to lag.
I think I did a bad job of pointing this out in my last comment. We are at a point now that server lag like your describing due to people being “on the same machine” really shouldn’t exist if they are using clusters to perform parallel computing. The simple reason being that it would never be exclusively “on the same machine.” This is one reason that I believe it’s a misnomer that having more server-side anti-hack detections would cause all this CPU lag on servers.
The biggest obstacle from a server standpoint is related to transmitting you information. For the sake of security, all data on every object in your virtual universe is stored on the server. Objects are anything except terrain, so they include things like nodes and widgets in addition to players and NPCs. The more objects that exist in your little area of the virtual world, the more data has to be transmitted to you about those objects.
Needless to say, if you are all alone in an area, then getting updates from the server on world objects and NPCs is not going to consume more than the expected amount of bandwidth. However, add in 400 more players to the same area and you now have a lot more objects that are all moving and casting and doing stuff. You absolutely NEED to see all that stuff happening because that’s what makes the world immersive and multiplayer. The server can process all this information quickly enough – it’s just that there is so much happening there is not enough time to tell you all that is going on in the world.
Imagine that I asked you submit an online form that had space for 250 words. Now most of the time, no one needs more than 100 words, but this time – you need to submit 500 words. Even after a bunch of fancy editing and condensing, you only manage to get it down to 350 words. Your choices are either submit the extra 100 words on a second form or transmit something that fits in 250 words but is not entirely complete. The issue is not whether or not you can write, it’s that you have more information to send then you can fit in the 250 word form.
This is one reason why “instancing” a zone likeEQ2 does is a popular way to have one world and lots of people in it. Namely, because the other people don’t exist to you if they are in a different instance. Those people in other “instances” are game objects that your client doesn’t have to draw and that the server doesn’t have to send you. WoW chose NOT to instance the world that way, but it means they have to artificially cap the population and some high-traffic areas are painfully frustrating.
I would surmise that around 90% of the lag that people experience in MMOs is related to the object issue I am describing. Either there are too many objects to draw (client limitation) or too much object information you need to receive (bandwidth limitation). It’s also worth pointing out that the bottleneck might not be YOUR bandwidth, but the bandwidth allocated by the game company to the server for each user. It would actually be quite logical for them to cap it on a per user basis.
FYI – In WoW, objects seem to get loaded into memory at or around 600 yards or roughly twice your minimap distance. The actual distance will throttle depending on the number of objects in memory and your own bandwidth. If you ever experience a bugged mob (can’t loot or whatever), you can often fix it by moving away 600+ yards and then back. You can test the range by targeting a mob and moving away. It will disappear when it exits your client memory.
Problems to make a bigger realm are:
1) Bandwidth in case of many players gathering on same place
2) Computation as every player action need to be checked against more players, more messages, more decisions, collisions (if game support them), LoS and whatever
3) computation client side
4) graphic power to display many more poligons than usual
5) gameplay
BTW, I agree that those are the major issues. BUT – the big two technical reasons are #1 and #4.
I think I did a bad job of pointing this out in my last comment. We are at a point now that server lag like your describing due to people being “on the same machine” really shouldn’t exist if they are using clusters to perform parallel computing. The simple reason being that it would never be exclusively “on the same machine.” This is one reason that I believe it’s a misnomer that having more server-side anti-hack detections would cause all this CPU lag on servers.
The biggest obstacle from a server standpoint is related to transmitting you information. For the sake of security, all data on every object in your virtual universe is stored on the server. Objects are anything except terrain, so they include things like nodes and widgets in addition to players and NPCs. The more objects that exist in your little area of the virtual world, the more data has to be transmitted to you about those objects.
Needless to say, if you are all alone in an area, then getting updates from the server on world objects and NPCs is not going to consume more than the expected amount of bandwidth. However, add in 400 more players to the same area and you now have a lot more objects that are all moving and casting and doing stuff. You absolutely NEED to see all that stuff happening because that’s what makes the world immersive and multiplayer. The server can process all this information quickly enough – it’s just that there is so much happening there is not enough time to tell you all that is going on in the world.
Imagine that I asked you submit an online form that had space for 250 words. Now most of the time, no one needs more than 100 words, but this time – you need to submit 500 words. Even after a bunch of fancy editing and condensing, you only manage to get it down to 350 words. Your choices are either submit the extra 100 words on a second form or transmit something that fits in 250 words but is not entirely complete. The issue is not whether or not you can write, it’s that you have more information to send then you can fit in the 250 word form.
This is one reason why “instancing” a zone likeEQ2 does is a popular way to have one world and lots of people in it. Namely, because the other people don’t exist to you if they are in a different instance. Those people in other “instances” are game objects that your client doesn’t have to draw and that the server doesn’t have to send you. WoW chose NOT to instance the world that way, but it means they have to artificially cap the population and some high-traffic areas are painfully frustrating.
I would surmise that around 90% of the lag that people experience in MMOs is related to the object issue I am describing. Either there are too many objects to draw (client limitation) or too much object information you need to receive (bandwidth limitation). It’s also worth pointing out that the bottleneck might not be YOUR bandwidth, but the bandwidth allocated by the game company to the server for each user. It would actually be quite logical for them to cap it on a per user basis.
FYI – In WoW, objects seem to get loaded into memory at or around 600 yards or roughly twice your minimap distance. The actual distance will throttle depending on the number of objects in memory and your own bandwidth. If you ever experience a bugged mob (can’t loot or whatever), you can often fix it by moving away 600+ yards and then back. You can test the range by targeting a mob and moving away. It will disappear when it exits your client memory.
Problems to make a bigger realm are:
1) Bandwidth in case of many players gathering on same place
2) Computation as every player action need to be checked against more players, more messages, more decisions, collisions (if game support them), LoS and whatever
3) computation client side
4) graphic power to display many more poligons than usual
5) gameplay
BTW, I agree that those are the major issues. BUT – the big two technical reasons are #1 and #4.
I would love cross-server dungeons on WoW, but before they do it they need a much better LFG tool. I don't know what the average number of characters per account is, but I'm sure by now it has to be more than one. The LFG tool should let you advertise a LFG for any of your toons for any instance, and you should be able to browse all LFMs, regardless of which toon you're logged in as. Currently, if I'm logged in as a level 70, I can't see the LFM list for Dead Mines, even though I might have a level 18 alt itching to get into one. And I can't be stuffed logging into all my alts every 10 minutes to check.
While we're at it, let us see all the LFMs in one big list that we can apply decent filters too! I hate having to select each instance from a dropdown just to find the list empty. No one uses this limited tool, which is why its always empty! Open it up a bit more and let us filter the complete list with some sensible options like... Groups looking for a particular class, groups looking for a particular role (and allow LFM to advertise for a role), instance level range, etc. I love to group but I've found WoW forces me to solo play 99% of the time because I just can't find anything with the LFG tool.
If we had a decent LFG tool, we might not even need cross-server dungeons since it would let the current servers population link up with each other a lot more through our alts, not just the toon we're currently logged in as... People complain about the disjointed community in WoW these days, and how the game seems to be less an MMO and more of a solo game we just rent... All these problems I believe stem from our inability to find each other to do the group content. It wasn't a problem back when the main characters were at the right level -- everyone was doing them and it was hard not to find a group -- now we need a decent tool to facilitate it for alts and it is long, long overdue. Blizzard, play DDO for 15 minutes and then go design a new LFG tool.
Some thoughts on cross-server dungeons while I'm here...
Players could effectively join a raiding guild on another server without transferring to that server. 95% of guild activity is held inside instances. I wonder if they would allow cross-server guild chat?
Players could transfer goods from one server to another. I believe they addressed this in BGs already, but I'm wondering how they would handle it for dungeons since some trading is necessary (loot distribution, rare patterns, consumables, etc).
While we're at it, let us see all the LFMs in one big list that we can apply decent filters too! I hate having to select each instance from a dropdown just to find the list empty. No one uses this limited tool, which is why its always empty! Open it up a bit more and let us filter the complete list with some sensible options like... Groups looking for a particular class, groups looking for a particular role (and allow LFM to advertise for a role), instance level range, etc. I love to group but I've found WoW forces me to solo play 99% of the time because I just can't find anything with the LFG tool.
If we had a decent LFG tool, we might not even need cross-server dungeons since it would let the current servers population link up with each other a lot more through our alts, not just the toon we're currently logged in as... People complain about the disjointed community in WoW these days, and how the game seems to be less an MMO and more of a solo game we just rent... All these problems I believe stem from our inability to find each other to do the group content. It wasn't a problem back when the main characters were at the right level -- everyone was doing them and it was hard not to find a group -- now we need a decent tool to facilitate it for alts and it is long, long overdue. Blizzard, play DDO for 15 minutes and then go design a new LFG tool.
Some thoughts on cross-server dungeons while I'm here...
Players could effectively join a raiding guild on another server without transferring to that server. 95% of guild activity is held inside instances. I wonder if they would allow cross-server guild chat?
Players could transfer goods from one server to another. I believe they addressed this in BGs already, but I'm wondering how they would handle it for dungeons since some trading is necessary (loot distribution, rare patterns, consumables, etc).
# posted by : 26/6/08 03:16
: 26/6/08 03:16
If we had a decent LFG tool, we might not even need cross-server dungeons since it would let the current servers population link up with each other a lot more through our alts, not just the toon we're currently logged in as... People complain about the disjointed community in WoW these days, and how the game seems to be less an MMO and more of a solo game we just rent... All these problems I believe stem from our inability to find each other to do the group content.
good luck with that your spitting in the wind expecting change there. This has been one of my pet peeves since the current LFG tool went live. You aren't saying anything that hasn't been posted on the wow forums by me and untold others. And I have actually had blue posts tell me that the tool was fine it just didn't work because people were stubborn and wouldnt' use it.
Post a Comment
good luck with that your spitting in the wind expecting change there. This has been one of my pet peeves since the current LFG tool went live. You aren't saying anything that hasn't been posted on the wow forums by me and untold others. And I have actually had blue posts tell me that the tool was fine it just didn't work because people were stubborn and wouldnt' use it.
<< Home
