Tobold's Blog
Wednesday, April 25, 2012
 
Gaming the matchmaking

A reader wrote me a mail with the above title, about how he games the matchmaking in Starcraft 2:
Hi Tobold,
I play a lot of Starcraft 2 ladder games and what I found most disheartening in the matchmaking system is when I realized that every single win takes me closer to be matched to harder and harder opponents. Basically the game systems edges you closer and closer to your physical limits - whatever those might be. You end up having to put in your best all the time. But that is not really why I play a game, I want to also relax.
Moreover there are many people that simply quit at the beginning (probably they want to drop in the ranking), plus there are cheesers and other people that find ways to exploit some features (quick and powerful all ins) so in the end the number of games that I win because I made a lot of good decisions is substantially less than 50%. That makes the game a whole lot less fun.
So this is when I realized the solution - if I am about to win a game (if I see that I am overrunning my opponent and they are left with almost nothing) I surrender. I found this to have many benefits and no downsides - for me whether or not the final page shows me winning or losing makes no difference whatsoever. I know that I have won and I got my satisfaction.
The other benefits are that I am not increasing in rank, therefore my next opponent won't necessarily be harder, moreover my opponents are also happy, they are still in the mindset that the little number of wins matters and in fact many of them thanked me for surrendering and overall are a lot nicer to me - I often get a bad mannered messages after I win but none when I surrender. So it is ending up a win-win!(BTW I am not doing this all the time, just from time to time)
I will say though that in the world of gaming there is no game as intense as SC2. It is a brutal game that with this trick is a lot more fun than before. Every game is a relaxing one that I don't have to push myself to limit while playing.
What he effectively does is winning more often by changing his personal definition of what a win is. The matchmaking system records a loss for the game he considers to have won. And that recorded loss makes it more likely that he "wins" the next game. The obvious problem with that method is that it leads to other people being paired by the matchmaker against an opponent who is far too strong for them, whom they only face because he manipulated his score downwards.

And that is basically the problem of all matchmaking systems. Everybody wants to win more than half of his games, but that simply isn't possible. Either you switch to a PvE system in which the computer doesn't mind losing, or you accept that the optimum outcome of a perfect matchmaking system is a 50:50 win chance. The only game I know that comes close to this perfect equilibrium is World of Tanks, and lots of players complain that their win chance isn't higher than 50% in that game.

There might be a future employment market in this. Instead of paying some guy in a Chinese sweat shop to power-level our characters or farm gold, we could pay him to lose PvP games against us.

Comments:
If you haven't read it, Gevlon essentially demonstrated that the matchmaking in World of Tanks "cheats" - instead of matching you against equally strong players, it simply makes all your shots less effective (and fills your team with lower-ranked players).

In any case, there are at least two seperate ways out of your perfect matchmaking "impossibility." The first is to simply accept that some will win 60/40 or higher, and essentially allow player churn - which would naturally occur regardless of how good the matchmaking is - to replace those discouraged by their <50% win rates. The second is to stop focusing so much on the literal win itself. Many FPS games do that these days with "RPG-elements" such as character/weapon unlocks that progress towards, win or lose. Normal WoW BGs did this too via Honor for losing, and even in Arena games in Wrath asked you to only play 10 games to get a minimum amount of Arena Points to purchase better gear.

In other words, the way out of the impossibility is to reduce the binary reward/penalty for winning or losing. These games are supposed to be fun regardless of what happens anyway, right?
 
I have been playing a lot of Wargame: European Escalation recently and I like the way that game handles winning.

In WEE you get grades of win, for example if your final score was reasonably close you get a draw. Sometimes you get a minor victory. Sometimes you do well and get a major victory.

It is a good system that seems a bit more realistic. You should not get more than a draw if your entire army was wiped out along with your enemy, and you just happened to kill one extra jeep.

As to the ranked games, I find it best to ignore the option entirely and only play lobby games.
I think this is true of all RTS. Ladder games are stressful and you end up playing against people that are out to win no matter what. For example a cheese rush of whatever is deemed the op unit of the time.
All you get for playing well in ladder games is an internet score. Who really cares about that?

I wonder why this guy is playing ranked games in Starcraft? I have never actually played it, is there not a solid lobby?
 
My first thought was that if he wanted casual games then why isn't he playing lobby games. Then I realised, with lobby you can still get opponents that are hard to beat. What he wants is a difficulty selector for PvP.
Are there any online games that let you select harder or easier players? It would be a great feature. Select easy mode and you get drawn against a weaker player that had chosen hard mode. Player on easy mode gets >50% win rating. Player on hard mode gets huge rewards for winning and some nice vanity gear to balance the pain of them always lossing. Alternatives would be handicap matches of 2v1 or boosts to one side only.
 
@Azuriel - I think your first alternative greatly underestimates how detrimental a good ranking system would be for the profitability of a game. As better and better players move into the bottom 10% as others leave, my opinion is the churn would be quite high for a mass market. Tiny indie gamers love that their game is not popular.

A mass-market game with all the customers being Bartle Killer archetypes seems quite complicated without a lot of new players to farm. And getting a million new players is both expensive to do and even more expensive to maintain, especially if you see your business model as ongoing (MMO) versus one-time console game.

Mass-market, the solution seems to me to leverage that, like the OP, different people have different definitions of success. What if in WoW you went into a BG with no PvP gear you got triple XP per hour or something useful like raiding gear? In EVE, what if you could insure several inexpensive industrials a week i that paid above 100% of ship plus cargo if destroyed in lo-sec?

A well done system would have people with different values each thinking they are winning as well as perhaps thinking the others are chumps.

I just decided this is another reason why long term I think the future is F2P/MT. I see most of the subscription games being hardcore gaming developers focused on making a "good and fair game" but sometimes losing sight of the fact that they are making a game that is good for them but that is not what everyone or even a majority of their target audience desires. I see F2P companies being more focused on understanding what customers want.
 
If you haven't read it, Gevlon essentially demonstrated that the matchmaking in World of Tanks "cheats" - instead of matching you against equally strong players, it simply makes all your shots less effective (and fills your team with lower-ranked players).

Sorry I don't get it. How is that cheating? It's just another type of matchmaking. And for every "moron" or "slacker" on your team there should statistically be the same amount on the enemy team so it shouldn't be a problem. Of course it varies from round to round but it should even out.

However, if you've got good players in stronger tanks it can really make a lot of difference depending on the situation. But that goes for the other team as well.
 
In PvP, if I'm at least winning "some" of the time I'm quite happy. I certainly don't need to get anywhere near a 50-50 win/lose ratio.

I generally don't get put off playing unless I find myself losing more than three matches out of four, so long as the three matches I lose aren't all complete steamrollers. I have plenty of other things to give me pleasure apart from a final win condition - how well I performed vis a vis my team/the other side/my own average standard, whether I'm improving, whether I had good individual fights and above all did I have fun.

Not sure if this would still apply in a game designed entirely around Ladders and Matchmaking but it works perfectly well in MMO PvP situations.
 
People would be happy with the 50/50 if the ranking would be visible and provide some "rewards", even if cosmetic. The problem is that there is no telling if you are in the second top 10% player or a bottom 10%. The very best are identified, but the good are blurred into the terrible.

The WoT cheats in a way that if you are good, you are artificially made worse and mixed with idiots who you must carry. So you won't be defeated by superior enemies, you'll be facerolled by average ones simply because you are with a knife in a gunfight. You learn nothing.
 
There might be a future employment market in this. Instead of paying some guy in a Chinese sweat shop to power-level our characters or farm gold, we could pay him to lose PvP games against us.

I suspect you were being ironic when you wrote this but I think it is a brilliant idea. What better way to keep your users happy then to ensure they win more often. Of course every match would not be against one of these paid opponents and the paid opponents would have to play in such a way as to loose in a non-obvious fashion. Perhaps they could even offer tips to players to improve their game when the player really needs some help. Seriously, I think this could work provided the expense could be kept manageable.
 
So this SC2 player manipulated the system to make the game less "intense" as he says? The truth is, the only time Starcraft 2 is not an "intense" game is if you are playing someone who you can destroy without trying very hard.

However, every time he enters such a game there is a human being on the other side that is grossly outmatched. And therefore has a very intense matchup that he will likely not win.

I would also argue that the more one performs this tactic, the worse they get at the game since you only really get better when playing better or equal opponents.

My advice to this player is to maybe play versus AI, or play a game that is less intense for his tastes.
 
Riot Games just hired a COGNITIVE NEUROSCIENTIST to lead their matchmaking overhaul project. I think that goes to show you how complicated this issue is, haha.
 
This comment has been removed by the author.
 
I agree that this is the inherent problem with matchmaking. If you are an above average player, you likely expect to win more than 50% of the time. And if you don't, you are probably not very happy.

But matchmaking systems ensure that you only win about 50% of the time.

This is one of the reasons (thought not the biggest reason) that I get burned out of playing LoL so fast. I don't feel like I am getting better because I still win and lose at the same rate.

-Michael Hartman
http://www.frogdice.com
@frogdiceinc
 
If everything is random (no pre-mades) and equal (gear/class/role/race/level equality) then yes, matchmaking should have a 50% win rate. If competitive players still get frustrated at this statistical norm, then the game may want to consider not counting victories at all but other things like kills, captures, etc (CoD for example).

Some matchmaking (like wow arena) is clearly more competitive than recreational (skirmishes). A good game would ideally have options for both types of players.

On a side, my loss rate in BGs used to annoy me so I would "cheat" by leaving a losing match early, so the loss did not appear on my record. This is an example of player behaviour outside of what devs intended, further exacerbating inequality.
 
I am not convinced that WoT's system has been conclusively proven to cheat. The matchmaker is pretty well documented ( http://wiki.worldoftanks.eu/Battle_Mechanics#Matchmaking ) and its been player tested. Gev played ~150 games with a partner and did well, then played 10, did "Average" and declared the system cheating. There are too many variables not considered to draw conclusions (including but not limited to: Time of day played, lack of a partner, the system getting a 'better' idea of his skill, bad luck..."the game is rigged" is more or less tin-foil hattery at this point...and before anyone throws out "Confidence interval" keep in mind the sample sizes (~150 v 10).

Anyways, that bit out of the way, the current model of "puts you against equal skill opponents" can lead to the exact feeling your player encountered Tobold. Where after a certain point a person feels required to play at their best to maintain some "ranking". I've done what your player did in other games...my Arena team once intentionally let our ranking tank just so we could play easier matches, and I've done the same at other games. On the other hand, it's a two sided problem...it is gaming the matchmaker, and it means there are lower skilled players that will end up losing when they didn't necessarily "deserve" to because a better player had shot down their own ranking.
 
The WoT cheats in a way that if you are good, you are artificially made worse and mixed with idiots who you must carry. So you won't be defeated by superior enemies, you'll be facerolled by average ones simply because you are with a knife in a gunfight. You learn nothing.

You are not personally made worse, but since you are placed with random people of course not all of them will be good. It's matchmaking, not cheating. And as with any random element of course you will think that it teams you with bad players almost all the time because it's easy to believe that. But over a long period of time you will be teamed with as many bad players as you are teamed against them.
 
+1 @Clockwork - I would certainly agree that "proven" way overstates the suspicions.

Look at the way, iirc, golf and bowling work. If a non-professional golfer who normally shoots a 72 were to shoot a 73 and their opponent who normally shoots a 96 were to shoot a 95, then the 95 would win the handicapped match. Some of the whiners would argue this is not fair since the 73 is a much better player. Although in that match, the less-skilled player played better than usual and their opponent did not.

Once again, when players and developers talk about fairness, I have to question why two different players have different gear options if people are concerned about fairness. Developers want to sell RMT or longer subscriptions to grind. Many players want to outgear the newer players they will farm. If you want better gear than your opponent, I think you have lost all rights to advocate for fairness.

Is the game system about trying to handicap team composition (have the 100 & 80 play the 90 and 92), personal performance in game (a 96 golfer gets to subtract 24 from their score - gets better WoT penetration?), or outcome (a 2400 chess player gets small rewards and huge penalties for losing to a 2000 ) Or a combination of all 3. Like most discussions, people start the argument before even having a shared understanding of what they are arguing about.

So there are handicapping systems where Gevlon is missing the point; An 80 golfer playing a 96 can learn because they are competing against their past performance. They are pushed to play better than they have in the past, regardless of who the opponent is. In this handicapping system, the best player in the game is always pushed since if they play normally and their opponent plays above average they will lose. I.e., if the goal is learning, then the handicap system, what I think Gevlon says WoT does, probably pushes learning/growth the most.
 
The problem is everyone wants to feel pro, and in reality are average.

So when you get frustrated at not being rewarded for your skills with wins, just remember that you actually just aren't that good. The vast majority of people are average and will hover around 50%.

It's not the game cheating, it's you being matched appropriately.

Matchmaking however assumes players want to be challenged and win fairly, when obviously as Tobolds post shows, this is not the case.

That player went out of his way to feel pro by steamrolling noobs, it's far more pathetic than actually caring about real wins against real opponents.

In a vacuum it wouldn't matter, but this actually affects other people...namely the noobs who get steamrolled by people cheating the system so they play against weaker opponents.

It is difficult for a dev to deal with this...but in a perfect world they would detect this gaming of the system and just ban that person from matchmaking.

I would have some system that Rey
 
I would have some system that detects constant surrendering, and gives a warning then bans that person.

I as a dev would have no sympathy for people who ruin others experiences, especially noobs.

On a difficulty slider...seriously? How exactly would that work? Everyone would pick Pro then lose, then go down to Noob to stomp noobs. So no difficulty other than the highest would come close to actually reflecting the skill of it's players.

Making the system pointless...unless you force those skilled enough to move up, in which case you have matchmaking.
 
Having actually played a Marder 2 during when Gevlon wrote his post I had similar results noticing a drop in my win percentage after Gevlon's result. Previously I had up to a 60.2% win and within a week had managed to drop that to 59.1%. For me this was significant because firstly I have over 500 games played meaning a 1.1% shift is pretty big considering before his post my last 50 games I was winning around 67%.

The problem with the WoT rigging isn't that it in anyway adjusts the players to have a proper 50/50 in response to their skill. It randomly hands out to the entire team an accuracy nerf and penetration nerf. The game also has a secondary matchmaking rigging where it sets up teams of tanks that will obliterate the other team if all is fair. These two examples of the game predetermining whether or not you should win or lose also work to mask eachother. If your team has five marder 2's and the enemy team was given a bunch of mediums and lights the team with the marder 2 should win. Now maybe some skill and luck will allow the other team to occasionally win but that isn't what happens. Frequently the marder team gets a sweeping accuracy nerf and the light and meds roll over you maybe losing 2-3 tanks.

WoT should just score your ability and arrange teams in relative even distribution of ability if they are looking for fair matches. They however don't and would rather feed teams losses so that players don't realize when they are playing bad because they just seem end up with the other team getting rolled.
 
@MagrothJ "You are not personally made worse, [...]"

You likely are, given how penetration works in that game. I suggest reading the article I linked to and either acknowledging or refuting the numbers. Moreover, the notion that an extremely skilled person being grouped with noobs vs a team of average players constitutes a "good match" is asinine. Perhaps some FPS games can be won or loss on the strength of a single individual, but WoT and most other such games are team-focused.

As Gevlon is stating, the pro player's abilities aren't being taxed (expect perhaps his/her patience); he/she is simply being placed in a no-win battle for the sake of tanking (*rimshot*) their win percentage. Matchmaking should be about matching based on skill levels: teams of pros fighting other teams of pros.

It would be like the top Arena players in WoW getting Rez Sickness forced on them and then fighting 1200-rated teams. It is simply insulting to both parties.

@Hagu

Your golf and bowling analogies would be fine if that was what was going on. It's not. You are not competing against your past score here, you are having the golf ball replaced with a billiards ball and the clubs with a tennis racket.

Of course, that is really besides the point anyway. Neither golf nor bowling are team activities; while you can have "teams," the actual activity is solo (a teammate isn't assisting you hitting the golf ball or nudging the bowling ball out of the gutter).

It is not good matchmaking to drop Brett Favre in as quarterback for a high school football team against another team with 300 lb linebackers simply because Favre won too many games.
 
The only way for a casual noob like me to play SC2 is to do it directly with friends and family. Preferably ones who are close enough for the relationship to survive the result.

Get beaten too often and you feel like your playing that game is just to pump their ego. Beat them too often, and someone calls the police.

Warcraft 3, Homeworld, Shogun: Total War, Red Alert, War Wind, Heroes of Might & Magic, Starcraft... My siblings and I have a tradition of playing the multiplayer of these for a few hours, then never speaking of such horrors ever, ever again. So that we may still remain friends, not just family.

Actually, that's a brilliant idea... I'm going to propose that we round up a line-up of those truly classic old RTSes and play them against each other without having touched them for years. Hopefully our skills will be so eroded that our mutual ineptitude will allow interesting, satisfying matches that don't require you to have invisible psychic commandos firing nuclear missiles in the first five minutes.
 
Sidenote, re: games with horrible matchmaking? Try APB Reloaded sometime. It's NOTORIOUS for its horrific match-making and for the pro players manipulating their notoriety in order to get into steamroller-fests with newbies.

Also, it features an open-world lobby, which is part of the match-making... but which also allows you to attain a 'wanted' rating so high that anyone can attack you... but by contrast, you can attack anyone, allowing you to grief pretty much anyone.

The game has great potential but my God, there are only so many nights you can spend logging in to get pounded into the dust over and over by the same couple guys fo rhours.
 
You likely are, given how penetration works in that game. I suggest reading the article I linked to and either acknowledging or refuting the numbers. Moreover, the notion that an extremely skilled person being grouped with noobs vs a team of average players constitutes a "good match" is asinine. Perhaps some FPS games can be won or loss on the strength of a single individual, but WoT and most other such games are team-focused.

As Gevlon is stating, the pro player's abilities aren't being taxed (expect perhaps his/her patience); he/she is simply being placed in a no-win battle for the sake of tanking (*rimshot*) their win percentage. Matchmaking should be about matching based on skill levels: teams of pros fighting other teams of pros.

It would be like the top Arena players in WoW getting Rez Sickness forced on them and then fighting 1200-rated teams. It is simply insulting to both parties.


I see nothing in Gevlon's post that suggests that penetration/damage is nerfed because you are more skilled. Possibly the opposite as he has an extremely high win percentage. You can see the total damage done but for that to be of any use you need to see total shots fired. Possibly broken down into shots hit/missed/deflected/penetrated.

Aside from that you and Gevlon seem to think that you are being targetted for impossible battles. That's not the case. If anything Gevlon's high win percentage should prove that. It's just a kind of matchmaking where they match up according to a specific tier table. You may not agree with it being the best way to do it (which I'm not saying you are not entitled to do), but it's NOT cheating.
 
@Cam re:RTS

I totally hear you on this one. The trauma of spending half an hour (or more) building a base only the watch it be destroyed because you didn't upgrade/attack earlier/expanded too soon etc.

These days I prefer PvP games which enable the loser to 'bounce back' quickly into the game.
 
Post a Comment

<< Home
Newer›  ‹Older

  Powered by Blogger   Free Page Rank Tool