Explaining the Microsoft TrueSkill ranking system – Rainbow Six Siege

0
6

What’s up guys, Rogue-9 here, I recently joined a podcast style discussion on Get Flanked’s channel where we talked about the ranked game mode in Rainbow Six Siege (link at the end of this video). As part of my preparation for this talk, I did some in depth research on the Matchmaking Rating system (or MMR system) in Rainbow Six Siege and when I say in depth… I mean reaaaallly deep.

And while doing this, I realised that there are quite a few misconceptions floating around the community. So today I want to finally explain the ranking system used in Rainbow Six Siege to maybe help clarify things for those of you who are still unsure as to how exactly it works. This explanation will include a functional description of the rules that drive your rank, the limitations of the ranking system and finally a little insight into the maths behind the algorithm (but don’t worry, I will keep things simple). Now to starts off the first myth I want to dispel, is the idea the Rainbow Six’ MMR uses the Elo method. The Elo rating system was invented in the late 1950s by physics professor Arpad Elo to provide a more fair and precise measure to compare the skill level of chess players to one another.

And while the basic concept, of continuously rating a player’s skill based on their ongoing performance versus other ranked players is also used in Rainbow Six, there is a significant limitation in the Elo system that makes it unsuitable for Siege. The Elo system is only designed to work in 1 vs. 1 player games and the same is true for the more advanced Glicko system. So the ratings system used by Rainbow Six Siege is neither Elo nor Glicko but instead it is a far more advanced system called TrueSkill. This ranking system was developed by Microsoft Research and has been used for matchmaking and player ranking on Xbox Live. Just like Glicko, the algorithm assigns both a skill rank and a second variable called a confidence factor to each player. I will go over some examples of how this works later on but at this stage, let me first give you some general principles about what your ranked score in Rainbow Six really means.

Your rank in Rainbow Six Siege is displayed in game as an MMR score of 100 times the underlying rating score used by the algorithm. This basically means that when you have 2500 MMR points, your actual TrueSkill rank is 25. At the beginning of every season, each player starts out on 2500 MMR and with each match will either gain or lose points. As we all know, for the first 10 games, the MMR score is hidden from us but the fact is that it is already working in the background; just like it does once you receive your rank. How many points you gain after a win or lose after a loss depends on a few factors. First, the rank of the players you face is taken into account, as well as you own rank. If a team of five platinum ranked players goes up against a mixed team of silver and gold players, the expectation is that the plats will win.

If this happens, their MMR scores will only go up by a relatively small amount and the other team’s will only go down by a small amount because the expected outcome of the game doesn’t really tell us anything new about the skill of the players on each team. If the plats lose on the other hand, the system will take this surprising result into account and will award the lower ranked team a larger gain (since they are likely to be better than their current rank would have suggested) and of course the plat team will lose more MMR. The second factor that is taken into account is how certain the algorithm is about the skill of each player. If a player has played a lot of ranked games, the system has a pretty high certainty of his or her true ranking and the MMR gains and losses after each match will be much lower than those of players that have only played a small number of games. So those are the basics and now let me address a couple of fundamental misconceptions about Rainbow Six Siege ranked that many players seem to believe in.

On the one hand, many players assume that the re-ranking at the end of a match is a reward or punishment for their personal performance in that game and this is simply not the case. Of course, it doesn’t help that the various ranks are labelled like medals (bronze, silver, gold, platinum etc.) and even worse, at the end of a season players are literally REWARDED with different charms based on the rank they reached but the fact is that the MMR gain or loss after a match is simply the system adjusting to try to provide you with better matchmaking in future. So it’s fully understandable why everyone is so confused. In a blog post from back in the days of operation Red Crow, Ubisoft clearly say that: “The MMR is not a reward system.” It says it right there but at the same time they name the ranks after medals and then provide actual reward charms based on the highest rank you achieve in a season. This is not only completely contradictory and stupid but also leads people to try to game the system by any means possible including cheating and paid boosting.

The result of this undecided approach is lower quality matchmaking at best and couple that with the incentive to cheat, boost or mess with the system in any other way and this ultimately leads to a poorer experience for all other players. So point one, MMR is not a reward! At least it’s not supposed to be. The next common belief is that your ranking is an indicator of personal skill but again, this is not really true. As you play more games, your rank will reflect your average expected performance more and more but it is a rather weak indicator of personal skill due to some significant limitations. First up, the MMR score of each player is not an absolute measure of skill but instead, it is a relative measure of expected performance compared to other players. And the first issue with a relative measure is that it is only valid against the other players in your comparison group, i.e.

The region you play in. That is why players can have separate ranks in different regions. If you normally play Rainbow Six in America but decide to join some friends in Europe for a few games, you will have to earn your rank for that region from scratch. And what this really means at the end of the day is that a pretty stable plat 1 player in one region is not necessarily exactly as good as a plat 1 player in a different region. Their actual abilities could be significantly different. And there is another factor that makes your MMR score relatively meaningless in terms of assessing your actual ability in the game. As mentioned earlier, your MMR will increase or decrease depending on wins or losses alone.

Personal performance within each match, such as score, K/D ratio, headshot percentage, accuracy or whatever other metric you wish to choose, will have no effect whatsoever on your MMR change. The amount of time you spend in a match or the number of rounds you play is also irrelevant. This means, for example, that if a group of five friends get together from the beginning of a season and they only play ranked games in the same five man squad for the entire season and never play with anyone else, their MMR scores will always be exactly the same, even if one of them is a very new player and only gets a KD of I dunno 0.5 every match and one is very experienced and carries almost every game.

And let me highlight this again… their MMR will be 100% identical throughout the entire season. The bottom line is that in reality, any player’s rank is not necessarily a measure of their skill but more of their likelihood to win. And yes, of course a high likelihood of winning CAN potentially be an indicator for high personal skill but this is in no way guaranteed. For instance, someone who always plays ranked in a dedicated 5 stack with voicecomms will automatically have higher chances of winning than someone who always solo queues and will therefore have a higher rank, even if the skill levels of the two players are identical. And one last little nugget of info that many of you may not know and may find interesting is that the matchmaking in casual games is driven by exactly the same algorithm as ranked.

Each and every player will have a second, separate MMR score for casual matches that is calculated and updated in exactly the same way as their ranked MMR. The only difference is that this score is kept hidden. Sure, there are a couple of little difference in the way the matches are played but apart from that, there is no difference between casual and ranked. Now with those misconceptions hopefully cleared up, I want to talk a little bit about the different ranks, specifically how they are meant to be distributed and how they are actually distributed. In their Operation Red Crow post, Ubisoft shared this neat little graph that had all players aligned on a bell curve, with the average players clustered around high silver/low gold ranks. The reality of the ranking distribution is far messier though. R6db.com shows us though, that the real distribution is quite different, with most players in the gold category, more plat 3s than gold 1s, more diamonds than plat 1s and more copper 4s than the rest of copper combined.

Another interesting fact is the low number of players ranked from silver 1 and downwards compared to gold. This lopsided distribution makes the actual ranks that players achieve even less meaningful than they already are and, ok, I guess Ubisoft could attempt achieve the nice clean normal distribution they are apparently aiming for by adjusting the MMR point thresholds required for each level but this would still not lead to very precise results. The only way I see of guaranteeing the targeted bell curve is by using a forced distribution that keeps the thresholds completely fluid. This of course could lead to the situation in which you keep gaining more and more MMR but never rank up because the players above you also keep gathering more points. I can see how this is not as motivating as being able to work towards a fixed MMR target but it would definitely give players a more accurate representation of how they compare to other players.

Which system would be better? I guess that is down to personal opinion more than anything else. And now finally, as promises earlier, I will go over a couple of real world examples to explain how the MMR and confidence values come together to form the basis of Rainbow Six’s matchmaking. So firstly, every player is assigned a skill level.

Then in addition to that, every player is also assigned a confidence value, which indicates how sure the system is of your rank. At the beginning of each season, the system has no information regarding your ability and will be relatively unsure of your actual skill level. We all start out on a default confidence of +/- 833 MMR until we play more ranked matches. This means that at the beginning of each season, the algorithm assumes that there is a 95% chance that every player should be ranked somewhere between 1667 and 3333 MMR. This is of course a very crude guess and that is exactly why the system hides our MMR score until we have played at least 10 matches. The confidence variable is an important value because it plays a role in deciding how much your MMR will increase or decrease after every match but let’s get to that in a bit.

If you want to find your current confidence value, you will not be able to see it in game but you can look it up on r6db.com so let’s go there for a second just so that I can demonstrate to you how this all works on a couple of practical examples. First let’s look up my abysmal stats and to illustrate the starting rank for each player, I direct your attention to my MMR on the Asian servers. Since I have never played a single game on Asian servers, my MMR is the default 2500. On American servers, I have played a total of 4 glorious matches with two wins and two losses and my MMR is still very close to the starting level and you can also see that the system currently assigns me an uncertainty value of +/- 8, which translates into 800 MMR points. On the EU servers, I have played 15 matches so far this season and as you can see, this has brought down the uncertainty value of my rank.

This means that the system is 95% certain that my matchmaking rank should be somewhere between 2,191 and 3,629. And as mentioned before, the more you play, the more certain the system becomes of your matchmaking rank. For instance, if we look up pro-player Pengu’s stats for this season, we can see that his current MMR is and imposing 6022 and the algorithm is 95% sure that his skills is somewhere between 5,674 and 6370). All of these ranking assessments can also be shown as normal distribution graphs. This would be the one for completely unranked players, mine is already far more accurate and a little further along to the right and of course Pengu’s is all the way over here and the narrowness of his curve shows that the system is already far more certain about his ability.

So there we go, I have gone over the history of ranking systems in general, provided some insight into the TrueSkill algorithm used by Rainbow Six Siege and highlighting what it can and more importantly what it cannot tell about a player’s individual skill. And now let me give you a couple of closing thoughts before wrapping the video up. In some parts of this video, I may have come across as if I am bashing the ranked system for its problems and deficiencies. And while I do believe that there are some serious issues with ranked, I also empathise with the devs.

As I explained, one of the biggest problems with the algorithm is that it does not evaluate personal performance at all, it only looks at wins and losses. And while this makes your rank a poor indicator of actual skill, this is also an issue that is very difficult to solve. Rainbow Six Siege is inherently a team based game where different team members will take on different roles. You will have players that specialise in fragging and will choose the appropriate characters for that role, both on attack and defence. The problem is that a team that is made up entirely of roamers or where nobody will pick support characters such as Thatcher or Thermite will be unbalanced and likely to lose their matches. Individual players can contribute to a team’s win by making call out’s or coordinating the team and those kind of actions are impossible to measure.

If the system rewarded individual factors such as kills or score, then players who provide intangible support to the team would be punished for the role they play. Additionally, this could create competition between team members because everyone would be trying to get as many kills as possible and might try to achieve this by purposefully not making any callouts. Solving the problem of measuring and evaluating individual performance in a highly team based game is an extremely challenging task and I am in no way surprised that ranked is still in Beta even after more than 2 years. As I spoke about earlier as well, another problem with the ranked system is player’s attitude towards it. Everyone has a natural tendency to view the MMR gain and loss as reward and punishment which is made even worse by the way in which Ubisoft have set the system up. I think that displaying the ranks in the game like earned medals and dishing out charms at the end of the season is completely counterproductive and causes negative consequences in terms of boosting, cheating and other undesirable player behaviours.

Ubisoft have publicly announced that the rankings system is not a reward system and yet everything in the game is set up to suggest that it is. This is completely silly in my eyes and I believe that to fix this problem, the algorithm has to either be changed in order to actually reflect personal performance or these confusing rewards need to be removed. I know that that is a harsh demand and that part of the fun of trying hard in ranked is to come away with better and better little badges on your profile and charms on your guns but the fact is that until the ranks can actually reflect personal proficiency in the game those things are meaningless anyway and just incentivise negative player behaviours. But those last thoughts are of course just my personal opinions and I do hope that I have managed to give you a better insight into how the TrueSkill ranking system works in Siege.

What are your thoughts in general on ranked and has any of the info I provided in this video changed your view in any way? Let me know in the comments section below and with that, thank you so much for watching, I hope you enjoyed the video and I will see you in the next episode!.

Read More: Kanal Rework Side By Side Comparison: Rainbow Six Siege