Evaluating Different Sources of Data About the MTG Meta
By: variancekills - 17 Jul 2021
One of the important aspects of competitive play is selecting which deck to use. Obviously, there are two general routes. You can either choose a list that already exists (and make modifications as you believe is needed), or brew something entirely new. Each of these have their merits, but this article is focused on the former. In particular, when you are choosing a deck from existing lists, there are many different sources of information at your disposal. In this article, we go over some of those sources and evaluate them.
Winning Decklists in MTGO
MTGO publicly releases lists of decks that went 5-0 in their constructed leagues. In addition, they also publish the decks that perform well during big tournaments like Format Challenges, Super Qualifiers, etc. These lists are extracted by other websites, and you can find them in sites like MTG Goldfish.
Each MTGO league event costs $10 to enter, format challenges cost $30, and other events require points accumulated from finishing well in leagues/challenges. Thus, I consider information from these lists as top notch when it comes to player quality. That is, the people who play these events typically play regularly and put money on the line every time. Indeed, some even make a living from grinding these events. This makes these lists truly battle-tested in a setting where every match has something at stake.
The biggest con of this source of information is that the lists of 5-0’s in leagues that are released now are curated. It used to be that Wizards released all the decks that 5-0 periodically. However, this has not been the case for quite some time now. Thus, the current releases cannot be relied on as a determinant of which decks, if any, are dominating the meta. This is a serious limitation. The fact that some strange list 5-0’d doesn’t mean that the list is dominating.
Another limitation of this source of information specifically for MTG Arena players is that it does not provide information about Historic since Historic is not available as a format in MTGO. This is actually a critical issue with Historic as a format. It does not have a steady source of data from events with non-zero stakes that is participated in by people who are financially invested in their efforts.
6+ win lists in MTGA
WotC periodically releases a list of decks that have gone at least 6 consecutive wins on the Arena ladder at platinum and up.
The only pro of this information is that it is taken from MTGA, which means that if you play MTGA, then this is taken from the same meta that you play in.
A major issue of this list is that it does not tell you much. The list is not aggregated, and so we do not know how many RDW decks were able to do 6+ wins relative to the other decks. This is the same for the MTGO lists though, but the difference is that you can have greater confidence in the MTGO lists being good by virtue of them going 5-0 in a paid league where everyone is trying their best to win. In contrast, 6+ consecutive wins on the ladder are much less valuable. No one wins anything from doing that, and the impact from a loss (dropping down the ladder) is practically nothing. The information from the list may be useful when viewed in its entirety (as our next item will show), but it would be folly to trust a random deck in that list, craft it and expect it to perform well.
Aggregated lists from 3rd Party providers
A number of websites take information released by Wizards as well as information from other sources, such as tournament results from mtgmelee.com and aggregate them to provide statistics for their consumers. Among those available, I mainly use MTG Goldfish. Some authors also specialize in content that aggregates tournament results. In particular, I always read articles produced by Yoman5 such as this one.
These aggregated lists provide a wide snapshot of the meta. These are especially good for spotting obvious choices. That is, when a deck is broken, it can be fairly obvious when you look at the Meta%. Even if the 5-0 lists, which are included in this aggregated info, are curated, the addition of info from both in-client tournaments on MTGO and externally organized ones on MTGA can be reliably counted on to reveal any obvious winners. This means whenever you see the top deck/s here having something like a 25% share or better, that is a fairly good indication that that deck is dominating. In addition to this, articles on these aggregated lists, specifically the ones written by Yoman5, provide considerably good insights on why certain decks are at the top of the list, and actually provide win rate estimates not just in general, but by matchup.
The biggest issue with these aggregated lists is that while they are good for spotting obvious winners, they are not good for identifying not-so-obvious ones. That is, when the meta share among the top 4 decks is close, the difference really does not matter so much. A 15% share for the top deck vs an 11% share for the next deck and then an 8% share for the next for example, does not really give substantial evidence for one to choose the top deck. Nonetheless though, even if you cannot use it to choose the best deck, it at least allows you to considerably narrow down your selection and to weed out obviously bad choices.
Another issue is that the sources of aggregated data may vary widely in terms of event quality. For example, estimates by MTG Goldfish (and presumably other similar sites), can include data from both small events with no entrance fees, to large events with significant entrance fees. The level of competition between such events can be night and day. Fortunately, you can always look into the specific events from which the data came from and pick out only the ones that you think matter. Unfortunately, you will have to do this manually.
The third-party apps that you can install to track your games on MTG Arena also aggregate user data. This data can include estimates of win rates, drawn from hundreds, sometimes thousands of matches. For some apps, it is possible to filter the data, such as only showing results for matches played in best-of-three, or only matches played on mythic.
Like the 6+ win lists released by WotC, one of the best aspects of this resource is that it lives in the same place as we do if we mainly play MTGA.
However, equally important is the size of the data from which these estimates are made. Unlike the 6+ win list released by WotC, the data from trackers incorporate both how well the list did and how many use the list. Thus, only decks that perform consistently well end up actually being detected.
In addition, if data is collected well, collecting more and more data will allow the estimate to approach the true value of what is being estimated.
An obvious issue with tracker data is that they are biased towards the experiences of the people who have the tracker and their opponents. However, this issue is not really that important since it is also reasonable to assume that the people who take the effort to install a tracker are also more likely to be players who try to be competitive and use at least what they perceive to be objectively good decks.
However, the major disadvantage of this resource is the quality of the ranked ladder as a basis for selecting decks for big/paid tournaments. Undoubtedly, many players play the ladder competitively and are serious about winning, but as previously pointed out, the lack of actual stakes does take away from the extent to which players would strive to win. This is also important to keep in mind when using the ladder to practice a deck, but that is a conversation for another time.
Each of these sources of information are useful, but it is important to understand their strengths and weaknesses to make sure that you make sufficiently informed choices. At the end of the day, we can never be 100% certain that we have selected the best deck, although there are some situations where we can be almost certain. Furthermore, selecting the deck is only half the battle. The best deck would not take you very far if you cannot play it without error.
Check out the youtube video for this article for another code giveaway! Follow this profile so you can catch more articles and giveaways as soon as I post them.
May the shuffler be with you!
Hi, I'm Mark. I've won exactly one World Magic Cup Qualifier, one Preliminary Pro Tour Qualifier, one Arena Open ($2k) and one CFB Pro Showdown (April, 2021), and I am looking to win more. I've played in almost every Mythic Championship Qualifier Weekend and made Day 2 four times (so far). Follow my FB page or subscribe to my Twitch channel for no frills, competitive Magic. You won't see my face, but I won't hide my gameplay and deckchoice flaws. I play both MTGA and MTGO and stream most of the time when I do. I will lose often, and I will make mistakes, but I try my best to let you know when I do (and I think I will still win a lot more times than I lose).
The challenge for me every season is to find the best decks and make the most use out of them before they're no longer the best decks, or they get banned. I play a ton of limited when a set is just released in order to complete most of the rares for the set within a short timeframe. With the availability of Premier Drafts and the Metagame Challenge, I've spent as little as 2000 gold completing a set's rares. I'm currently sitting on over 800k gold and over 70k gems but have spent a grand total of $105 in the game.
I produce occassional video articles about the game that cover things that interest me, such as how to know when your rank is safe for a month's MCQW slot, how much you need to spend to get to Day 2 of an Arena Open, or how MDFCs affect Arena's Bo1 shuffler. I also mythbust a ton of crap about the game that you've no doubt heard or read about somewhere.
Finally, I'm a dad and husband first, a statistician, teacher, and researcher second (I know those are 3 things but bear with me), a Magic player third, and a content creator only because I am a Magic player.
So yeah, let's play some Magic and may the shuffler be with us all.