Sunday 2 October 2016

Math on Treasure Chests, New Entry/Prize Structure, and EV

WotC recently announced a huge amount of changes coming for MTGO. I'm not going to touch here on Redemption changes (which basically seem to be straight negatives for the consumer - only silver lining is that online cards won't be cheaper; older cards having value suggests MTGO economy won't completely collapse, but more than that I can't really say). However there have been many other changes announced, dealing with Entry Fees, Prize Payouts, and the new "Treasure Chests". Let's use some analytical mindset, some math, and try to break it down.

I want to note before we begin that, I've spent a decent chunk of time (several hours) putting this together. I've gone through my methodology in a decent amount of detail. Having said that, it's very possible I'm wrong about one or more things here. And I expect, given my conclusions, that many people will want to shout that I am wrong, because they seem dead-set that my end conclusion is wrong. That's fine - if you think I am wrong, I actually actively want to know. But I don't want just a "You're wrong!" or especially a bunch of vitriol without explanation. I want you to point out, at what point am I going wrong? Which of my estimations or assumptions or methodology is wrong? And why? In short, tell me I'm wrong, but give me a reason. Thanks in advance.


Pack Prices:
The most obvious thing here that most people are jumping on is that non-pack entries to drafts/sealed events are lower now than before (while entry fees including packs haven't changed). The supposition is then that pack prices are going to decrease, which de facto will reduce the price of packs and therefore any prize support which includes packs. And this isn't nothing - improving the quality of options competing with using packs should make the packs less valuable to some extent.

Things, however, are not so cut-and-dried. If you look at current booster prices, they're well below 3.33 tickets per pack. The cut-off point is simply not a huge factor as to why they are priced where they are. So what does cause this? Supply and Demand, of course. More specifically in this case, the supply is determined by how many packs people buy from the store plus how many packs are being paid out as prizes. The demand is how many packs people want to crack plus how many they use in limited events.

I assume that far more boosters are being used for limited than just cracked. This is simply because cracking packs is really bad EV. Prices of cards is a factor in this, but that bottom cut-off, like the top cut-offs of buying from the store or being worse than equivalent entry using non-pack entry options, is far enough away from actual price points that it doesn't come into consideration.

I also assume far more packs get put into the system from prizes than are being bought from the store. Again, this assumption stems merely from the fact that it's far cheaper to get packs from other places than buying them from the store.

Given these sources of supply and demand, I'm actually expecting pack prices to increase (slightly, if we don't account for the Redemption Change anyway). This is because supply should be going down - the number of packs awarded in constructed events have been slashed fairly significantly, in favor of other prizes. In the meantime, the demand for packs should not change terribly much. Sure, lower price on the Play Point/ticket entries should change that somewhat, but because that option was and still will be more expensive in practice, this difference shouldn't be a large one. A more significant factor will be actually how many people want to play in those limited events - which has a lot to do with the quality of the format. Of course, being a totally different format is going to make a 'real' comparison somewhat difficult. But the important point here is that if you think the decrease in prizes from constructed events is relatively larger than the decrease in demand because people will enter for play points/tix, then pack prices should go up rather than down. I certainly believe this to be likely, but more importantly, I find it very unlikely that it's so far out of whack that pack prices will significantly drop.


Treasure Chest value:
Each treasure chest has 3 items in it. One of the 3 is guaranteed to be a random Rare/Mythic from a modern set OR a 'curated' card OR some number of Play Points. I'm going to call this a 'value' slot hereafter, even though a lot of the time, this slot will have virtually 0 value. The other two slots will usually be a common or uncommon from Standard, but there's a 1 in 4.5 chance that you'll get one additional 'value' slot and a 1 in 239 chance that all three slots will be 'value' slots. This means you expect 1.23 'value' slots per Chest, on average.

I'm going to assume standard-legal commons and uncommons have 0 value. This isn't strictly true, of course, but typically very few of them have significant value, and there are many many many commons and uncommons in standard, so this approximation is very likely to not be wrong by more than 1 cent or so, which is small compared the estimation errors we have to make from approximations.

So then we need to figure out what the EV of a 'value' slot is. This is hard, because there are a good number of unknowns. Let's start with the knowns though. I will note that for the following, I'm taking everything Pre-Kaladesh, since that set isn't online yet, and the economy for that set hasn't stabilized. Kaladesh will change the numbers a bit, but likely not by much (and probably slightly upward at first, given that most of the cards about to rotate out of Standard have lost most of their value already)

Random Rares/Mythics from Modern:
There are 3063 different rare printings in Modern sets (including Timeshifted cards, and each printing separately). There are 434 different Mythic Rare printings (again, counting each printing separately). In the chests, each rare appears twice as often as each mythic, so there is a (3063*2)/(3063*2+434) = 93.4% chance of hitting a rare, leaving 6.6% to hit a mythic.

I went through https://www.mtggoldfish.com/ set lists and added the prices of each rare and mythic, rounding for convenience and generally ignoring cards less than about half a ticket. I realize that these prices are sell prices rather than buy prices, but I will try to account for that later on (if I knew of a source that had clean lists of buy prices, I would use that instead, as it would be more accurate. Please please let me know if you have seen someone do this analysis more precisely somewhere else). (I should also note that I think I used non-premium versions for everything, which is wrong for a few cases I believe). I came out with a total value of 2067 tickets (this looks more precise than it is; I expect I'm off by maybe 15 or 20 tickets one way or the other, wouldn't be surprised if it's a bit more). This gets to .675 tix per rare, on average, in modern (again, sell prices). I will note that most of the value here comes from the pre-mythic era, with another significant amount on Standard cards. Mythics come up to 1170 tickets (again, I expect to be off by 10-15 tickets in some direction or other), which gets us to 2.70 tickets per Mythic on average. When you combine all of this, you get to a weighted average of 0.81 tickets per Rare/Mythic from Modern. This is pretty closely in line with the number produced here, which I saw after doing my calculations.

Now, as I mentioned before, this is using sell prices, whereas to actually determine the value to most people opening the chest (i.e. people who wouldn't be buying the card), you need a buy price. Doing spot checks on the differences between these, they seem to, for the most part, be a little bit below 90% of the sell price. So I am going to be a little conservative here and estimate a 15% reduction in the sell price to get an estimated buy price of (after artificially rounding down) 0.68 tickets per Rare/Mythic in a Treasure Chest.

Curated Slots
The list of curated cards can be found here: http://magic.wizards.com/en/MTGO/articles/archive/magic-online/treasure-chest-curated-card-list-2016-09-29 I've seen the (unweighted) average of prices of the list as anywhere from 5 to 7 tickets. The best analysis I've seen is here: https://www.reddit.com/r/magicTCG/comments/554g55/value_of_mtgo_treasure_chests_curated_cards/?st=itpryd7g&sh=468d6663 and it suggests slightly over 5 tickets per card, not including any value from the Gearhulks (which will probably bump the average between .1 and .2). It's very hard to know the actual value of what these will be, because the frequency of the different cards are different. I haven't seen anyone who thinks the opened average will be anything besides less than the unweighted average (generally they are going to want to keep the rarer cards rarer); it's just a question of how much less. My guess is that it will probably between the 3-4 range, closer to 4. I've seen other people estimate on the order of 2.23. When I calculate different potential EVs for a chest, I'll present a number of different options.

Play Points
I'm going to use a conversion of 10 Play Points = 1 ticket. This holds for entry into events. Tickets are obviously more liquid assets, and thus more valuable. But the loss in value really only comes in in one of two places: First, if you're winning enough that you're always running a surplus of Play Points. In this case, you end up with excess play points, rather than other assets, and because you can't transform them into other assets, it's merely a waste. Second, if you want to sell all the assets out of your account, you can't get any value out of the play points. However, there are very few people in the first scenario (plus they are very profitable already, just less so than they otherwise would be). And my assumption is that people will typically play in more events at a far higher rate than they will sell out of their account. So all in all, yes, I would definitely rather have 1 ticket than 10 play points. But over the large scale, the differences are so small that I'm using the conversion. Take that for what you will.

I will add for a moment that these changes, because they increase the number of play points going out as prizes, necessarily will make more people fall into category one, where they are left with mounds of play points that just go up and up and up over time - particularly players who are pretty good at constructed but rarely if ever draft. That's a serious negative for that group.

In terms of how many Play Points you can expect from Treasure Chests, in the video where Lee Sharpe demonstrates the chests, we see him open 10 total chests. By counting the number of cards contained (26), and subtracting that from the total number of slots (30), we can see that 4 of the slots must have been devoted to Play Points. We also see that he opened a total of 50 Play Points, which gets us to a displayed average of 12.5 Play Points per Play Point payout. Obviously this average is based on a very small sample, so it could be off by a reasonable amount. My guess is that this was 3 sets of 10 Play Points and 1 set of 20 Play Points. So I'm guessing there's some distribution for Play Points which is unknown, but most likely 10 is the lowest and most common payout, sometimes you get 20, maybe sometimes you get more.


Adding it all up
If we look at the same video, we can see that Lee opened, from his 10 Chests, 12 'value' slots, which is close to the average we'd expect based on the reported numbers (in fact, it's the most likely number; DEFINITELY within normal variation). Of these 12 'value' slots, as I said 4 were Play Point bundles. Two were curated cards (Remand and Force of Will). This leaves the remaining 6 as being random rares/mythics from Modern (unless I've missed somewhere that one of those was curate - please let me know if that's the case).

This gets us to seen averages of 1/6 Curated Cards, 1/3 Play Points, and 1/2 random rare/mythic from Modern. Again, this is small sample size, and there's a very good chance it's off a little bit in some direction or the other, but it also seems quite plausible that this is the distribution.

If we take the seen averages as real averages, then if we take average curated card at 3.5 tickets, we end up with 'Value Slot' EV of:
(1/6 chance of curated)*3.5 Tickets + (1/3 chance of play points)*1.25 tickets + (1/2 chance of Rare/Mythic) *.68 Tickets = 1.34 Tickets. This leads to a Chest EV of 1.65 Tickets.
If we shift Curated average to 3 tickets, we fall to 'value slot' EV of 1.26 Tickets and Chest EV of 1.55 Tickets.
At Curated = 2.5, 'value slot' = 1.17, Chest = 1.44. And at Curated = 2.0, 'value slot' = 1.09, Chest = 1.34.

Other values, you can do the Math yourself (or if you ask nicely, I will probably get back to you).


In terms of the bottom line on restructuring of Prizes and Entries:
This post  has the best breakdown I have seen so far. If we apply current pack prices and make our 10 play points to 1 ticket equivalency, then Competitive Leagues need Treasure Chest value to be .733 Tickets/Chest in order to be equal overall (weighted average of all records). As you can see, even the most conservative calculations above are showing Chest values to be will in excess of this figure, which means effectively that prize payouts in these leagues have increased (though not necessarily that they've increased by a large amount).

In Friendly leagues, if we also ignore QPs being gone, then Chests need to have an EV of 1.46 tickets in order to break even overall before compared to now. The pessimistic view would then show these leagues as now paying out less. However the more optimistic or average-case-best-guess estimates show an increase in payouts for friendly leagues, too.


Bottom Line: Payouts are increasing for Competitive Leagues almost certainly, and there's a good chance (but less sure) they're going up for Friendly Leagues as well.





Caveats and Downsides:
It's not all good.

Of course, we don't actually know the distribution of things. I think you'd have to be quite cynical to think that they've rigged what Lee showed in the video to be unrepresentative of average in a big way, but it's of course possible he got slightly luckier than average on categories. The big question, though, remains in the Curated Cards. If they weight it such that the expensive cards are WAY less likely than the Atogs of the world, EV will plummet. At the worst case of Curated Card EV = 0, Chest EV goes all the way to .93 Tickets per chest, which means Competitive Leagues would STILL go up, but if you factor in that Play Point and other distribution might be different, it could be a little worse.

Furthermore, WotC's lack of transparency on these points is troubling. I certainly don't expect them to tell us the EV of a chest, since that will be in Flux, and they want to avoid appearances of paying a set rate of $$ so as to not look like gambling. However, I see little reason why they can't give us more information about the distribution of the Curated cards (or their methodology here) and especially why we can't know how often you get Play Points vs Curated vs random Rare/Mythic. It's also somewhat bothersome that they have given us no information about the distribution of the different sized bundles Play Points come in - or even that there are bundles of different sizes.

Also, this prize situation creates an enormous Variance in prize payouts beyond what existed before. So even though on average things work out ok, in the short run, there's a big chance that you are worse off than before, because many curated cards and most random rares/mythics are worth very little (and also somewhat harder to trade than packs, because they all have different names). People want to have stable prizes. They also want to make sure that a 5-0 will get them a bigger prize than a 4-1, which will be bigger than a 3-2, which is no longer the case. Yes, we were getting more packs before, and those could be opened which also has big variance. But WotC knows, just as the community knows, that people weren't, for the most part, using these packs to open them; they are either entering limited events with them or trading them away - neither of which are things that Treasure Chests can do. Making the Chests themselves tradeable would go a long way to solving the variance problems, but of course it's not a panacea.

As I mentioned above with play points, the prizes now are less liquid than before. And more players will be stuck with playpoints they aren't using (at which point those play points become valueless). This has its definite negatives, and... no real positives for the consumer.

Card prices for a lot of the curated cards (as well as the random rares/mythics from Modern) will also likely fall to some extent following this, which will eat into the EV. It's really hard to know exactly the size of this effect, and it probably won't be much, but that is something which can definitely be a negative as well. Actually let me expand on this a little bit. For any given Modern rare, the chance you get one from any slot is 1/2 of 2 in 6560, so for any chest it's about 1 in 5330. The average league run hands out 1.03 chests. So 1 in 5170 leagues will add an Engineered Explosives to the system (for example). A given Mythic is half as common. 1000 leagues per day (very rough guess) means 1.3 to 1.4 of any given Rare, and somewhat less than 1 of any given mythic, are entering the system per week. The numbers for Curated cards are going to be likely on the same order of magnitude - there is less of a chance to get a curated card, but there are fewer of them as well, and we again don't know the weighting between the different cards. In any event, it seems very clear that the influx rate is significantly lower than we're getting from the Flashback drafts. This is a continual increase in supply of course though, and would need to be offset by a sustained increase in demand over time to keep prices from falling; however, the rate of a few per week entering the system means that not that much net increase in game play (of that specific card) has to occur for prices to remain fairly stable. Therefore, I expect prices to drop on cards which are expensive mostly because they've been stupendously rare (e.g. Rishadan Port), while remaining reasonably steady (not bigger changes than normal variations we've seen in the past) for cards which are expensive because they see gobs of play (e.g. Tarmogoyf, Liliana of the Veil).

Finally, and this one hasn't been talked about much, contents of the chests are set on opening, rather than on obtaining the chest. This is not surprising given that it would be a bit of a disaster to try to show "this is the chest you got on date X, this is the one you got on date Y", etc. But the important thing is, it encourages you in many ways to hold your chests until they change the curated card list/distribution to be more monetarily favorable. In fact, there's a decent chance that this is the reason they aren't telling us the curated frequencies. But if that's the case, it's a poor remedy, a medicine that is likely worse than the disease. What would be far preferable would be a way to solve the original problem, to remove the incentive for sitting on chests (by not allowing that to be possible/plausible).

2 comments:

  1. """
    This gets us to seen averages of 1/6 Curated Cards, 1/3 Play Points, and 1/2 random rare/mythic from Modern. Again, this is small sample size, and there's a very good chance it's off a little bit in some direction or the other, but it also seems quite plausible that this is the distribution. """

    Bold assumption off an extremely small sample from a promo video (which had a 0 chance of not producing a good card, they are selling us the idea). Lets see if you are right.


    ReplyDelete
  2. Big point is that for competitive leagues, all three options are above necessary EV. So the distribution doesn't matter for better/worse on competitive leagues. (Obviously it does affect actual value).

    As for the other part, if you go in assuming they're being dishonest, then sure, you'll continue to think so.

    ReplyDelete