Experience-Driven Development in cEDH

Over my years playing Magic, it seems more and more common that players I meet have an interest in or actively work in software development. I am no exception! It makes sense too, given the similarities Magic has to the functionality of software development. Something I don’t see nearly as commonly though, is high level principals being ported from the software development world to Magic. This perplexes me, given the natural fit, and during the first session of coaching for all my coaching clients, I put forth an iteration process using tools derived from agile software development to more effectively organize and track improvement. Testing for tournaments and events in an iterative way has been paramount in my journey to use the precious little amount of time I have as efficiently as possible.

How To Develop Decks

To develop, or “tune”, decks you need to have a baseline for assessing each iteration to improve. The most common approaches I have seen for this fall into one of three different approaches: experience-driven, data-driven, or theory-driven. For example, say you are playtesting Modern and want to know how good/bad the Burn matchup is for UR Murktide. Assuming you conclude the matchup is unfavored, here are three different paths to get there based on approach.

Experience-driven: I think the Burn matchup is unfavored because when I played against Burn the game that I won was close, I needed my opponent to brick on burn spells for a draw step, involved a quick Murktide Regent, and counterspells were drawn at exactly the right time. It was a lot to ask and there wasn’t much help out of the sideboard outside more cheap counterspells. The games I lost didn’t even feel very close because I lacked pressure and traded on mana at or below rate.

Data-driven: I think the Burn matchup is unfavored because, over the last 4 tournaments attended by 200 or more people, UR murktide has only won 41% of the time against Burn.

Theory-driven: I think the Burn matchup is unfavored because Burn has a quick kill using similar, interchangeable parts that do a lot to mitigate Murktide’s counterspells, Eidolon of the Great Revel has traditionally lined up well against UR decks that sometimes get stuck spinning their wheels with cantrips, and there is no playable life gain in the colors to help the matchup out.

The example conclusions here are concise, and each is backed up by evidence of varying reliability. As Andrew Elenbogen so eloquently put it, the sample sizes are essentially always too small to make any claims with complete 100% confidence and accuracy, but some evidence is better than none. I believe all three of these methodologies have their place, and the most effective players use all three methods when appropriate. Experience-driven development, however, I have found to be the most universally effective. It is also the most under-utilized.

Players feel more comfortable having their claims backed up by data, no matter how small and flawed, than trusting their “feelings” on something based on their own small experience sample. While understandable, this leaves your most effective tool on the table. After all, YOU are going to be the one playing the games in the tournament, so the things that happen in-match for you during playtesting are liable to happen in-match come tournament day as well! Any number of people can make claims to the contrary but they are not going to be the ones sitting in your seat playing your matches. The foremost among many reasons to trust yourself and your experiences are that it factors in how you play matchups, your experience level, and your approach to the game as a whole.

Deck Development in cEDH

When turning to a multiplayer format like cEDH, there are many more variables to work with given the added players and it is important to turn the knobs a bit on your process. The massive increase in variables delivers even more blows to the already dubiously overused data-driven and theory-driven approaches. Despite this, I was shocked when I began my venture into cEDH spaces to see that the vast majority of deck development and iteration was done via theory-driven approaches in online forums. Not only is the information tucked away, so people can’t necessarily find the discussions as easily to contribute, but it also created a rift in the perception of what cEDH even is with respect to power level. Given the drought of data from lack of events in the cEDH space, it was no surprise that there was little to be gained talking about that approach unless you are tooting your own horn about your win rate; although I would argue even that is still mostly experience-driven.

Even today, the most densely populated areas of discussion about cEDH are dominated by theory-driven approaches and hand-waving away optimization advice through concerns with regional metagames. For the most part, this mixture either grinds the iteration process to a halt or regresses the deck to a less potent state than before any effort was put in. This happens because much of the justification is based on the weakest evidence that exists and, unless brewing something from the ground up, should mostly be avoided. The most iconic example of where theory-driven approaches fail is spoiler season. Most rares or mythics are simultaneously called “broken” and “unplayable” in the same thread. The best way to find out who is right? Play games!

Applying Experience-Driven Development

Experience-driven development in any format has pitfalls. Fundamentally experience-driven approaches are logical fallacies because they’re all proof by example. Just because something happens in games once (you get blown out by Otawara, Soaring City on a Underworld Breach combo turn, your Ad Nauseam whiffs, your Thassa’s Oracle gets Stifled) is not indicative of what is going to happen every time and cannot be assessed as such. Given that full-fledged proofs can’t be accomplished in Magic, the best way to communicate effectively about experiences is twofold. First, state the experience with context. Second, avoid making overly specific conclusions.

This tweet by Matt Sperling sums up the concept nicely. Let's look at a concrete example that is entirely made up and not based on real events at all:

❌ EDD done wrong: I keep every hand with Sylvan Library in it because drawing two additional cards each turn for two mana is just one of the best effects in green, especially if games go long like they do in my stax heavy meta. It works most of the time. I definitely would put it in your Codie, Vociferous Codex deck.

✅ EDD done well: Sylvan Library triggered 8 times in my all stax pod when I was on Kinnan, Bonder Prodigy and helped keep me in the game so it may be worth keeping in if there is a lot more stax but I have died after casting it on turn 2 in more combo-focused pods so I may be better served with more interaction in that same slot given a diverse pod composition. Going to keep playing and see if I just need to change how I sequence Sylvan Library and my stax development plan in faster pods and if still isn’t performing I am going to try Minor Misstep in that spot to see if that helps.

Despite being a little exaggerated, the above “wrong” example is, unfortunately, very commonly how discussions are being had and it can be detrimental to newer players looking for advice. The best way to avoid running into this issue is naturally to practice, but specifically to avoid reading cards to people, and when asserting your experience as something worth listening to, make sure you list all appropriate contexts. If in doubt, add more context! It doesn’t make your experience or your argument invalid to add context and goes a long way towards communicating effectively. It is also extremely important not to attach self-worth to any of your in-game experiences, or you will risk getting overly defensive and hindering effective communication.

Keep in mind that your own experiences and iteration process are what matters at the end of the day, and your experiences are going to be the best indicator of what your future experiences are going to be, barring adjustment. It can be completely valid for two people playing the same deck, in the same situation, and both have valid, defensible strategies for approaching that situation. Both may be effective, and a reasonable conclusion from there is to do further testing to see which is more successful. Experience-driven development works best via collaboration, and arguing right or wrong in sweeping abstracts is counterproductive.

If looking to improve at Magic: the Gathering at any level in any format, focusing more on your in-game experiences and communicating your conclusions to peers is one of the best places to take a look. Even if you believe yourself to already be doing this effectively to some degree, there is always room for more improvement and I recommend spending time reflecting on your process for playtesting and collaboration to seek areas of improvement. Akin to effective software development teams, there is always room to iterate! Effective use of experience-driven development and communication skills are critical to “leveling up” in Magic and becoming the best player you possibly can be. Given time and practice, you will notice you get more value out of conversations with your peers and vice versa.

For now, thanks for reading!