As I close out this final installment of my 2018 NCAA Tournament postmortem analysis, I wanted to return to some of the metrics that I developed to quantify each coach's performance in the NCAA tournament, which are explained in more detail in my previous post, found here. The first metric "PARIS" essentially calculates the number of games above or below average a coach is for a given seed playing in a given round. For example, Tom Izzo is 3-0 as a 5-seed in Round 2, but all 5-seeds in history are 55-55 (50%) in Round 2, which means Izzo had won 1.5 more games than average (50% times 3) as a 5-seed in 2nd round games. Performing the same mathematical operation on all 68 on Tom Izzo's NCAA games results in a PARIS of 8.09.
The second metric of interest is the "PAD" which is similar to PARIS, except that it instead considers each coach's performance relative to their seed and the seed of their opponent in each game, independent of round. The reason that I developed the PAD metric was due to the fact that not all NCAA tournament paths are created equally. For example, of Tom Izzo's 3 games as a 5-seed in Round 2, two of those games were against 4-seeds (where the 5-seed historically wins 44% of the time), but one of those games was against the 13-seed (which a 5-seed will historically beat 82%) of time. Izzo's performance against those exact seeds is actually only 1.3 games better than average. Because of this more exact treatment of each exact seed combination in all games, I consider the PAD rating to be the most accurate way to measure a coach's tournament performance against expectation. But, I think that it can tell us a little more than that.
In this case, Izzo's PAD score is slightly lower than his PARIS score because he was fortunate enough to draw the lower seed in 1 of 3 tries, which is slightly higher than the odds that a 13-seed upsets a 4-seed (which is 1 in 5). Had Izzo gone 3-0 in Round 2 all against 4-seeds, this would have been slightly unlucky relative to average, and Izzo's PAD would have been 1.69. What this simple 3-game example demonstrates is that the mathematical difference between the PARIS and PAD score is a quantitative measure of how lucky a coach's draw is in the NCAA tournament, whether they win or lose a given game. So, I decided to calculate and plot this "luck differential" (PARIS minus PAD) for all coaches in the modern (1979-now) era. The graph below shows the result of this raw calculation for all coaches with more than 10 tournament games played.
A slightly different way to attempt to normalize this data a bit is to scale the luck score by the number of games each score has played past the first round of the tournament (where the PAD and PARIS are essentially equal). This gives a bit more information about the intensity of the luck each coach has experienced and that distribution is shown below.
In this case, the tails of the distribution are now dominated by coaches with few Tournament attempts (such as former Virginia coach Pete Gillen, who only coached in eight total games past the first round, three of which were against 10 seeds or worse). Some coaches such as Coach K and Roy Williams are now closer to zero, but coaches like Billy Donovan, John Beilein, and Bo Ryan are all still very much in the lucky category.
Finally, speaking of John Beilein, there was a lot of talk in the 2018 Tournament of Michigan's seemingly very easy run all the way to the Title game. So, an obvious question is how does this path compare to other paths teams have taken to reach the Final Four or win the Title? There are several ways to look at this. One metric is simply to use the Luck Differential (PARIS-PAD) metric summarized above, although a relatively lucky draw and an easy draw are slightly different concepts. Another would be to add together the sum of the seeds of all opponents that each teams faces on their particular tournament path. The problem is that, by design, the higher a team's seed, the easier their path is supposed to be. For example, a 1-seeds ideal "seed sum" to reach the Four Final is 16+8+4+2= 30, while for a 5-seed this sum is 12+4+1+2=19. So, a potentially more fair method is to calculate the deviation from this ideal "chalk" seed sum that a given team actually faced. For example, in 2001, 1-seeded MSU faced a 16-9-12-11 seed on the way to the Final Four. This is a "seed sum" of 48, which is 18 seeds higher than the 1-seed chalk value. Finally, I thought that it would also be instructive to show the best seed that each team defeated on their journey. So, I decided to tabulate all four metrics, sorted by the seed differential, for all teams that have made the Final Four, Championship Game and won the Title. I will start with teams' path to the Final Four (where the metrics were calculated only up to that point).
Finally, the table above shows the full path for all 40 NCAA Champions in the modern era. The undisputed champion of the easy path goes to the 1990 UNLV team whose best win of the tournament was over 3-seed Duke in the Finals. Kansas's 1988 team, Florida's 2006 squad, and the 1986 Louisville team all remain in Top 5 as well. The only relative surprise in this table is the 2013 Louisville team (16, 8, 12, 2, 9, 4) whose score was aided by a surprisingly weak set of opponents in the Final Four. At the bottom of the table we see that no team have ever won the title by taking the hardest possible path, but Jay Wright came close in 2016, as did Dean Smith in 1993 with UNC. However, the closest in reality was the 2015 Wisconsin team who would have accomplished this feat had they beat 1-seed Duke. No other team has entered the Finals with this opportunity.
Well, that just about does it for 2018. I hope that everyone enjoyed this 5-part series of mine. Thanks for reading, and until next time, enjoy!
As an added bonus, I have one more plot to share. After completing this analysis, it occurred to me that I now have two metrics, one that measures raw tournament performance against expectation (PAD) as well as a metric that measure the luck of the draw (PARIS-PAD). If I plot the data for each coach, I can generate a 2D scatter plot that categorize coaches based on whether they are lucky and good, unlucky and good, lucky and... not-so-good, and unlucky and not-so-good. That plot is shown here:
The second metric of interest is the "PAD" which is similar to PARIS, except that it instead considers each coach's performance relative to their seed and the seed of their opponent in each game, independent of round. The reason that I developed the PAD metric was due to the fact that not all NCAA tournament paths are created equally. For example, of Tom Izzo's 3 games as a 5-seed in Round 2, two of those games were against 4-seeds (where the 5-seed historically wins 44% of the time), but one of those games was against the 13-seed (which a 5-seed will historically beat 82%) of time. Izzo's performance against those exact seeds is actually only 1.3 games better than average. Because of this more exact treatment of each exact seed combination in all games, I consider the PAD rating to be the most accurate way to measure a coach's tournament performance against expectation. But, I think that it can tell us a little more than that.
In this case, Izzo's PAD score is slightly lower than his PARIS score because he was fortunate enough to draw the lower seed in 1 of 3 tries, which is slightly higher than the odds that a 13-seed upsets a 4-seed (which is 1 in 5). Had Izzo gone 3-0 in Round 2 all against 4-seeds, this would have been slightly unlucky relative to average, and Izzo's PAD would have been 1.69. What this simple 3-game example demonstrates is that the mathematical difference between the PARIS and PAD score is a quantitative measure of how lucky a coach's draw is in the NCAA tournament, whether they win or lose a given game. So, I decided to calculate and plot this "luck differential" (PARIS minus PAD) for all coaches in the modern (1979-now) era. The graph below shows the result of this raw calculation for all coaches with more than 10 tournament games played.
As one might expect, the majority (>85%) of coaches fall into the middle of this chart. Some notable coaches have been a little bit lucky (Izzo, Few, Dean Smith, Pitino, Knight, and Calipari), and others have been a little unlucky (Sean Miller, Matta, Dixon, Painter, and Jay Wright). But, the interesting parts of this chart are the extreme ends. Based on this analysis, the luckiest tournament coaches in history include Bill Self, Tubby Smith, Bob Huggins, Jim Calhoun, Jim Boeheim, John Beilein, Coach K, and Billy Donovan. In contrast, at the other extreme of the chart include the most unlucky coaches in history including Gary and Roy Williams, Lute Olson, and Rollie Massimino.
The data for Billy Donovan is clearly the biggest outlier on the entire chart. But, a glance at his tournament opponents over the years helps to explain this value. Donovan has advanced to the 2nd round 11 times, and 5 of those times he has faced 11-seeds or worse. He has advanced to the Sweet 16 eight times and faced a team seeded 7 or worse three times (including the only 15-seed ever to make the Sweet 16). In his 7 Regional Final appearances, he has faced a 1-seed, two 3-seeds, two 4-seeds, an 8-seed and an 11-seed. In his four National Semifinal games, he has faced a 2-seed, 7-seed, 8-seed, and an 11-seed. Donovan has only faced a 1- or 2-seed a total of 6 times in his entire career, which is by far the lowest of any coach in the Top 20 of tournament wins in the modern era. The average for Top 10 coaches is over 17 and the average is over 10 for the coaches in spots 11-20 of total wins. Tom Izzo has faced 19 1- and 2-seeds. So, in other words, Donovan's luck score makes sense.
Now, this is not to say that all of Donovan's success in March is due to luck. He still made it to four Final Fours and won two titles. He is a very good to elite coach. But, at the same time, the math doesn't lie. It is possible to be both lucky and good at the same time. Billy the Kid is both.
In this case, the tails of the distribution are now dominated by coaches with few Tournament attempts (such as former Virginia coach Pete Gillen, who only coached in eight total games past the first round, three of which were against 10 seeds or worse). Some coaches such as Coach K and Roy Williams are now closer to zero, but coaches like Billy Donovan, John Beilein, and Bo Ryan are all still very much in the lucky category.
Finally, speaking of John Beilein, there was a lot of talk in the 2018 Tournament of Michigan's seemingly very easy run all the way to the Title game. So, an obvious question is how does this path compare to other paths teams have taken to reach the Final Four or win the Title? There are several ways to look at this. One metric is simply to use the Luck Differential (PARIS-PAD) metric summarized above, although a relatively lucky draw and an easy draw are slightly different concepts. Another would be to add together the sum of the seeds of all opponents that each teams faces on their particular tournament path. The problem is that, by design, the higher a team's seed, the easier their path is supposed to be. For example, a 1-seeds ideal "seed sum" to reach the Four Final is 16+8+4+2= 30, while for a 5-seed this sum is 12+4+1+2=19. So, a potentially more fair method is to calculate the deviation from this ideal "chalk" seed sum that a given team actually faced. For example, in 2001, 1-seeded MSU faced a 16-9-12-11 seed on the way to the Final Four. This is a "seed sum" of 48, which is 18 seeds higher than the 1-seed chalk value. Finally, I thought that it would also be instructive to show the best seed that each team defeated on their journey. So, I decided to tabulate all four metrics, sorted by the seed differential, for all teams that have made the Final Four, Championship Game and won the Title. I will start with teams' path to the Final Four (where the metrics were calculated only up to that point).
For the paths to the Final Four, shown above, I only included roughly the top 10% and bottom top 10%, as there have now been 160 total Final Four participants. While each metric tells a slightly different story, there are clearly a set of Final Four runs that were soft based on all four metrics. Based on seed differential, Nolan Richardson's 1990 Arkansas squad is in 1st place, which makes a lot of sense considering they faced a 13-, 12-, 8-, and 10-seed on the way to the Final Four. Syracuse's' 2016 path (7, 15, 11, 1) is in the Top 4, as is Kansas's 1988 run (11, 14, 7, 4), and MSU's 2011 run (16, 9, 12, 11). The aforementioned Michigan 2018 path (14, 6, 7, 9) is tied for 12th place. As for the toughest paths, those are naturally the 26 teams who played the toughest possible teams in the first four rounds and thus had a seed differential of zero.
In the case of the path to the Finals (the top and bottom 25% of which are tabulated above) the rankings change a bit. In first place here is the 2018 Michigan squad, which has the distinction of being the only team to reach the Finals without at least beatings a 5-seed. Tied for 2nd is UNLV in 1990 (16, 8, 12, 11, 4), Kansas's aforementioned 1988 team, and Florida's 2006 squad (14, 11, 7, 1, 11). I would also highlight the 1986 Louisville team (15, 7, 3, 8, 11) who scored a particularly high luck value, and the 2005 UNC squad (16, 9, 5, 6, 5) who never faced a team seeded better than 5 until meeting 1-seed Illinois in the Final. The bottom, unlucky, half of the table shows that five teams in history have taken the hardest possible path to the Finals (seed differential =0), and roughly 25% of all Finalists have a seed differential of 3 or lower, including MSU's 2009 team.
Finally, the table above shows the full path for all 40 NCAA Champions in the modern era. The undisputed champion of the easy path goes to the 1990 UNLV team whose best win of the tournament was over 3-seed Duke in the Finals. Kansas's 1988 team, Florida's 2006 squad, and the 1986 Louisville team all remain in Top 5 as well. The only relative surprise in this table is the 2013 Louisville team (16, 8, 12, 2, 9, 4) whose score was aided by a surprisingly weak set of opponents in the Final Four. At the bottom of the table we see that no team have ever won the title by taking the hardest possible path, but Jay Wright came close in 2016, as did Dean Smith in 1993 with UNC. However, the closest in reality was the 2015 Wisconsin team who would have accomplished this feat had they beat 1-seed Duke. No other team has entered the Finals with this opportunity.
Well, that just about does it for 2018. I hope that everyone enjoyed this 5-part series of mine. Thanks for reading, and until next time, enjoy!
As an added bonus, I have one more plot to share. After completing this analysis, it occurred to me that I now have two metrics, one that measures raw tournament performance against expectation (PAD) as well as a metric that measure the luck of the draw (PARIS-PAD). If I plot the data for each coach, I can generate a 2D scatter plot that categorize coaches based on whether they are lucky and good, unlucky and good, lucky and... not-so-good, and unlucky and not-so-good. That plot is shown here:
Comments
Post a Comment