Glass Blown Open Preview: Win Probabilities

Win, top-5, and top-10 probabilities

April 25, 2018 by Aaron Howard in Analysis, Preview with 0 comments

The pro-side of the Glass Blown Open — the second PDGA National Tour event of the season — starts tomorrow and we’re starting our preview of the event a little early by introducing win, top-5, and top-10 probabilities.

A few months ago we introduced 2017 Elo ratings for the Open and Open Women’s divisions. This week, I am using those ratings¹ to estimate win, top-5, and top-10 probabilities for the Glass Blown Open. Unsurprisingly, the model is quite “chalky” at the top: the best players are a small handful. Paul McBeth and Ricky Wysocki have the greatest chances of winning in Open because, well, they are the best,² and Paige Pierce is the clear favorite in Open Women. Immediately below them, there are not too many surprises with Nate Sexton and Jeremy Koling, and Catrina Allen and Sarah Hokom being the next favorites in their respective divisions.

The tables below show sortable columns containing: current Elo rating, change in Elo rating due to 2017 GBO,³ and Win, Top-5, and Top-10 probabilities (as percentages) for the 20 players in each division with the highest win probabilities.⁴ For players that have not played in any NT or DGPT events this year or last, their Elo ratings were set at 1500, which is the value I use to start estimating a rating for any player.

To get these predictive probabilities, I used players current Elo ratings and how their ratings changed after the 2017 GBO. In other words, the model uses how a player is playing now (using our most current data), and how the player played in the event the year before.⁵ For example, Chris Clemons and Lisa Fajkus both finished third last year, and significantly improved their ratings and chances at this years GBO. I also included strength of field in the model.⁶

When developing models like this it is important to verify their predictive value. You do this using something called cross-validation, where you develop a model using half of the dataset and see how the model does predicting the outcomes using the other half. Using this method, I correctly classified players 96.7-percent, 89-percent, and 82-percent of the time for the winner, top-5, and top-10 portions of the model.⁷ So, it does pretty well. For example, the model correctly predicted the Open and Open Women’s winners at the Jonesboro Open.⁸ It also forecast, exactly, the 5th place finish of Dave Feldberg. This prediction was, in part based on his 8th place finish at the tournament last year. A bit more surprisingly, it predicted that Will Schusterick would finish in 11th place, which many observers would have probably considered too high before the event given the injuries he has struggled with recently. He played well and finished 13th, just two strokes out of 11th place.

What these results hopefully indicate is that this predictive model and the associated ratings have some value. They provide an alternative to the ratings and other statistics produced by the PDGA. Also, they are useful when thinking about important factors that influence a player’s chance to win or place well in a tournament (finishing inside the top-5 or top-10).

In that way, it can provide some information regarding players to potentially (like the earlier examples of Clemons, Fajkus, Feldberg and Schusterick show) keep an eye on, beyond those at the very top of the field. We will continue to preview the 2018 GBO by taking a closer look at some of those players to keep on eye on in a second preview article tomorrow.

1. I added results from 2012 through 2016 for the Open division and 2016 and 2017 for the Open Women’s division, which currently adds up to 134 events for Open and 20 events for Open Women. This is one of the reasons the values look so different for each division and why, at this point, it would be inappropriate to compare Elo ratings between divisions. ↩

2. Note that their Elo ratings are almost identical, which is consistent with their PDGA ratings. Both ratings systems also indicate what is widely known to those who follow the pro game: they have been both the best and most consistent players over the past several years. ↩

3. We know that the courses played at the 2017 GBO and 2018 GBO are different and that the 2017 event was shortened due to dangerous weather conditions. This makes this small portion of the model somewhat questionable for this specific event. Nonetheless, the tournament is at roughly the same time of year in the same city/general location and is still at the PDGA NT level with a similar size field. In those respects, not every variable between each year is completely different. Additionally, it is important to include this aspect of the model now—as the model is developed and refined—as high-level events become increasingly standardized. Comparing how this aspect of the model functions for the 2018 GBO compared to how it functions for the 2017 and 2018 Masters Cup, for example, will be illuminating considering the 2018 Masters Cup next month will be played on the same courses, in the same order as they were in 2017. ↩

4. A table of probabilities for all players is at the end of the article. ↩

5. To estimate win and top 10 probabilities I used a logistic regression, which is a type of statistical model that deals with categorical outcomes (win, no win, or top 10, no top 10). Including how a player played the year before turns out to be important because the model “fits” turn out to be better. Including data from the tournament two years back does not seem to be as informative, perhaps due to significant changes in tournament layouts, player abilities, or diminishing returns due to reduced sample size in the model. ↩

6. More specifically, I calculated strength of field as the mean Elo rating for all players in playing the event. ↩

7. The cross-validation was based on all tournaments in the dataset. See endnote 1. ↩

8. There has been a great deal of parity this year regarding winners in the MPO division. This is very different than in past years and is not reflected in the model. But if this trend continues, the model will adjust accordingly. ↩