1/13 WAB: A Brave New World?

Next week in Indianapolis there is going to be a conference on reforming the metrics used by the selection committee. Here were the people invited:

Ken Pomeroy: The founder of KenPom.com. He started the modern statistical revolution in college basketball. My basketball work on this site is due in large part to the data he provides.

Jeff Sagarin: An old school statistical analyst. He’s been doing statistical work across numerous sports for years.

Jerry Palm: CBS’s resident bracketologist. He’s taller than you think.

Kevin Puaga: Creator of the KPI. I talked about the KPI on Monday (https://sethburn.wordpress.com/2017/01/09/19-wab-resume-power-rating/). Effectively, it is a proxy for where the committee ranks the teams, and as such, it is a derivative of the RPI.

Ben Alamar: He’s done solid work for ESPN. Much of his stuff is proprietary, so I am not as familiar with his work as I am of the people listed above.

Andy Glockner: Author of Chasing Perfection. Writer for Sports Illustrated.

I’m very happy that Ken Pomeroy and Jeff Sagarin will be there providing insight. I presume Ben Alamar will be very helpful as well. I’m a little more skeptical about Jerry Palm’s inclusion. He knows a great deal about how the selection committee works, but I’m not sure what he is bringing to the table in terms of guiding reform. Bracketology is all about predicting the committee’s actions. It doesn’t necessarily provide much guidance in terms of how the committee might want to change its process, but perhaps I am underestimating his insight. Either way, his inclusion doesn’t concern me. Kevin Puaga’s does.

It’s not that Kevin is a bad guy. From all accounts he’s a great person to work with and be around. It’s that his metric, the KPI, is designed to predict the committee. It has no value in terms of reforming the committee’s selection process:

The KPI looks and feels like a traditional predictive metric, but it is not. You can check this out from his website:

http://www.kpisports.net/2016/03/15/defining-the-accuracy-of-advanced-metrics-2016-edition/

He discusses how the KPI was highly successful at predicting the field and the seeding, more so than the KenPom’s or Sagarin. He later gives this key paragraph:

“It is important to note the intended purpose of each given ranking.  There is a substantial difference between results-based metrics and predictive metrics.  KPI is intended to “rank resumes” in order to determine the most qualified field of 68 teams. There is no human interference in the math, just a multi-step algorithm ranking performance.  Other ranking systems may have different definitions (may focus more on offensive/defensive efficiency for example) and may also have different intents (intended to rank who will win, not how a team should be seeded in the NCAA Tournament).  There is nothing wrong with that.  There are a lot of people doing some fantastic work to try and evaluate teams.”

Let’s break it down sentence-by-sentence:

“It is important to note the intended purpose of each given ranking.”

100% agreed.

“There is a substantial difference between results-based metrics and predictive metrics.”

Very much so, but I’d prefer to use the distinction “Predictive vs. Descriptive” metrics.

” KPI is intended to “rank resumes” in order to determine the most qualified field of 68 teams.”

No. KPI is intended to model the committee’s ranking of resumes. That is quite different than ranking resumes in and of itself. Being an expert bracketologist does not make one an expert at ranking resumes. It makes on an expert and accurately predicting what decisions the committee will make make when choosing between various resumes. It’s a very important distinction and one I will come back to later when we discuss aggregation.

“There is no human interference in the math, just a multi-step algorithm ranking performance.”

OK. I’m all for that. WAB, the KenPoms, and Sagarin’s all all the same in that respect.

“Other ranking systems may have different definitions (may focus more on offensive/defensive efficiency for example) and may also have different intents (intended to rank who will win, not how a team should be seeded in the NCAA Tournament).”

The intents are the key here. Sagarin predictor and the KenPom ratings are indeed predictive of team strength. I’ll have the WAB numbers for both at the end of this post.

“There is nothing wrong with that.  There are a lot of people doing some fantastic work to try and evaluate teams.”

Correct. Jeff Sagarin and Ken Pomeroy work to try and determine which teams are best. I try to determine which teams have the best resumes, given that data. Both predictive and descriptive metrics have their value, but combining them in an aggregation tells us nothing.

The committee is going to have to determine what criteria they want to use for admission into the dance, and what criteria they want to use for seeding. If they are looking for the 34 best at large teams, using the Sagarin & KenPom ratings is highly advisable. If they are looking for the 34 most deserving teams, I’d recommend they take a look at the respective WAB’s. In neither case would I recommend using the KPI.

As for best or most deserving, I’d 100% recommend inviting based off of resume. Seeding is trickier, because if you don’t seed based on team quality, you have highly unequal regions. It’s a conundrum, but I’m getting ahead of myself. There are concerns about  1 point win being effectively the same as a one point loss:

I’m not sure what exactly the committee wants, so I can’t determine how to solve for it. If they want an aggregate of team quality, KenPom, Sagarin, Massey, and a few others would do nicely. If they want resume, WAB’s (or percentiles) based off of KenPom, Sagarin, etc. would work. If they want a hybrid, then it gets very, very tricky. Aggregating predictive and descriptive metrics doesn’t give you a discrete answer to any specific question. The absolute best thing I can think of if your Pythag via the power rating being combined with your implied Pythag from your resume.

I’ll discuss this further next week. For now, here are the KenPom and Sagarin based WAB’s:

Rank School KP WAB Sag WAB
1 Villanova 4.83 4.76
2 Baylor 4.35 4.35
3 Kansas 4.01 3.96
4 Creighton 3.95 3.8
5 Gonzaga 3.88 3.87
6 UCLA 3.63 3.64
7 Florida St. 3.51 3.34
8 Butler 3.25 3.19
9 Notre Dame 3.2 3.11
10 Louisville 3.18 3.17
11 Arizona 3 2.9
12 Kentucky 2.97 2.98
13 North Carolina 2.73 2.66
14 Florida 2.67 2.61
15 Wisconsin 2.53 2.74
16 Cincinnati 2.4 2.33
17 Minnesota 2.35 2.2
18 West Virginia 2.32 2.16
19 Xavier 2.29 2.18
20 Oregon 2.22 2.27
21 Virginia 2.17 2.04
22 Duke 1.97 1.82
23 Saint Mary’s 1.95 1.83
24 Maryland 1.89 1.9
25 Purdue 1.4 1.37
26 South Carolina 1.36 1.29
27 UNC Wilmington 1.29 1.05
28 TCU 1.26 1.29
29 USC 1.26 1.13
30 Virginia Tech 1.08 1.15
31 Texas Tech 1.06 0.97
32 Northwestern 1.02 0.91
33 Iowa St. 0.84 0.83
34 VCU 0.73 0.39
35 Middle Tennessee 0.72 0.47
36 Seton Hall 0.66 0.62
37 Clemson 0.62 0.53
38 SMU 0.59 0.52
39 Pittsburgh 0.59 0.47
40 Michigan St. 0.59 0.44
41 Wichita St. 0.55 0.56
42 Nevada 0.47 0.37
43 Arkansas 0.39 0.3
44 Kansas St. 0.36 0.33
45 Dayton 0.09 -0.14
46 Illinois 0 0.04
47 Marquette -0.13 -0.17
48 New Mexico St. -0.14 -0.18
49 UT Arlington -0.14 -0.33
50 Akron -0.16 -0.31
51 Valparaiso -0.18 -0.19
52 Oklahoma St. -0.19 -0.06
53 Oakland -0.39 -0.45
54 Georgia -0.42 -0.53
55 Wake Forest -0.43 -0.76
56 Houston -0.45 -0.55
57 California -0.46 -0.59
58 Miami FL -0.46 -0.39
59 Boise St. -0.57 -0.72
60 North Carolina St. -0.57 -0.74
61 Utah -0.61 -0.7
62 College of Charleston -0.64 -0.85
63 Illinois St. -0.73 -0.82
64 Chattanooga -0.77 -0.97
65 Florida Gulf Coast -0.87 -1.19
66 Indiana -0.89 -0.99
67 UCF -0.93 -1.05
68 Belmont -0.96 -1.07
69 Fort Wayne -0.96 -0.98
70 East Tennessee St. -1.01 -1.2
71 Michigan -1.02 -1.14
72 Monmouth -1.03 -1.19
73 Vermont -1.04 -1.29
74 Nebraska -1.05 -1.13
75 La Salle -1.06 -1.22
76 Georgia Tech -1.09 -1.16
77 Auburn -1.12 -1.1
78 Mississippi -1.13 -1.25
79 Ohio -1.17 -1.33
80 Winthrop -1.29 -1.47
81 Louisiana Lafayette -1.34 -1.59
82 Tennessee St. -1.38 -1.5
83 Mississippi St. -1.48 -1.56
84 Northeastern -1.48 -1.68
85 Memphis -1.51 -1.63
86 BYU -1.53 -1.71
87 Rhode Island -1.55 -1.64
88 Providence -1.59 -1.7
89 Texas A&M -1.61 -1.78
90 Iowa -1.7 -1.63
91 UNC Greensboro -1.71 -1.99
92 Ohio St. -1.82 -1.95
93 George Mason -1.83 -1.86
94 Temple -1.84 -1.97
95 Tennessee -1.85 -2.03
96 Yale -1.93 -1.97
97 Arkansas St. -1.97 -2.26
98 UNC Asheville -1.98 -2.26
99 Alabama -2.02 -2.11
100 UMBC -2.06 -2.23
101 LSU -2.1 -2.15
102 Bucknell -2.11 -2.39
103 Stanford -2.12 -2.31
104 Colorado -2.13 -2.14
105 North Dakota St. -2.14 -2.29
106 Georgetown -2.14 -2.19
107 Penn St. -2.25 -2.24
108 Old Dominion -2.29 -2.49
109 Wyoming -2.37 -2.46
110 Texas Southern -2.45 -2.89
111 Canisius -2.5 -2.77
112 Massachusetts -2.53 -2.68
113 Vanderbilt -2.55 -2.83
114 Marshall -2.55 -2.85
115 Harvard -2.55 -2.64
116 Rutgers -2.62 -2.68
117 Syracuse -2.63 -2.81
118 Saint Joseph’s -2.68 -2.84
119 Rider -2.69 -2.83
120 St. Bonaventure -2.73 -3
121 Richmond -2.78 -2.97
122 Eastern Washington -2.78 -2.83
123 UAB -2.8 -3
124 Loyola Chicago -2.81 -2.87
125 Penn -2.82 -2.95
126 Eastern Michigan -2.83 -2.96
127 Colorado St. -2.84 -2.93
128 Samford -2.88 -3.12
129 William & Mary -2.88 -3.09
130 Green Bay -2.91 -3.13
131 Princeton -2.96 -3.13
132 Georgia Southern -3.01 -3.3
133 Arizona St. -3.05 -3.12
134 Cal St. Bakersfield -3.07 -3.2
135 Davidson -3.09 -3.25
136 New Hampshire -3.11 -3.23
137 Rice -3.12 -3.38
138 Tulsa -3.14 -3.15
139 Sam Houston St. -3.19 -3.37
140 San Francisco -3.2 -3.43
141 Grand Canyon -3.25 -3.33
142 Louisiana Tech -3.28 -3.51
143 Portland St. -3.34 -3.44
144 Missouri St. -3.35 -3.46
145 San Diego St. -3.37 -3.45
146 Lehigh -3.38 -3.61
147 Iona -3.52 -3.68
148 George Washington -3.52 -3.62
149 Oklahoma -3.52 -3.64
150 Central Michigan -3.61 -3.79
151 Washington St. -3.66 -3.7
152 New Mexico -3.69 -3.88
153 Weber St. -3.69 -3.75
154 Fairfield -3.71 -3.91
155 Little Rock -3.72 -3.89
156 Furman -3.74 -4.03
157 Fresno St. -3.79 -3.83
158 North Carolina Central -3.79 -3.87
159 Georgia St. -3.8 -3.93
160 Portland -3.83 -4
161 Toledo -3.84 -4.13
162 Northern Kentucky -3.85 -3.95
163 New Orleans -3.86 -3.97
164 Washington -3.96 -4.04
165 Boston University -3.97 -4.11
166 Wright St. -4.02 -4.14
167 North Dakota -4.04 -4.18
168 Nebraska Omaha -4.06 -4.3
169 Texas -4.09 -4.12
170 Northern Illinois -4.1 -4.13
171 San Jose St. -4.15 -4.27
172 Boston College -4.18 -4.26
173 Houston Baptist -4.24 -4.31
174 Tennessee Martin -4.26 -4.43
175 Evansville -4.26 -4.5
176 Texas A&M Corpus Chris -4.27 -4.46
177 Nicholls St. -4.31 -4.55
178 Southern Illinois -4.32 -4.44
179 Ball St. -4.36 -4.39
180 Jacksonville St. -4.38 -4.65
181 Jacksonville -4.4 -4.53
182 Loyola Marymount -4.42 -4.59
183 UC Irvine -4.46 -4.62
184 South Dakota -4.48 -4.68
185 St. John’s -4.5 -4.56
186 Elon -4.51 -4.66
187 Radford -4.53 -4.77
188 Connecticut -4.56 -4.5
189 Brown -4.57 -4.66
190 LIU Brooklyn -4.6 -4.82
191 Denver -4.62 -4.76
192 UNLV -4.63 -4.64
193 Lamar -4.63 -4.69
194 Kent St. -4.68 -4.78
195 Texas St. -4.74 -4.88
196 Columbia -4.75 -4.84
197 Utah St. -4.75 -4.78
198 Buffalo -4.8 -4.97
199 Duquesne -4.86 -4.98
200 Loyola MD -4.88 -4.99
201 Gardner Webb -4.96 -5.29
202 East Carolina -5.02 -5.22
203 Holy Cross -5.18 -5.4
204 Campbell -5.26 -5.39
205 DePaul -5.3 -5.39
206 USC Upstate -5.33 -5.52
207 Saint Peter’s -5.34 -5.49
208 South Carolina St. -5.39 -5.61
209 Charlotte -5.4 -5.58
210 Northern Colorado -5.41 -5.58
211 UC Davis -5.42 -5.56
212 Albany -5.44 -5.63
213 Montana -5.46 -5.55
214 Troy -5.5 -5.63
215 Towson -5.51 -5.85
216 Southeastern Louisiana -5.51 -5.67
217 Stony Brook -5.52 -5.75
218 Coastal Carolina -5.54 -5.82
219 Appalachian St. -5.61 -5.75
220 Air Force -5.65 -5.73
221 Northwestern St. -5.66 -5.69
222 Utah Valley -5.68 -5.75
223 San Diego -5.68 -5.8
224 Western Michigan -5.74 -5.81
225 Santa Clara -5.78 -5.88
226 Cal St. Northridge -5.79 -5.82
227 Mercer -5.85 -6.08
228 UMKC -5.86 -5.96
229 Western Kentucky -5.89 -5.96
230 Hofstra -5.9 -6.16
231 Miami OH -5.96 -5.99
232 South Alabama -5.98 -6.17
233 Bowling Green -5.99 -6.17
234 Murray St. -6 -6.21
235 Liberty -6.01 -6.19
236 Lipscomb -6.01 -6.07
237 Stephen F. Austin -6.04 -6.16
238 Northern Iowa -6.05 -6.04
239 Illinois Chicago -6.07 -6.11
240 The Citadel -6.13 -6.51
241 Bradley -6.18 -6.28
242 Pacific -6.21 -6.24
243 Eastern Illinois -6.22 -6.26
244 South Florida -6.38 -6.51
245 Long Beach St. -6.38 -6.36
246 Mount St. Mary’s -6.4 -6.58
247 Seattle -6.4 -6.42
248 Fairleigh Dickinson -6.44 -6.56
249 Hawaii -6.52 -6.61
250 Cal St. Fullerton -6.54 -6.57
251 Binghamton -6.56 -6.76
252 UTSA -6.66 -6.72
253 NJIT -6.67 -6.81
254 McNeese St. -6.68 -6.82
255 IUPUI -6.74 -6.92
256 Louisiana Monroe -6.75 -6.98
257 Indiana St. -6.79 -6.79
258 Wagner -6.91 -7.01
259 Longwood -6.92 -7.11
260 Fordham -6.97 -7.13
261 Missouri -6.97 -7.11
262 Alcorn St. -6.99 -7.13
263 Wofford -7.03 -7.32
264 Western Illinois -7.03 -7.14
265 Jackson St. -7.05 -7.2
266 Youngstown St. -7.05 -7.18
267 Army -7.09 -7.29
268 Idaho -7.11 -7.23
269 Abilene Christian -7.17 -7.29
270 Siena -7.18 -7.54
271 St. Francis PA -7.28 -7.37
272 High Point -7.35 -7.61
273 Lafayette -7.42 -7.59
274 Western Carolina -7.42 -7.71
275 VMI -7.43 -7.65
276 Eastern Kentucky -7.43 -7.61
277 Morehead St. -7.46 -7.56
278 Incarnate Word -7.46 -7.6
279 North Texas -7.46 -7.72
280 Navy -7.54 -7.89
281 Grambling St. -7.59 -7.65
282 Hampton -7.59 -7.83
283 Manhattan -7.6 -7.77
284 Cornell -7.62 -7.72
285 UMass Lowell -7.63 -7.69
286 Delaware -7.68 -7.93
287 Morgan St. -7.68 -7.75
288 Oral Roberts -7.77 -7.86
289 North Florida -7.81 -7.89
290 Florida Atlantic -7.86 -7.93
291 Charleston Southern -7.88 -8.05
292 Quinnipiac -7.94 -8.2
293 Cleveland St. -7.95 -8.05
294 Delaware St. -7.95 -8.14
295 Robert Morris -7.96 -8.22
296 Savannah St. -7.98 -7.98
297 Cal Poly -7.99 -8.04
298 Drexel -8 -8.28
299 Southern -8.08 -8.3
300 UC Riverside -8.14 -8.22
301 UT Rio Grande Valley -8.16 -8.27
302 Norfolk St. -8.16 -8.29
303 Central Arkansas -8.22 -8.39
304 American -8.34 -8.56
305 Saint Louis -8.37 -8.48
306 Bryant -8.38 -8.54
307 Tulane -8.41 -8.42
308 Southern Miss -8.42 -8.52
309 South Dakota St. -8.43 -8.52
310 Stetson -8.44 -8.53
311 Pepperdine -8.48 -8.67
312 Maryland Eastern Shore -8.5 -8.56
313 UC Santa Barbara -8.51 -8.62
314 Sacred Heart -8.53 -8.7
315 Dartmouth -8.54 -8.77
316 Howard -8.59 -8.63
317 Marist -8.64 -8.81
318 Milwaukee -8.75 -8.9
319 Sacramento St. -8.78 -8.89
320 Prairie View A&M -8.83 -9.08
321 Kennesaw St. -8.9 -9.09
322 Southeast Missouri St. -8.98 -9.06
323 Idaho St. -9 -9.06
324 Tennessee Tech -9.1 -9.27
325 Drake -9.15 -9.12
326 Chicago St. -9.22 -9.31
327 Maine -9.25 -9.52
328 Mississippi Valley St. -9.34 -9.28
329 Austin Peay -9.35 -9.37
330 Hartford -9.4 -9.57
331 Presbyterian -9.46 -9.75
332 Montana St. -9.53 -9.79
333 Niagara -9.68 -9.89
334 Oregon St. -9.91 -9.88
335 Florida A&M -9.92 -10.04
336 James Madison -10.04 -10.45
337 SIU Edwardsville -10.13 -10.11
338 Detroit -10.17 -10.39
339 Arkansas Pine Bluff -10.28 -10.35
340 Alabama St. -10.32 -10.49
341 UTEP -10.57 -10.67
342 FIU -10.68 -10.94
343 Coppin St. -10.7 -10.82
344 Southern Utah -10.78 -10.91
345 Bethune Cookman -10.85 -11
346 Colgate -10.92 -11
347 St. Francis NY -10.97 -11.14
348 Northern Arizona -11.02 -11.07
349 Central Connecticut -11.62 -11.74
350 Alabama A&M -11.94 -12.02
351 North Carolina A&T -12.42 -12.56

 

 

 

 

 

 

Advertisements
1 comment
  1. David Hess said:

    I haven’t thought this through, but what happens if you calculate a version of WAB that uses “win shares” instead of “wins”, where a 10 (or 7 or 5 or whatever) point win is a full share, and a 1 point win is, say, 0.6 shares (with the losing team getting whatever the winning team doesn’t)?

    I guess that would end up being screwy, because a narrow win over a terrible team would be negative, whereas presumably all wins should be non-negative.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: