1/9 WAB: Resume != Power Rating

Rob Dauster came out with an interesting column this week were looked into the concept of what do the polls mean:

“But you’ll have a hard time convincing me that every person that slots Baylor at the top of their rankings will believe that Baylor is the best team in the country, better than all of Kansas, Kentucky, Duke, UCLA, North Carolina, West Virginia, Louisville, Gonzaga, etc.

Which brings us back to an issue that will always be argued about: What is the criteria for ranking teams? Their body of work, or who you actually believe to be the better team?”

Now, I’m not an AP Poll voter, and I have no idea what the instructions are. If I were to rank the top 2 teams in the country, I’d go with Kentucky and West Virginia, although I’d have to dig deeper to be certain that the latter is better than Villanova. North Carolina, Kansas, Duke, and Virginia would all be in my top 10. UCLA wouldn’t be anywhere near the top 5, or even in the top 10.

Rob believes it is the work of Gary Parrish (of Poll Attacks fame) who has pushed voters towards ranking on resume instead of overall team quality. Rob laments this:

“My take: since these rankings, these polls, mean absolutely nothing beyond the little number you see next to their name on TV, what’s the point of ranking based on the body of work? That’s what Bracketology is for. That’s what those seeding projections are for. That’s what the computer rankings of sites like KenPom and KPI and Sagarin do.”

Oh dear. Let’s take that from the top. He’s correct that the polls themselves have no weight or power. They don’t matter in terms of Selection Sunday. Whether or not they should measure resume or power rating is an open question, but I agree that ultimately they don’t matter one way or the other. He then suggests that Bracketology provides a good judge of the body of work. I’d agree, with a caveat: Bracketology is trying to predict the committee. They aren’t trying to grade the best resumes, but rather, predict how the committee will grade the top resumes. As such, it behooves them to accurately gauge any committee biases.

He then follows up “That’s what those seeding projections are for.”, which is of course, correct, with the same caveat that applied to Bracketology. However, here’s where he stumbles a bit:

“That’s what the computer rankings of sites like KenPom and KPI and Sagarin do.”

Oh boy. There’s a lot to unpack in this sentence, so let’s start with KenPom. The KenPom ratings are designed to be predictive. He has the tools to grade resumes, but that isn’t what the site is about, and his ratings are power ratings, not resume ratings. He has Kentucky on top, but that doesn’t mean he grades Kentucky’s resume as superior to Baylor’s.

I’m going to jump ahead to Sagarin. I’ll come back to KPI in a second. The Sagarin ratings are pretty clearly power ratings. Predictor is obviously so, but I don’t think Saragin would suggest that Baylor has the 10th best resume in the country. He used to have Elo ratings, which were resume ratings, albeit of an unusual sort in college basketball. Those are no longer available on the page. In any event, neither KenPom, nor Sagarin are computing resume ratings (or rankings).

I am computing resume ratings (based off the KenPom power ratings). I could do so with the Sagarin predictor ratings, or the Massey power ratings, or any other published power rating. My methodology is reasonably simple. I run an average bubble team through each schedule and solve for the expected amount of wins. In the case of Baylor, an average bubble squad would expect to have roughly 10.76 wins. Baylor has 15, so they have a WAB of 4.24.

Percentile is a bit trickier. In this case I am solving for how likely it is for a bubble team to pull off your record. In Baylor’s case, a bubble squad would expect to be 15-0 0.26% of the time. In Gonzaga’s case, 0.74% of bubble squads would be undefeated. In North Carolina’s case a bubble squad would expect to be 15-2 or better 3.14% of the time, and 14-3 9.45% of the time. Their percentile is then 100% – 3.14% – 9.45%/2. Both WAB and Bubble Percentile judge resume, but in different fashions, as WAB is a counting stat, and Bubble Percentile is a rate stat.

KPI is doing something a bit different. KPI is trying to hardwire the committee. As with Bracketology, KPI wants to accurately include any committee biases. If the committee is undervaluing home court advantage, or overrating the weaker major conference schools, KPI wants to do the same. Let me give a hypothetical example to illustrate how this works:

UNC Wilmington plays a road game at College of Charleston. Michigan St hosts Minnesota. According to the KenPoms (and my rating of a bubble team), a bubble team should beat Minnesota at home 59.88% of the time, and win at CoC 57.62% of the time. In other words, if UNC Wilmington wins at CoC, their WAB will improve .4238. If they lose, it will drop .5762. Similarly, if Michigan St. wins, their WAB will jump .4012. If they lose, it will drop .5988. This is all reasonably simple and straightforward. Anyone can compute this so long as they have some approximation of home court advantage and a control team (aka, the average bubble squad). Now, the KPI works a bit differently.

KPI assigns a value to each win, and a value to each loss. However, the values for each individual game don’t sum to 1, nor do they dovetail the expected win %. A couple of years ago I took a deep dive into the KPI ratings because I found them to be fairly odd. Specifically I looked at Villanova and Kansas. At the time Kansas was 22-7, and had a KPI grade of 9.99. Villanova was 26-3 and had a KPI grade of 9.04. Setting WAB to generate 19.03 wins over those games I generated this:

Kansas KPI Net WAB Net Villanova KPI Net WAB Net
Game 1 0.85 0.8375   Game 1 0.89 0.8064
Game 2 0.77 0.7542   Game 2 0.69 0.7665
Game 3 0.76 0.7269   Game 3 0.63 0.6957
Game 4 0.73 0.6974   Game 4 0.61 0.6478
Game 5 0.7 0.6337   Game 5 0.56 0.6375
Game 6 0.56 0.617   Game 6 0.54 0.6281
Game 7 0.56 0.5829   Game 7 0.53 0.6146
Game 8 0.56 0.581   Game 8 0.48 0.5456
Game 9 0.49 0.5448   Game 9 0.45 0.5286
Game 10 0.49 0.5476   Game 10 0.44 0.5436
Game 11 0.47 0.5412   Game 11 0.43 0.4914
Game 12 0.45 0.5284   Game 12 0.36 0.4558
Game 13 0.44 0.47   Game 13 0.35 0.4092
Game 14 0.41 0.4372   Game 14 0.34 0.3848
Game 15 0.41 0.4189   Game 15 0.26 0.3385
Game 16 0.33 0.3926   Game 16 0.25 0.3263
Game 17 0.3 0.3204   Game 17 0.23 0.3044
Game 18 0.28 0.305   Game 18 0.2 0.2977
Game 19 0.28 0.2712   Game 19 0.15 0.2712
Game 20 0.13 0.2072   Game 20 0.13 0.2667
Game 21 0.08 0.1207   Game 21 0.13 0.242
Game 22 0.06 0.0868   Game 22 0.11 0.2118
Game 23 0 -0.1286   Game 23 0.1 0.147
Game 24 0 -0.1954   Game 24 0.1 0.1335
Game 25 -0.01 -0.2329   Game 25 0.07 0.1327
Game 26 -0.02 -0.305   Game 26 0.02 0.0914
Game 27 -0.03 -0.3191   Game 27 0 -0.1374
Game 28 -0.03 -0.3881   Game 28 0 -0.1717
Game 29 -0.03 -0.4002   Game 29 -0.01 -0.2327
           
  9.99 8.6531   9.04 10.3769

The first thing that should jump out at you is how little the losses hurt. Kansas played a game at Colorado that an a bubble team should win 40% of the time. They lost, but only dropped .03 in KPI. There are other games where the loss was less than a 0.005 drop. The way KPI works, you can schedule your way to the top via games that are effectively freerolls. Now remember, KPI is designed to mimic the committee, so the underlying thesis here is that the committee thinks about resumes in terms of good wins and bad losses. In fact, KPI is effectively a much more sophisticated version of Joe Lunardi’s winning points.

Apologies for the article being gated. Vegas watch had an excellent breakdown of it. It’s flaws were roasted on Twitter:

https://twitter.com/BPredict/status/426438295183429633

The thing is, Joe Lunardi is trying to predict the committee. So is the KPI. The committee leans on the RPI quite heavily, despite the fact that the RPI is 75% a measure of strength of schedule. Long story short, it is more important to face good teams than to beat them. Beating them is better than losing, but ultimately you are rewarded for your strength of schedule. That is why Joe Lunardi’s methodology was such garbage, and the KPI is no better. Nietzsche said it best:

https://s-media-cache-ak0.pinimg.com/736x/82/31/44/823144e15e1cbe13e3fbbecc9805400d.jpg

Coming back to Rob’s column, let’s look again at this quote regarding measuring resumes:

“That’s what the computer rankings of sites like KenPom and KPI and Sagarin do.”

No, sort of, and no. KenPom and Sagarin produce predictive ratings. KPI is Winning Points dressed up in a little more statistical sophistication. Kevin Puaga (the creator of the KPI) is basically sitting in Plato’s cave, trying to predict the committee, so ultimately the issue lies with the people selecting the tournament.

It was the lack of good objective resume ratings that lead me to create WAB. You can generate WAB for any set of ratings and results. If for whatever reason you didn’t link the KenPom ratings, and wanted to use Massey instead, you could generate the Massey WAB ratings (and I’ve done so). Should the polls measure resume? Or should they be based on who would win on a neutral court? I have no idea. The polls aren’t my concern, but I will ask this. When Rob has UCLA 3rd and West Virginia 11th, is that a resume rating, or his view on which team is better? Does he think Kentucky has the second best resume? My guess is that this is his version of power ratings. If so, I think UCLA is going to eventually break his heart.

Rank School WAB Bubble Percentile
1 Baylor 4.34 99.89%
2 Villanova 4.19 99.73%
3 Gonzaga 3.57 99.67%
4 Creighton 3.41 98.50%
5 Butler 3.42 97.86%
6 Kansas 3.39 98.73%
7 UCLA 3.17 98.92%
8 Louisville 2.81 95.83%
9 Florida St. 2.81 97.66%
10 Arizona 2.65 95.18%
11 Minnesota 2.63 95.37%
12 Notre Dame 2.52 95.50%
13 Kentucky 2.38 94.69%
14 Florida 2.25 91.79%
15 North Carolina 2.25 92.17%
16 Xavier 2.19 92.34%
17 Wisconsin 2.13 92.26%
18 Oregon 2.15 91.35%
19 Duke 2.04 91.51%
20 Virginia 1.97 89.28%
21 Purdue 1.86 88.03%
22 Cincinnati 1.80 89.89%
23 Saint Mary’s 1.77 89.64%
24 West Virginia 1.56 88.06%
25 USC 1.55 84.45%
26 Arkansas 1.31 80.90%
27 Maryland 1.34 81.15%
28 UNC Wilmington 1.15 77.17%
29 Clemson 1.18 76.22%
30 Seton Hall 0.83 70.86%
31 South Carolina 0.81 68.53%
32 Virginia Tech 0.76 69.58%
33 SMU 0.73 66.59%
34 Dayton 0.72 66.27%
35 TCU 0.70 68.37%
36 Northwestern 0.66 65.16%
37 Kansas St. 0.60 65.97%
38 Pittsburgh 0.56 62.49%
39 Middle Tennessee 0.46 59.80%
40 VCU 0.47 60.25%
41 Nevada 0.38 57.66%
42 Texas Tech 0.39 60.99%
43 Wichita St. 0.36 58.07%
44 Oklahoma St. 0.14 53.75%
45 Michigan St. 0.14 53.16%
46 Iowa St. 0.12 52.83%
47 North Carolina St. 0.04 49.96%
48 Miami FL -0.03 47.78%
49 New Mexico St. -0.15 42.69%
50 Wake Forest -0.23 43.89%
51 Valparaiso -0.26 41.31%
52 UT Arlington -0.33 40.01%
53 East Tennessee St. -0.34 38.53%
54 Akron -0.44 33.65%
55 Illinois -0.44 38.38%
56 Oakland -0.59 31.99%
57 Indiana -0.56 34.59%
58 Mississippi -0.61 33.71%
59 Houston -0.63 32.94%
60 Marquette -0.64 33.35%
61 Boise St. -0.70 30.82%
62 Rhode Island -0.71 31.19%
63 California -0.68 32.37%
64 Michigan -0.71 31.89%
65 Utah -0.83 25.35%
66 College of Charleston -0.84 27.27%
67 Chattanooga -0.88 25.30%
68 Providence -0.89 28.40%
69 Georgia -0.93 26.85%
70 Memphis -0.94 25.90%
71 Fort Wayne -1.03 21.73%
72 UCF -1.06 21.98%
73 Florida Gulf Coast -1.10 21.89%
74 Monmouth -1.10 22.35%
75 Illinois St. -1.08 22.90%
76 Arkansas St. -1.14 22.82%
77 Belmont -1.17 18.05%
78 Nebraska -1.16 24.07%
79 Vermont -1.26 19.58%
80 Bucknell -1.37 18.02%
81 Auburn -1.34 18.96%
82 Winthrop -1.39 12.99%
83 Louisiana Lafayette -1.43 14.29%
84 Tennessee -1.44 16.94%
85 Temple -1.45 18.42%
86 Ohio -1.51 10.32%
87 Tennessee St. -1.57 13.19%
88 Alabama -1.60 13.76%
89 Northeastern -1.71 13.60%
90 Georgia Tech -1.66 12.75%
91 BYU -1.72 12.64%
92 Ohio St. -1.72 12.07%
93 Colorado -1.78 12.70%
94 LSU -1.76 12.18%
95 La Salle -1.83 9.58%
96 Texas A&M -1.81 9.57%
97 Saint Joseph’s -1.89 11.26%
98 Yale -2.04 6.05%
99 Samford -2.10 5.60%
100 UNC Asheville -2.13 7.89%
101 Rutgers -2.14 6.77%
102 Marshall -2.14 8.10%
103 Mississippi St. -2.09 6.06%
104 Davidson -2.16 6.30%
105 St. Bonaventure -2.18 6.80%
106 UMBC -2.26 3.56%
107 William & Mary -2.28 4.17%
108 Old Dominion -2.26 7.61%
109 Northern Kentucky -2.28 4.46%
110 George Mason -2.27 6.57%
111 North Dakota St. -2.31 4.78%
112 Vanderbilt -2.30 8.31%
113 Penn St. -2.31 8.23%
114 UNC Greensboro -2.33 4.65%
115 Iowa -2.34 6.65%
116 Stanford -2.47 6.50%
117 Texas Southern -2.62 5.55%
118 Wyoming -2.52 4.04%
119 Loyola Chicago -2.54 4.38%
120 Georgetown -2.53 6.81%
121 Canisius -2.64 4.26%
122 Grand Canyon -2.63 1.99%
123 Syracuse -2.57 5.07%
124 Rider -2.71 3.13%
125 Harvard -2.71 2.47%
126 North Carolina Central -2.75 2.37%
127 Fairfield -2.79 2.76%
128 Princeton -2.82 3.00%
129 Eastern Washington -2.82 2.39%
130 Penn -2.90 1.85%
131 Georgia St. -2.93 1.90%
132 New Mexico -2.91 3.22%
133 Central Michigan -2.95 1.84%
134 New Orleans -2.97 1.83%
135 Colorado St. -2.93 2.71%
136 Massachusetts -2.96 2.96%
137 Richmond -2.96 3.00%
138 Fresno St. -3.00 1.94%
139 UAB -3.02 2.34%
140 Little Rock -2.99 1.07%
141 San Francisco -2.98 2.29%
142 Georgia Southern -3.07 1.80%
143 Green Bay -3.08 1.33%
144 Toledo -3.10 2.08%
145 Eastern Michigan -3.13 1.30%
146 Washington St. -3.10 1.39%
147 Cal St. Bakersfield -3.19 0.71%
148 Rice -3.20 1.47%
149 Arizona St. -3.21 2.58%
150 Oklahoma -3.23 1.44%
151 George Washington -3.26 2.34%
152 Sam Houston St. -3.31 0.66%
153 New Hampshire -3.32 0.62%
154 Wright St. -3.33 1.59%
155 Houston Baptist -3.37 0.74%
156 Portland St. -3.38 0.56%
157 Texas A&M Corpus Chris -3.42 0.35%
158 Nicholls St. -3.52 0.81%
159 Tulsa -3.50 1.18%
160 Louisiana Tech -3.54 0.22%
161 Missouri St. -3.54 0.96%
162 San Diego St. -3.56 0.86%
163 Portland -3.54 0.65%
164 Texas -3.56 1.40%
165 Iona -3.63 1.28%
166 Southern Illinois -3.67 0.52%
167 San Jose St. -3.67 0.38%
168 Radford -3.70 0.56%
169 Washington -3.65 0.48%
170 Evansville -3.72 0.59%
171 South Dakota -3.76 0.89%
172 Weber St. -3.80 0.44%
173 Lehigh -3.83 0.69%
174 Furman -3.86 0.62%
175 Kent St. -3.84 0.37%
176 Loyola MD -4.04 0.30%
177 Buffalo -4.10 0.34%
178 Utah St. -4.06 0.37%
179 Boston University -4.14 0.37%
180 North Dakota -4.19 0.12%
181 Gardner Webb -4.23 0.19%
182 St. John’s -4.18 0.55%
183 Northern Illinois -4.30 0.06%
184 Tennessee Martin -4.37 0.28%
185 Ball St. -4.39 0.05%
186 Jacksonville St. -4.49 0.31%
187 Loyola Marymount -4.39 0.20%
188 Nebraska Omaha -4.50 0.35%
189 USC Upstate -4.49 0.11%
190 Brown -4.51 0.02%
191 East Carolina -4.51 0.12%
192 Jacksonville -4.55 0.04%
193 Boston College -4.50 0.16%
194 Charlotte -4.58 0.13%
195 LIU Brooklyn -4.58 0.12%
196 UC Davis -4.58 0.13%
197 Albany -4.63 0.12%
198 Northern Colorado -4.64 0.05%
199 Denver -4.65 0.13%
200 UC Irvine -4.68 0.16%
201 Lamar -4.71 0.02%
202 Elon -4.71 0.14%
203 Utah Valley -4.80 0.07%
204 Columbia -4.77 0.02%
205 Texas St. -4.80 0.02%
206 Connecticut -4.81 0.21%
207 San Diego -4.87 0.03%
208 UNLV -5.01 0.08%
209 Miami OH -5.07 0.02%
210 Duquesne -5.07 0.02%
211 Northern Iowa -5.21 0.08%
212 Hofstra -5.21 0.04%
213 Bowling Green -5.25 0.01%
214 Western Kentucky -5.25 0.05%
215 South Alabama -5.27 0.01%
216 Holy Cross -5.31 0.06%
217 Eastern Illinois -5.33 0.02%
218 Montana -5.48 0.04%
219 Seattle -5.49 0.00%
220 Long Beach St. -5.54 0.03%
221 Saint Peter’s -5.47 0.01%
222 Campbell -5.46 0.00%
223 South Carolina St. -5.55 0.01%
224 Coastal Carolina -5.51 0.03%
225 South Florida -5.49 0.01%
226 Southeastern Louisiana -5.64 0.01%
227 Fairleigh Dickinson -5.65 0.01%
228 DePaul -5.64 0.01%
229 NJIT -5.70 0.01%
230 Appalachian St. -5.71 0.00%
231 Stony Brook -5.68 0.02%
232 Northwestern St. -5.75 0.00%
233 Troy -5.74 0.01%
234 Towson -5.72 0.04%
235 The Citadel -5.79 0.01%
236 Air Force -5.79 0.01%
237 Binghamton -5.86 0.00%
238 IUPUI -5.95 0.01%
239 Santa Clara -5.96 0.01%
240 Western Michigan -6.02 0.00%
241 Cal St. Northridge -6.04 0.00%
242 Savannah St. -6.12 0.00%
243 Wagner -6.09 0.00%
244 UMKC -6.09 0.00%
245 Indiana St. -6.09 0.02%
246 Lipscomb -6.13 0.00%
247 Jackson St. -6.14 0.00%
248 High Point -6.17 0.00%
249 Missouri -6.12 0.00%
250 Stephen F. Austin -6.18 0.00%
251 Youngstown St. -6.22 0.00%
252 Army -6.22 0.00%
253 Liberty -6.22 0.00%
254 Murray St. -6.26 0.01%
255 Western Illinois -6.28 0.00%
256 Abilene Christian -6.30 0.00%
257 Mercer -6.29 0.01%
258 Illinois Chicago -6.29 0.00%
259 Bradley -6.35 0.01%
260 Idaho -6.37 0.00%
261 Alcorn St. -6.41 0.00%
262 Pacific -6.39 0.00%
263 Longwood -6.42 0.00%
264 Incarnate Word -6.50 0.00%
265 Quinnipiac -6.48 0.00%
266 Western Carolina -6.53 0.00%
267 Mount St. Mary’s -6.63 0.00%
268 Eastern Kentucky -6.60 0.00%
269 Lafayette -6.60 0.00%
270 UTSA -6.67 0.00%
271 McNeese St. -6.68 0.00%
272 Morehead St. -6.72 0.00%
273 Hawaii -6.65 0.00%
274 Cal St. Fullerton -6.70 0.00%
275 UMass Lowell -6.73 0.00%
276 Louisiana Monroe -6.82 0.00%
277 Florida Atlantic -6.92 0.00%
278 Cleveland St. -7.04 0.00%
279 Delaware -7.06 0.00%
280 Wofford -7.13 0.00%
281 Cal Poly -7.16 0.00%
282 Siena -7.21 0.00%
283 Robert Morris -7.23 0.00%
284 Southern -7.31 0.00%
285 UT Rio Grande Valley -7.32 0.00%
286 St. Francis PA -7.45 0.00%
287 Saint Louis -7.42 0.00%
288 Fordham -7.44 0.00%
289 Pepperdine -7.51 0.00%
290 VMI -7.52 0.00%
291 UC Santa Barbara -7.52 0.00%
292 North Texas -7.50 0.00%
293 Drexel -7.54 0.00%
294 Grambling St. -7.60 0.00%
295 Hampton -7.63 0.00%
296 Cornell -7.67 0.00%
297 South Dakota St. -7.66 0.00%
298 Stetson -7.68 0.00%
299 Manhattan -7.72 0.00%
300 Morgan St. -7.77 0.00%
301 American -7.80 0.00%
302 Maryland Eastern Shore -7.89 0.00%
303 North Florida -7.87 0.00%
304 Navy -7.83 0.00%
305 Southern Miss -7.87 0.00%
306 Oral Roberts -7.99 0.00%
307 Charleston Southern -8.05 0.00%
308 Idaho St. -8.10 0.00%
309 Kennesaw St. -8.18 0.00%
310 Norfolk St. -8.20 0.00%
311 Central Arkansas -8.40 0.00%
312 Delaware St. -8.42 0.00%
313 Tennessee Tech -8.40 0.00%
314 UC Riverside -8.42 0.00%
315 Bryant -8.46 0.00%
316 Marist -8.54 0.00%
317 Dartmouth -8.54 0.00%
318 Hartford -8.58 0.00%
319 Howard -8.63 0.00%
320 Sacred Heart -8.60 0.00%
321 Tulane -8.58 0.00%
322 Montana St. -8.73 0.00%
323 Niagara -8.81 0.00%
324 Presbyterian -8.83 0.00%
325 Sacramento St. -8.85 0.00%
326 Milwaukee -8.95 0.00%
327 Florida A&M -8.95 0.00%
328 Southeast Missouri St. -8.99 0.00%
329 Prairie View A&M -9.09 0.00%
330 SIU Edwardsville -9.18 0.00%
331 Drake -9.32 0.00%
332 Maine -9.42 0.00%
333 James Madison -9.40 0.00%
334 Alabama St. -9.48 0.00%
335 Mississippi Valley St. -9.53 0.00%
336 Austin Peay -9.52 0.00%
337 Chicago St. -9.58 0.00%
338 FIU -9.79 0.00%
339 Southern Utah -9.86 0.00%
340 Bethune Cookman -9.88 0.00%
341 Oregon St. -9.95 0.00%
342 St. Francis NY -10.01 0.00%
343 Arkansas Pine Bluff -10.26 0.00%
344 Detroit -10.31 0.00%
345 UTEP -10.72 0.00%
346 Coppin St. -10.84 0.00%
347 Central Connecticut -10.82 0.00%
348 Alabama A&M -11.04 0.00%
349 Colgate -11.17 0.00%
350 Northern Arizona -11.16 0.00%
351 North Carolina A&T -11.61 0.00%

 

Advertisements
1 comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: