In Defense of Statistics

by Bleed Blue 'n White on Oct 20, 2008 8:47 PM EDT Comment 40 comments

This is the first in (hopefully) a series of posts to try to explain the way that the statistical rankings work in the BCS.

The first thing I'm going to say is this: Statistical rankings are easy to understand. Very easy. There's no magic about them, and with a few crackpot exceptions (Billingsley) they're all basically doing the same thing.

The basic idea is this: first, assume all teams have an intrinsic "true strength" - one number that determines how likely team A is to beat team B. That "true strength" is how the teams are ranked. Note that this means that we're assuming that matchups can't play a significant role - that is, you can't have a situation where Penn State is more likely to beat Wisconsin, who is more likely to beat Ohio State, who is more likely to beat Penn State. This may seem like a big assumption, but first off, it's necessary, and second, you can do some tests to determine if it's likely that loops like that are real, and they don't seem to exist in football. At least, not significantly.

OK. Now, with that rating, you need a "game output function." You need some function that gives you the likelihood that team A will beat team B. Whatever function is chosen, it's going to look something like this, where the X-axis is something like "A-B", where A is the true strength of team A, etc.

Now, armed with a game-output function and the results of each game, you can form a rating. You do this by figuring out what ratings make the total likelihood of the entire season highest, formed by multiplying everything together.

Since the BCS is out for the first time this week, there are some interesting questions we can ask by looking at the computer rankings:

Why is Ohio State so high? They're ranked 5th, which is universally higher than USC, who beat them. Quite soundly.

Because Ohio State's only lost one game, and that was to a good team. USC lost one game, too, but that was to a much weaker team. Losses to weak teams pin a team much more than losses to good teams. This is where human polls and statistical polls differ: a human sees the pounding USC gave Ohio State and does not see it as a fluke, but sees the USC-Oregon State loss as a fluke. The statistical polls see *both* as an upset, but the USC-Ohio State loss as a much more likely upset.

Why is Penn State so low? In some cases (Wolfe's rankings) they're below Ball State!

Undefeated teams are a kindof "gotcha" for ranking systems. If a team's true strength rating is "infinite", then all of its wins will have a GOF of "1", and that maximizes the total probability. So really, the biggest difference between all of the ranking systems is how they treat undefeated teams. Most of the ranking use something called "maximum likelihood," which is basically what I described above. When they do that, they correct undefeated teams by using something called "Bayesian inference," which starts by assuming a likelihood that a team that good actually exists. You don't get an infinite ranking, because no team in college football is that good. So it's a combination of "how likely is it that they're this good?" with "how likely is it that they would've won all their games if they're this good?"

This ends up being a "floor" rating, typically - which means that frequently, undefeated teams will get ranked lower than defeated teams, simply because the prior distribution (the likelihood of a great team existing) is low.

An important opinion note here: I hate this method. Frankly, it's stupid: because you can't bias yourself and say "Penn State's more likely to have a great team than Ball State," you have to assume the prior distribution is the same for all teams. This is violently, and provably, wrong. Bayesian statistics is just not a good model for separating undefeated teams. The problem is that it probably looks good, on average. But it underrates "major" programs, and overrates "minor" programs, which is what's happening for Ball State.

I can't really justify Ball State's ranking at all, though. The wins we have, straight down the line, are better than all of Ball State's, by Wolfe's rankings (who has Ball State above Penn State). The only advantage Ball State might have is that they've got one more away game (thus converting a "godawful team" into a "might not be so bad" team). But even there that doesn't make a lot of sense, considering the opponent advantage we have is very large.

OK, so why is Penn State so much higher in Colley's rankings?

Because Colley uses a different method of finding the ranking that maximizes the likelihood for each teams. Basically, in the end, Colley's rankings end up putting unbeaten teams midway between the "worst" and "best" rankings. I like this method a lot more, since you don't have a preassumed prior distribution. Also, winning more games helps to pull you higher and higher. With a Bayesian approach, winning doesn't help much unless you win a better game than your "best" game, because the prior distribution is likely to be the one that's pinning you back.

Shouldn't the statistical rankings be allowed to use scoring margin?

No. Very much so, no. Penn State's 46-17 victory over Michigan is a resounding victory in terms of scoring margin, but I'm sure that a lot of people in the human polls looked at that victory and said "hmm, Penn State might have some weaknesses." If not, they should've. I wouldn't blame them. The point is that you play to win the game - period. That's the only fair way to measure how well a team played. A statistical model cannot include all the variables that go into a game, so rather than use a flawed model to get imperfect information, it's best to use a simple model to get perfect information.

Does this suck for Penn State? Yes, and no. Yes, in the sense that the statistical rankings don't help us. But if the human polls worked the way they're supposed to, Penn State should be fine - because the human polls shouldn't get worked up about the fact that PSU has a weak schedule. We beat the crap out of our opponents. That's all they need to know - the statistical rankings deal with the weakness of the opponent.

Thanks for the fanpost! Please do not post any content from a premium site that requires a subscription. Also, if you just want to share a link consider using fanshots instead. Thanks.

7 recs | Comment 40 comments

Story-email Email Printer Print

Comments

This is money

Keep doing these.

One comment about MoV, I read Sagarin say that including it does seem to make his results more ‘accurate’, but my biggest problem is that if the coaches know it counts they will act differently at the end of games then they otherwise would have, thus adding more problems to the data.

Kevin @ Black Shoe Diaries

by Kevin HD on Oct 20, 2008 11:24 PM EDT reply actions 0 recs

Sagarin’s comment is really misleading. It doesn’t make his results more “accurate” at all. It makes them more predictively accurate – that is, if you want to know “who will beat who,” it’s more accurate.

That’s not what you want with a rating system used to determine a champion. In that case, you want a descriptive accuracy (also called retrodictive accuracy). Why? Because you’re trying to rate the season.

by Bleed Blue 'n White on Oct 21, 2008 1:16 AM EDT up reply actions 0 recs

Awesome post

If you plan on writing more of these, I can’t wait to read them

by borisborisboris on Oct 20, 2008 11:29 PM EDT reply actions 0 recs

Aaaaaaannd this is why we need a playoff.

by Tailgate Shogun on Oct 21, 2008 7:25 AM EDT reply actions 0 recs

A playoff wouldn’t change the need for a statistical ranking system. With 119 teams, boiling it down to anything reasonable (a 4 or 8 team playoff) will require just as stringent selection as you have now. Going from 119 to 2 isn’t much different than going from 119 to 8. The sport’s just too big to use arbitrary playoff rules (like the NFL). You need something based in statistics due to the sparse connectivity of the sport.

The choice between a playoff or a simple championship game is much more complicated. Each additional playoff tier reduces the “minimum bar” for the possible champion as well as extending the year (and, by attrition, reducing the overall strength of the teams).

Out of curiosity, why do you think we need a playoff? What problems, specifically, do you think it will solve? I’m not attempting to be critical, I just want to know why you think a description of statistical rankings implies that a playoff would be better.

by Bleed Blue 'n White on Oct 21, 2008 5:38 PM EDT up reply actions 0 recs

It's can be totally subjective.

It depends on how many teams you consider to be "National Championship Calibre" at the cession of the regular season. Essentially, how many finals matchups would you consider to be viable in determining the champion? In my opinion, the number varies between one matchup and six depending on the year. That means 2-4 teams total. There is a valid case in many years for the #3 team at the end of the season to have a chance at a title, but there are few cases for the #5 team and practically no cases of a #9 team in the discussion as equal substitutions.

This is entirely subjective, so it can be determined by human polls.

Interestingly, I saw in the SI Vault a 1965 College Football preview which included a playoff scenario for the 1965 season were the NCAA to sanction one (Sept., 20, 1965, page 40). The playoff discussion is not a new one.

by Cairo on Oct 21, 2008 6:55 PM EDT up reply actions 0 recs

Nothing against statistics...

I think the stats and the BCS formula are great if we were trying to narrow down to a 4 or whatever team playoff, like you stated. If the 1-3 teams go undefeated (or more for that matter or if we get jumped by an undefeated OK ST) what did all of these statistics really do for college football that the old AP/UPI and coaches poll rankings not do for us in the past? That will probably mean we get yet another undefeated team who doesn’t get to settle out on the field.

I can appreciate the work you and all of the stat guys have done and I truly mean that. But if we are on the outside looking in again then you tell those kids why they didn’t play for the title. I’d love to see a statistic/poll seeded short playoff and let chips fall where they may.

If you can smile when things go wrong, you have someone in mind to blame.

by TheMightyErik on Oct 21, 2008 10:42 PM EDT up reply actions 0 recs

But if we are on the outside looking in again then you tell those kids why they didn’t play for the title.

My point is, how would this be any different if you had a 4-team playoff? Some teams would still be on the outside looking in, and the only ‘reason’ for why they’d be on the outside is either a statistical ranking, a human ranking, or some arbitrary playoff tiebreaker.

All you’ve really done is transferred the “people on the outside looking in” to more teams, curiously.

I think the problem you’re talking about – difficult to decide who’s worthy of being the champion – can only really be fixed by either increasing the length of the season or reducing the size of the league. Deciding the champion of a 119-team league that plays 12 games is always going to be frought with “outside looking in,” regardless of how the championship is determined.

by Bleed Blue 'n White on Oct 21, 2008 11:10 PM EDT up reply actions 0 recs

Some teams would still be on the outside looking in, and the only ‘reason’ for why they’d be on the outside is either a statistical ranking, a human ranking, or some arbitrary playoff tiebreaker

Many people who see no problem with the current system make that argument. It is somewhat true, but given everything that has gone on during the BCS era, I’m sure the new teams that would be left on the outside looking in would be able to look back at the current system and agree every time that the playoff system works better. The fact that there is a possibility that an undefeated BCS conference team could be denied a chance at the title is absolutely ridiculous.

How about we consider a similar situation in college football…instant replay. After review, do some refs still get some calls wrong? They sure do. However, every coach will probably agree that college football with instant replay is better than college football without instant replay (even if that coach is Bob Stoops and Oklahoma is playing Oregon). That’s because the purpose of instant replay is to get it right, and that’s what it does most of the time.

Will a playoff system make sure ALL the right teams have a chance at the NC every time? Maybe not, but it would be a lot closer than the BCS.

by bbeck223 on Oct 22, 2008 2:06 AM EDT up reply actions 0 recs

Will a playoff system make sure ALL the right teams have a chance at the NC every time?

The thing people frequently don’t realize is that playoffs can hurt “the right” team’s chances at a NC just as much as the current “2 teams only” BCS selection mechanism. Playoffs necessarily extend the regular season, and lower the bar of entry for a playoff contender. Since the better team doesn’t always win, this necessarily means that your chances for not having “the right” teams play for a NC probably grows with the size of the playoff.

People are used to this dilution in most other sports, however, since playoffs are common. I’m not suggesting that playoffs are a bad thing, however, but the arguments that people use to support them just don’t hold water. Playoffs are common, and so their limitations are accepted. A system like the BCS is not, and so its limitations aren’t.

The fact that there is a possibility that an undefeated BCS conference team could be denied a chance at the title is absolutely ridiculous.

There will always be that possibility, so long as BCS conferences have multiple independent schedules! It’s not the selection mechanism. It’s the scheduling, and the size of the sport.

by Bleed Blue 'n White on Oct 22, 2008 1:40 PM EDT up reply actions 0 recs

The thing people frequently don’t realize is that playoffs can hurt "the right" team’s chances at a NC just as much as the current "2 teams only" BCS selection mechanism

I think we can all agree that the team that ends up winning the national championship in a playoff is the “right” team. No one ever disputes this in other sports.

There will always be that possibility, so long as BCS conferences have multiple independent schedules! It’s not the selection mechanism. It’s the scheduling, and the size of the sport.

When was the last time that there were four undefeated teams from BCS conferences in one season? Until that happens, getting through any schedule undefeated while in a BCS conference is a hell of an accomplishment…and one that shouldn’t go unrewarded.

I guess I would like to throw out one out-of-conference game and have an eight-team playoff. Everyone can say this will be too many games and such, but how about giving those teams two weeks off to get ready instead of 30-40 days?

by bbeck223 on Oct 22, 2008 9:54 PM EDT up reply actions 0 recs

I think we can all agree that the team that ends up winning the national championship in a playoff is the "right" team. No one ever disputes this in other sports.

That’s because in other sports, teams are far more connected, so the argument is poor. You can’t say that the Eagles should’ve been the Super Bowl champions in 2007, because they played the Super Bowl champions twice and lost. Every team in the NFL has a second-order relationship (Eagles play Bucs play Broncos, etc.) with every other team.

But loose connectivity, plus a short playoff, is something that doesn’t exist in basically any sport. College basketball has a huge playoff, plus better connectivity (in the Big 10, you’ve got 18 games among 10 opponents, as opposed to 8 games among 10 opponents in football), and basketball is a lower-attrition sport (and can therefore stand a longer playoff).

I think basketball has better parity than college football does, too. Not sure on that, though.

When was the last time that there were four undefeated teams from BCS conferences in one season?

We came within a field goal of having 4 undefeated BCS teams in 2004, and there were two non-BCS teams undefeated that year, too.

It should also be noted that the problem isn’t really just “many undefeateds”. It’s more “many one-loss teams.” In that situation, a playoff becomes tough to justify because you almost always have some random arbitrary rule stuck in there. You hear this in the NFL (“such and such got left out because the awful NFC West gets a bye”) and in other sports, too.

by Bleed Blue 'n White on Oct 22, 2008 11:02 PM EDT up reply actions 0 recs

Super Bowl comparison

granted, there is more connectivity in pro football, but if they used a BCS-like system, then the superbowl would have probably been played against the Patriots and Cowboys (or god, if it was like before even the BCS, they would have just handed it to the Patriots)

by The JuggerNitt on Oct 23, 2008 8:24 AM EDT up reply actions 0 recs

I don’t really see the problem. The Giants certainly weren’t “clearly, absolutely” an NFL champion. For one thing, they were a wild card: had a few other teams won a few more games (games that had nothing to do with the GIants), they wouldn’t’ve even been in the playoffs.

The fact that they won is exciting if you enjoy surprises and unexpected twists. The fact that they won is annoying if you preferred to see the best teams face each other.

by Bleed Blue 'n White on Oct 27, 2008 5:20 AM EDT up reply actions 0 recs

but that's what makes sports so enjoyable

are the surprises and twists (at least for me). And if you look at last year, the Patriots were clearly, absolutely an NFL champion…until they lost the championship game.

by The JuggerNitt on Oct 27, 2008 8:02 AM EDT up reply actions 0 recs

maybe we need something like the NHL's President's Trophy

To award the team that finished the regular season the “best”. If you notice, though, no one ever really brags about that one…I do recall a little thing called the Stanley Cup, though ;-)

by The JuggerNitt on Oct 27, 2008 8:04 AM EDT up reply actions 0 recs

Yes!

I think the problem you’re talking about – difficult to decide who’s worthy of being the champion – can only really be fixed by either increasing the length of the season or reducing the size of the league. Deciding the champion of a 119-team league that plays 12 games is always going to be frought with "outside looking in," regardless of how the championship is determined.

Banning games against lower-division competition and whittling the size of the FBS to 80 would go a long way here. But given the current situation, bootstrapping data is a popular method when the sample size is extremely limited. It would essentially add artificial data points by estimating game outcomes that never happened. This is done by sampling maximum likelihood functions generated from actual season data. While I’m sure that would sit so well with the intuitive football-watching crowd, it does improve the confidence intervals of many properties of the data, and might improve the overall ranking algorithm.

by gumbercules on Oct 22, 2008 9:18 AM EDT up reply actions 0 recs

Um, I don’t think that’s bootstrapping.

Bootstrapping is taking a set of data and expanding it by replicating (with replacement) the data that’s already in the set. Not adding data that doesn’t exist already.

Bootstrapping is basically best when you’re trying to get a robust measure of the variance and fluctuations of a quantity. It can’t improve the “best estimate” of a quantity, because that is (of course) always best estimated by the actual data in existence.

by Bleed Blue 'n White on Oct 22, 2008 2:23 PM EDT up reply actions 0 recs

Well

I’m working off of fuzzy memories of one lecture from a graduate course elective I took years ago, but I think your statement is true strictly for vanilla bootstrapping. I’m pretty sure there are plenty of variations that incorporate subtle modifications of the resampled data in one form or another. I’m not particularly interested in digging around in Google Scholar, though.

by gumbercules on Oct 22, 2008 4:32 PM EDT up reply actions 0 recs

I actually used statistical bootstrapping in my Ph. D. dissertation, so I’m pretty familiar with it, and most of the variations. If you look up some of the earliest papers on bootstrapping, you’ll see that the basic idea was to be able to extract information from the available distribution in an unbiased way (one that doesn’t require the assumption of normally-distributed data).

There are some which modify the resampled data, but that breaks the “unbias” assumption of the original bootstrapping. In complicated situations that can still be useful (if, for example, you’re missing some values in an N-tuple of measured values), but in this, it really wouldn’t be.

by Bleed Blue 'n White on Oct 22, 2008 7:15 PM EDT up reply actions 0 recs

Off-Topic

Where’d you do your Ph.D? The Machine Learning course I took was taught by Carlos Guestrin (fantastic guy), and while he was able to convey the importance of it all, I probably only absorbed a tenth of what came out of his mouth. Namely, that things like bootstrapping and support vector machines were hot at the moment (2006) and genetic algorithms and neural networks had taken a temporary backseat (I think). It’s fascinating stuff, but man, after working and earning a real income, I’d sooner commit suicide then spend 4-6 years doing more graduate study. Props to you for having the intestinal fortitude necessary, and I hope it pays off.

by gumbercules on Oct 22, 2008 9:40 PM EDT up reply actions 0 recs

Stating where I did my Ph.D. is grounds for death this week – I did my undergrad at Penn State, and 4 out of 6 years of grad school at Penn State, but then my adviser went to Ohio State, so I followed him there to finish. Realistically, I did my Ph.D. at Penn State – I took all of my classes there (save one, as Ohio State couldn’t graduate me without a GPA), and all I really did was my dissertation at OSU.

My adviser is really the one who taught me (OK, ‘pointed out to me and had me go read’) about statistics – specifically, bootstrapping. The huge benefit of bootstrapping, like I said, is that you get to throw out the assumption of normally distributed data.

after working and earning a real income, I’d sooner commit suicide then spend 4-6 years doing more graduate study.

Strangely, my logic was, if I went out and tried to earn a real income, I’d commit suicide.

and I hope it pays off.

Well, I’ve got a decent paying job at the University of Hawaii, so I’d say that’s a bit of a payoff. But really, the “paying off” part is “never having a boss who’s a moron.”

by Bleed Blue 'n White on Oct 22, 2008 10:49 PM EDT up reply actions 0 recs

Why we need a playoff

because there have been cases where the third place team has as good an argument (or even better) than the 1st/2nd place teams. Sure, if there were an 4, 6, 8, 12, 16 team playoff system, then #5, #7, #9, #13, and #17 would all complain (and probably even a few teams after those). Ideally the FBS (the artist previously known as Div 1-A) would be reduced to a smaller, more manageable size, but that doesn’t seem likely to happen, especially with the trend of actually adding more teams.

Being a statistics guy, you should want to have the highest confidence that the #1 team at least has a shot to PLAY for the national championship. Part of that would be to increase the playoff field until you reach at least a 90/95/99% confidence level. Obviously with the small sample size and not a lot of teams ever facing each other it is hard to come up with that. I think it could be argued, though, that the BCS formula is at least good at getting the #1 team into the top 8 (and maybe even the top 4, but again, since teams play against different opponents it makes this hard to determine).

An 8 team playoff would consist of 3 weeks of games. Since they’ve already extended the bowl season into the second week of January (the 8th), I don’t see why a game on the saturday before new years, the saturday nearest new years, and the saturday after new years (or whatever schedule best fits three games in by Jan 8th or so) would really be so difficult for them to achieve (especially since the bowl season already extends for that long). This wouldn’t interfere with the “OMG so important that I’m sure every student is studying so hard for it, and not just partying and slacking off like every college kid I’ve ever met” first week of the second semester, nor the end of the first semester (at least any more than current bowls do).

I think an alternate method that shouldn’t have too many detractors would be that the ~120 teams (which currently form 11 conferences, plus some independents) either redistribute into 10 conferences with 12 teams, 12 conferences with 10 teams, or just stay as they are. But then you take the champion of each of the leagues, and fill in remaining playoff spots (for either a 12 or 16 team playoff, which would be a bit longer than the format above at 4 weeks) with “at large” teams, decided either by a selection committee, or by taking the top teams from the BCS formula that weren’t conference champs. Basically acting just like the NCAA basketball tournament. That seems to work without much problem for a lot of people.

by The JuggerNitt on Oct 22, 2008 10:42 AM EDT up reply actions 0 recs

Oh and I know I'm preaching to the choir

or at least to people who don’t have any influence in the matter, but I like to rant about things that annoy me :-p

by The JuggerNitt on Oct 22, 2008 10:43 AM EDT up reply actions 0 recs

One additional "format" choice

that would be hard to really argue against, but wouldn’t really give a strict format for every year, so it would be tricky, is that you could do something like the +1 format (where the top 2 teams after the BCS bowl games then face off in the championship), only it would continue until you only have 1 undefeated team (or if there are no undefeated teams, then only one 1-loss team, etc). From what I can tell, the worst case scenario in something like that would be that an undefeated non-BCS team manages to then face the sole remaining BCS undefeated team, and happens to win, though in that case I would say, “ok, they were undefeated, even against the best BCS team, they deserve the crown”

by The JuggerNitt on Oct 22, 2008 1:19 PM EDT up reply actions 0 recs

I think it could be argued, though, that the BCS formula is at least good at getting the #1 team into the top 8

Where was Hawaii at the end of the BCS standings? I’m not saying that they were a #1 team, but it’s theoretically possible that they could’ve been and we might not have been able to tell.

That seems to work without much problem for a lot of people.

Basketball has a ludicrously-lower attrition-per-game rate than college football. A 16-team playoff is 4 extra games – and also represents a significant dilution in the importance of the regular season, mind you! – and those 4 extra games represent an increase in the size of the season by one third. One-third extra games means more injuries, and means the teams look a lot different at the end of the season than they do at the end of the playoffs.

In addition, it’s important to remember that easily one third, and more likely one half, of those 120 teams are total fodder. In that situation, the conference alignments will have way too much importance – you’ll essentially be giving a bye to teams who play in conferences where the 2 or 3 strong teams are in a weak stretch.

You also still don’t fix the connectivity problem, either. Realistically, I think you either accept the fact that college football can’t be “fixed,” and suggest that a playoff is just more accepted in sports, or suggest splitting Division I-A in two – or possibly three.

by Bleed Blue 'n White on Oct 22, 2008 2:20 PM EDT up reply actions 0 recs

well

as for Hawaii, see: Sugar Bowl: UGA 41, Hawaii 10.

As for conference alignments, I’d actually like to see a more balanced alignment of conferences, but we know that that isn’t likely, and probably not that good, since conference affiliations and rivalries are one of the beauties of college football.

I don’t see it a playoff cheapening the regular season if you still need to win your conference (or be one of a small few “at large”) for a shot at the title.

I do agree with your point of extending hte season by 1/3, though only for the teams that keep winning. I don’t know how much I’d mind seeing extra games by the best teams, even if they are different than the teams that entered. Greater chance of injuries would suck, especially for the guys wanting to go pro, but I know if I was a player, I’d want to be on the field playing, even at the risk of injury, ESPECIALLY if a championship is at stake.

The connectivity problem isn’t really solved, but you would be able to say that you’re beating the best of one conference, who beat the rest of the teams in their conference. As it is now we can say we beat Coastal Carolina, Syracuse, Temple, and Oregon State, which doesn’t really give any indication of how good we actually are. If we beat Pitt, Ball State, and USC, (or whoever you think the conference champs will be) then we could probably reason that we would beat the teams that are worse than them.

by The JuggerNitt on Oct 22, 2008 2:33 PM EDT up reply actions 0 recs

as for Hawaii, see: Sugar Bowl: UGA 41, Hawaii 10.

What will your defense be when it happens the next time, and the next version of Hawaii wins?

The connectivity problem isn’t really solved, but you would be able to say that you’re beating the best of one conference, who beat the rest of the teams in their conference.

So what? If you look at the Big East this year, it’s almost a disaster. And that’s a BCS conference – a reallocation of teams would produce conferences worse than the worst BCS conference, on average.

I think you’re slightly missing the point, though – the problem is that teams in weak conferences will get into the playoffs easier. You can imagine some perverted scenario where, in a very weak conference, a team like USC gets into the playoffs, and in a strong conference, you’ve got, say, Texas and Georgia, each with two losses (Georgia beating Texas, etc.) but Texas getting in on some weird tiebreaker. Texas then runs the table in the playoffs, and faces USC in the NC and either wins, or loses. Doesn’t matter.

Georgia would have a strong case that the playoff system screwed them, because USC got in, and really didn’t deserve to, and they, in fact, beat the national champion.

Wildcards slightly help there, but not really: the problem is that a non-deserving team (USC) got an automatic berth, which reduced the overall strength of the teams in the playoffs, and necessarily forced some deserving team out of the playoffs.

Again, I don’t want to make it sound like I’m saying that playoffs suck: the problem is that a short playoff, in conjunction with a gigantic sport with massive parity issues, is going to be difficult. You should be sensing a theme: the problem is that the sport is too big, with waaay too big a gap between #1 and #119.

by Bleed Blue 'n White on Oct 22, 2008 7:26 PM EDT up reply actions 0 recs

well, my case for if Hawaii wins next time: I think any undefeated team should get a slot in the playoff. My biggest gripe(s) are that undefeated teams can get shafted, and that even 1 loss teams can get shafted. I just want a playoff system that includes as many of the undefeateds (and 1 loss teams) as possible. Obviously there’ll be problems if something like last year happens, where everyone has a loss, and so even a 2-loss team makes it to the championship game (and won), but in a season like that, there were only I think 12 teams with only 1 or 2 loss, which would also be doable for a playoff chance.

In my “conference champ goes to the playoff” system, then you’d have the “problem” seen in other playoffs (NCAA basketball, football, baseball, etc). Does anyone really expect the Ivy League champion to do much in the playoffs? If anything, having this easy team in the playoffs is almost like a BYE to reward the top teams, while still recognizing the legitimacy of the non-BCS conference champs with a chance to win it all.

In a case you cited where Georgia and Texas are in the same conference, but due to a weird tie-breaker rule Texas goes on…well that would be a gripe that Georgia has with the conference, and in the current system they wouldn’t be given a shot at the title anyway. I’m a big proponent of the “if you can’t win your conference, then you shouldn’t get an automatic shot at the National Championship” If they’re still good enough to get a wild card/at large type slot, then they didn’t totally get screwed. Ideally for this situation, all the conferences would have the same, or very similar, rules for determining the champion, and there would be no real “ties”. Conference Championship games do a lot for at least forcing the 2 best teams to play each other (unlike the Big 10 where you could have 2 undefeated teams who never faced each other…though again, if they’re undefeated, then they’d likely get a wildcard slot).

The case about an undeserving team getting into the playoff is a matter of opinion. I think if you win your conference and/or go undefeated, you are pretty deserving. Maybe you’d even see more parity, since players would start flocking to these “easy” conferences, knowing they’d have a better shot of making it to the playoffs. Heck, I’d even be in favor of “dropping” teams into a lower subdivision, but I almost wish it was like the premier soccer league in Europe, where the bottom teams from the top division move down and the top teams from the lower division move up.

OK, I’m really just starting to ramble, and I think basically state the same thing. I see and understand the points you’ve made. I understand the attrition aspect (I’d be in favor of dropping the non-conference games, especially against 1-AA teams, to institute another round of playoffs), and I definitely understand the connectivity and parity issues. I think the best way to resolve such issues, though, is to play MORE games (playoff of some type) instead of just playing 1 game (current system).

by The JuggerNitt on Oct 23, 2008 8:48 AM EDT up reply actions 0 recs

also about hawaii

that was why I indicated possibilities of conference champs going to the playoff, or having the 1(1+1+1) system

by The JuggerNitt on Oct 22, 2008 2:35 PM EDT up reply actions 0 recs

The Problem with a Playoff

The larger you make the playoff, the greater the likelihood the best team will not win. A good anecdotal example is MLB in the Wild Card era. A large sample size of data where every team plays essentially the same schedule, like a 162 game MLB schedule, is the best means of determining a champion. European soccer leagues have long understood this as the champion is the team that has the best record playing each opponent home and away. At any rate, now that there are 8 teams in the MLB playoffs instead of 4, the odds are stacked against the best team. Short 5 and 7 game series’ are little better than coin flips at determining the better team (especially when superior teams often win no more than 60% of their games) which means that the team with best regular season record from either league is unlikely to make it to the World Series.

In college football a round-robin schedule is clearly not feasible. But football does have the advantage of requiring a smaller sample size to produce statistically meaningful results. In baseball a one game series is about as meaningful as a coin flip. Superior football teams, even those in competitive leagues, can usually win 80% or more of their games. A short season is a pretty decent predictor of quality.

But here’s where I’m skeptical of a college playoff. A 2 team playoff potentially excludes the best team on occasion. But that single game has a high likelihood of determining the better team out of those two teams. A 4 team playoff will almost always include the best team, but if we (generously) estimate the best team would win 70% of its games against top 4 teams, the real “best” team would win the championship only 49% of the time. And I’d bet even a program like USC would be hard pressed to win even 60% of the time against the very best of the best in reality.

The larger you make the playoff, the greater the likelihood the top team will be upset. That’s why the NCAA tournament is so much fun—the odds are stacked against the top seeds! And in fact the current FBS system is a playoff of sorts that doesn’t and cannot reward the best team. More often than not it rewards a very good team that has had the best season (i.e. 0 or 1 losses). The odds are simply against even the best program going undefeated. More often the truly best team loses a game, while one or two slightly weaker teams from the pool of 119 go undefeated.

Personally, I like the idea of rewarding the team that has had the best season. Playoffs are not a solution. For a short season with a large league, the best bet is to use statistical rankings to select the 2 (or maybe 4) teams having the best season to face each other. Even then a 3 game series between #1 and #2 would be more telling than a single game.

by gcdyersb on Oct 22, 2008 12:11 PM EDT reply actions 0 recs

I see your point

but I still would rather have the playoff. Regular season would still be meaningful, and perhaps moreso since you could lose a game or 2 and still try, opposed to folding up everything and going home to practice for next year.

I guess I’m also in the school of thought that would rather have it played out on the field. Going with your “the best team would only win it 49% of the time” We’ll say that the rankings 1-4, that the favored team would be favored by 10 points if they were 3 ranks higher, 7 points if 2 ranks higher, and 3 points if they were 1 rank higher (this is obviously a complete hackjob on numbers, but it is just an example). Then using historic win % based on certain spreads, that would equate to 77% wins, 70% wins, and 57% wins by the favored teams. Calculating out all the possiblities, then, and assuming that the seeding was perfect (which it likely wouldn’t be), then #1 would win a playoff of 4 teams 45% of the time, #2 would win 30% of the time, #3 would win 17% of the time, and #4 would win 8% of the time.

Again, on assumption say that the #1 vs #2 matchup had a 3 point spread, then #1 would win 57% while #2 would win 43% of the time. Sure they drop when you add in more teams, but not really by a ridiculous amount (about 12% less for each team). That’s not too horrible, considering you are giving #3 and #4 and INFINITELY better chance at winning the title.

I guess in the end all these calculations are pointless, though, since you can’t really define the true % of time a team would win, and the seeding wouldn’t be perfect, but they can give somewhat of an approximation.

by The JuggerNitt on Oct 22, 2008 1:15 PM EDT up reply actions 0 recs

I give up...

We should stop playing the game altogether. Neither the BCS or a playoff can ever find a true champion. Every team should just take turns.

If you can smile when things go wrong, you have someone in mind to blame.

by TheMightyErik on Oct 22, 2008 1:17 PM EDT reply actions 0 recs

Except Pitt....they don't get a turn.

I bleed Blue and White.

by Horse N Buggy on Oct 22, 2008 1:28 PM EDT up reply actions 0 recs

Oh, my bad!

You are correct! I can also name several others:

Notre Dame, OU, USC, ’Bama: You guys already had enough turns
tOSU: Your mascot is ridiculous
Nebraska: -1 turn for ripping off one of ours

and more I am sure I am forgetting :)

If you can smile when things go wrong, you have someone in mind to blame.

by TheMightyErik on Oct 22, 2008 1:35 PM EDT up reply actions 0 recs

The point that I’m trying to make – and I’m apparently not making it that clearly – is that instituting a playoff, while not fixing the underlying problem of “it’s really freaking difficult to figure out team quality in a huge sport with little connectivity” doesn’t fix most of the problems that people think a playoff would fix.

Unfortunately, the problem is that the economics of college football don’t allow the best solutions: a reduction in the size of the league and forced scheduling. Without those two things, I’m just not sure that a playoff buys you much. If you look at last year, a team like Hawaii might get left out – and while a ranking system, and human polls, can tell that they’re likely not worthy, there’s no way to know.

People have been arguing about “who the champion is” in college football for its entire existence. I don’t think there’s a “magic bullet” for fixing it. The problem is intrinsic to a sport with such screwed up economics that it requires the existence of fodder to support the larger teams.

by Bleed Blue 'n White on Oct 22, 2008 1:50 PM EDT up reply actions 0 recs

Dude, I get your point

I agree and there will never be an easy way out of this. You are making great points and I concur that there will NEVER be that ‘magic bullet’ that will satisfy the masses. A playoff is what everyone would like to see and yet people will still bitch about it. I am with ya, man. I was just trying to lighten the mood a bit. :)

If you can smile when things go wrong, you have someone in mind to blame.

by TheMightyErik on Oct 22, 2008 2:05 PM EDT up reply actions 0 recs

Actually, to state my point better: I think a much better argument for a playoff is that basically every other sport has one, and it’s well-entrenched in our sports culture that playoffs are the best way to determine a champion.

by Bleed Blue 'n White on Oct 22, 2008 2:05 PM EDT up reply actions 0 recs

you are right, there is no magic bullet

and I suppose it all depends on what we are really looking for when we say “national champion”. Do we want to choose the best team in college (which is virtually impossible to do, especially when you consider are you talking about over the course of the whole season…at the specific point at the end of the season, etc), or are you asking, “which team has proven itself on the field and won the championship”?

I think for the latter, having a playoff with a reasonable entrance requirement (conference champions, top N positions in the BCS formula, or a mixture of the 2 such as “at large” bids) is a pretty good way of determining “what team won the championship game from the pool of qualified teams.” In my opinion it is at least better than the “which team(s) are we guessing/estimating are the champions.”

I suppose if there were some way of actually quantifying the teams as which is the real “best” team, even in retrospect, we could figure out the probability that the “best” team is in the top 1, 2, 3, 4…8….16….120 of whatever ranking system you use

I’d imagine it looks something like (with completely made up, and probably underestimated numbers)
top 1: 60%
top 2: 75%
top 3: 85%
top 4: 90%
top 8: 99%

And yes, there is hte problem that the “best team” didn’t actually win the championship, but for some reason I really just don’t have a problem with that. Maybe it is because if they couldn’t win it, then perhaps they weren’t great enough of a team for me to have been happy to just “vote” them the champion in the first place. Or maybe it is like you said, I’m used to playoffs in other sports, and am willing to accept the downfalls. Also: I like systems where I get more top level college football games to watch :-D (and nobody can convince me that having any team with a winning record go “bowling” does anything to achieve this).

by The JuggerNitt on Oct 22, 2008 2:45 PM EDT up reply actions 0 recs

Also: I like systems where I get more top level college football games to watch :-D

It’s important to remember that the longer the season goes, the worse the teams will be, on average. Shorter seasons = better football. Attrition sucks.

by Bleed Blue 'n White on Oct 22, 2008 7:29 PM EDT up reply actions 0 recs