BCS Rankings Downfall and Lessons Learned

The BCS College Football Era was filled with strong opinions.

Most of those opinions were negative, but you know what they say…

The grass is always greener on the other side.

Today, you’ll hear calls for NCAA football to bring the BCS back. There’s no question the BCS was flawed, but scrapping it all together probably wasn’t the right idea either.

Here’s what we’ve learned about why the BCS rightfully failed, why it should bring it back (with some changes), and general sports analytics takeaways from its downfall.

What was the BCS?

The BCS ranked Division I NCAA teams using a combination of human polls and computer mathematical models. It started out by giving equal weighting to both human and computer rankings, but after the controversy that the computer rankings weren’t great, they revised the BCS ranking to give the human polls twice the influence on the rankings compared to computer pools

Spoiler alert: controversy didn’t end (more on this later)

The BCS rankings determined the NCAA Division I Championship Game. It also helped determine four other key bowl games, but these games were not directly determined from the ranking. There were automatic bowl qualifiers, mostly based on the major conference winners.

When was the BCS?

The BCS rankings were from 1998-2013. It wasn’t the same ranking system this entire time. There were many changes in the computer models and some changes in the human polls that were averaged in. The biggest change was in 2004 when it gave human polls double the weight of computer models.

‘Cracking the Egg’ Criticisms of the Mathematical Portion of the BCS

The BCS couldn’t get away from the heat of criticism.

Some criticized the computer models. They highlighted times when computer models weren’t close to meeting the “eye test.”

One critic was Jon Dokter who poked the NCAA-bear by highlighting flaws in its computer models in an article he called “Cracking the BCS Egg.” What made Jon’s criticisms hold more weight has created some of the most respected computer predictive models for multiple sports.

Jon seemed to have a few major criticisms:

1. Mathematical models improve with more data, and NCAA Division I football seasons are only slightly over 10 games.
2. To make point #1 worse, the NCAA wouldn’t allow margin of victory to be used in the calculations. This took out a critical piece of data when data was already limited.
3. NCAA kept some of it’s rankings in a “black box,” so the public could see rankings seemed off but it fueled the fire to not exactly know why.
4. The BCS seemed to blindly accept computer models, when they easily could have used the models on past seasons to see how well they worked.

For point #4, Dokter didn’t just throw out the criticisms. He did exactly what he said the NCAA should. He rereated the Colley Matrix (one of the BCS computer models), and started using it on historical years. The first year he tested it on 1997 showed it’s failures. Here’s the Colley Matrix top 5 for 1997:

Ranking	Team	Record (wins-losses)
1	Tennessee	11-2
2	Michigan	12-0
3	Nebraska	12-0
4	Florida	11-2
5	Florida State	11-2

You can look at the table and it immediately raises a red flag. Why is Tennessee ranked ahead of undefeated Michigan and Nebraska, when they have two losses?

Maybe when you dig deeper, you’ll find that Tennessee just played better competition. But here’s where it gets worse:

Tennessee and Nebraska played head to head, and undefeated Nebraska cruised to an easy 25-point win. And to throw in one other controversy, #4 Florida beat Tennessee by 13 points.

It didn’t take a lot of digging, and a quick eye test could tell the NCAA that maybe some of the computer ranking systems it was using needed a little tweaking.

Criticisms of the Human Element of the BCS

There’s nothing groundbreaking here, but it’s worth pointing out:

Humans have limitations when it comes to rankings. Especially humans that have big money incentives for a certain result. The human polls averaged into the BCS rankings were a top 25 list from sportswriters and coaches.

Do you think sportswriters and coaches could ever have incentives for one team to be ranked over another?

Sure, you hope they all have integrity. And I’m sure most at least want to be unbiased, but there’s no question that the ranking was flawed.

Now, college football uses a playoff system, and there’s a committee that determines who’s in and who’s out. The problem is there are many conflicts of interest in this committee (just follow the money).

So now that the BCS doesn’t use computer models, you’ll hear many angry fans wanting it back. And maybe they’re, right? You can go through almost every year in NCAA College Football Playoff history and find controversial results from the committee.

So should the NCAA bring the BCS rankings back?

I can’t get away from the hunch that you can’t rank sports teams fairly without at least partially using a computer.

Most of the reasons the BCS failed in the first place can be fixed. One fix is already in place: you have a playoff at the end of the year instead of any kind of system that tell you who the two most deserving teams out of 200+ area after only 10-15 games.

The other is you put a little more thought in your computer systems. The NCAA just let a few selected people make computer systems, and they didn’t appear to make any effort to check them (see above). They also handicapped these data scientists by not allowing them to use important data for creating a ranking (mainly margin of victory). Finally, data science and machine learning has come a long way.

So no one system is going to be perfect, but I think having a computer as a starting point and then a human element that tells us maybe where the computers have gone would be perfect. Well, as perfect as it could be. The not-so-perfect results can be compensated by having a playoff. And now we have that.

Takeaways from the BCS for College Sports Rankings and Betting

After you sift through all the controversy, there are some takeaways from the BCS ranking era…

1. There will always be unhappy sports fans, and maybe that’s what makes it great.

I just don’t think there’s any way around sports controversy. And maybe that’s what makes sports great. We can passionately argue our side, but you can’t ask Google “Who should have been ranked as the #1 NCAA Division I football team in 1997?” and get any answer other than someone else’s opinion. People hated the BCS. Now, some people want it back.

2. Computers are far from perfect.

There have been many different NCAA football computer rankings systems through the years, but none are close to perfect. In fact, there are very few that get close to competing with Vegas lines. And for the record, Vegas also isn’t close to perfect on predicting games.

3. Computers models can be pretty good.

Many people miss the unbiased element of the computer models in BCS. And some people realize that a big reason the computer models failed was the human element–the main one was that the BCS limited the data mathematicians were allowed to use in their computer models. If you allow that, computer rankings will still have the odd ranking but can be pretty good and likely can improve with time.