Recruiting Analysis: Do Stars Matter? Yes and No

I know we beat this topic to death last week complete with a big twitter fight and everything, but I've been so busy putting together We Are Penn State 2011 I didn't really get a chance to put my thoughts together on the issue. Today I read some new thoughts on the topic, and I got a few hours to kill, it's the offseason, and I'm sick of working on WAPS 2011, so lucky you.

I was reading Linebacker U this morning and I see our friend Junny has a fine recruiting update post where he gives us a lot of good information and another slightly out-of-place image of some random celebrity with nice... boobs...........Sorry. I got distracted there. Back to the topic at hand.

What caught my eye in the post is that once again he's beating the drum that kind of makes me squirm in my seat.

Stars Matter

Stars Matter

Stars Matter

Long story short, stars do matter, but so does player development. Everyone from 11 Warriors to mGoBlog to EDSBS and everyone in between (including numerous message boards) has covered this article, and rightfully so.

The article that Junny is referring to is over on Black Heart Gold Pants where an astute reader points out the correllation between recruiting stars and where a kid goes in the NFL draft. There is a very nice chart there that breaks down the average draft position based on the number of stars a kid had coming out of high school. Just looking at the chart, it looks pretty obvious that five-star kids go higher in the draft than four-star kids who go higher than three-star kids and so on.

But then they add this at the end. (Emphasis added)

Ok, stars matter, but they certainly don't tell the whole story. Even among those bluest of the blue-chippers, the 5-star recruits, the large majority will never be drafted! Motivation, injury and life all play a big role, but the differences between programs must matter. Player development - its something Iowa fans have heard about as Kirk Ferentz is known as one of the best. But is it true? Can we see it in the numbers?

The writer then goes on to break down which schools are better at developing talent than others. It's fascinating stuff, but it's not what I want to discuss here. So I'll let you go read that on your own if you want to. I want to address this thought that "stars matter".

I'm an engineer that works in a technical field. As such, I'm always analyzing data, and part of that analysis is to question the data. Where did it come from? What does it tell me? Can I use it? Can I trust it?

To me, stars are data. I look at them. I use them to form opinions and theories. In some context I trust them to back up an opinion. In other context I don't.

If I'm analyzing an entire recruiting class, stars matter. A class full of four-star athletes is going to outperform a class full of three-star athletes 99.9% of the time. But if I'm looking at just two kids, and one has five stars and the other has three stars and you tell me the five star kid is going to be a better player and be a higher NFL draft pick, I'm not going to take that as a foregone conclusion.

I have a lot of problems with the star ranking system. Stars aren't something you can measure on a kid. It's not like his height, his weight, his 40 time, how much he can bench or squat. Stars are an arbitrary ranking bestowed on a kid by a recruiting service. These people watch a kid perform in a camp, they watch a few minutes of film, and then they hang some stars on him.

As an engineer acutely trained to analyze data, I have a lot of problems with putting a lot of faith in these stars. First of all, the good people at Rivals and Scout can't possibly see every kid in America. As such, kids from big schools in metropolitan areas get more attention than the kids from smaller rural schools. Kids that go to their camps and combines naturally get more attention than the kids that don't. There are politics involved. I see a lot of instances where they hang three stars on a kid, but then he gets offers from Notre Dame and USC and then he mysteriously gets a fourth star.

Call me crazy if you want. But listen to reader Carolinaeasy, a high school football coach in South Carolina who has the unique perspective of having access to the biz. This is how he describes the sausage being made. (emphasis mine)

The star rankings in the NCAA recruiting process are extremely corrupt. The people applying those ratings have had a vested interest in selling their product to the masses. In order to make their particular product stand out more they have to get access to the "biggest" stars around, sometimes they will jack a kid’s star ranking up because it will sell more subscriptions. In short those in charge of applying rankings have extrinsic motivation that can skew their end product.

In other cases some of those applying the star rankings can be influenced by the coaches recruiting these players. A coach can get on the phone make a few phone calls and get a kids star ranking improved by as much as two to three stars. Not by anything they did on the field, but by a phone call by a recruiter who wants to have a "higher" profile due to his efforts.

Star rankings are corrupt and as such I don’t take them into account. If the player isn’t one of the top 10 recruits in the country I believe it is a crap shoot as to how things turn out.

This, and other comments I've read through the years, is why I don't get all worked up over the star system. I don't want to suggest the good people that run Rivals and Scout and 24/7 are doing anything shady to sell subscriptions. But they know that their star rankings are arbitrary. In the words of The Big Lebowski, "That's just, like, your opinion, man." Stars are a way of building buzz around certain kids and getting fans worked up. When fans get anxious, they want to know more information, and when they want more information, they're willing to pay for it. That's just their business model. You can choose to take part in it if you want to. Some people don't.

I actually do buy the subscriptions. Like I said, I put a little stock in the star system, but I don't sell out on it. Stars are a useful piece of information, and as an engineer I never refuse information unless I can prove it is invalid. The people at Rivals and Scout, though they may manipulate the stars for a particular kid, they still have a vested interest in getting it right. If they misevaluate too many kids people will see their information isn't credible, and if their information isn't credible, people stop paying for their service. This is bad for the business model. So it pays them to be as accurate as possible. As carolinaeasy said, I think they get the top 10 or 25 kids pretty accurate. But in my opinion the difference between a two-star kid and a three-star kid cannot be measured. A kid with one-star or no-star cannot be simply dismissed as a wasted scholarship before he ever sets foot on campus. We can't forget that the coaches have a vested interest in getting it right as well. They only have 85 scholarships to play with, so they can't afford to miss on too many kids.

So in conclusion, stars are a useful tool in ranking the overall value in a class, but I don't look at stars as a map for how an individual student's career will unfold.