I do not see a NFL starting QB in Buffalo Bills’ Josh Allen, says ESPN’s Aaron Schatz

SackMan518 · May 26, 2019

He might need to take a look at that Cover 1 article breaking down Allen's play before making these statements.

Many national NFL analysts are starting to come around on the Buffalo Bills. Don’t count Aaron Schatz, ESPN NFL analyst and Head of Football Outsiders, among them.

Schatz appeared on WGR-550′s Schopp and the Bulldog on Friday to discuss the NFL offseason. Along the way, Schatz was asked about the Bills and the sudden amount of optimism surrounding the team. When asked if he was among those who were optimistic about Buffalo, Schatz stated he was not despite the team making some good moves in free agency.

“I’ve noticed a lot of optimism about the moves that the Bills have made in the offseason," Schatz said. "That they’ve made a lot of good moves. And here’s the thing, I think they’ve made a lot of good moves. The problem is that where they were going to be before those moves is not as good as I think where Bills fans think they would be before those moves. So even after making a lot of good moves, I’m not that optimistic about them.”

Perhaps the biggest reason why Schatz does not believe in Buffalo is due to quarterback Josh Allen. Despite leading the Bills to five of their six victories last season, Schatz told Schopp and the Bulldog that he doesn’t view Allen as a NFL starting quarterback.

“I’m just not a Josh Allen believer. I do not think there is an NFL starting quarterback in there waiting to come out and develop. And if I’m wrong then, you know, they’ve done a lot to give him help, but if I’m right then they’re still in trouble.”

Schatz isn’t the first NFL analyst to be down on Allen and he certainly won’t be the last, but the quarterback improved significantly when he returned from an elbow injury. By the end of his rookie year, Allen passed for 2074 yards and added 631 yards on the ground in 12 games (11 starts). He finished the year with 18 total touchdowns and 12 interceptions.

In his second season, Allen will be expected to make strides on these numbers, as well as his completion percentage (52.8). Buffalo made several upgrades along the offensive line to protect the quarterback and his wide receivers are much improved from one season ago.

DC2007 · May 26, 2019

You really can’t blame him, 52% is pretty awful. He’s been a gamer, but is he a QB or a giant Tyrod? That’s to be seen. I think he can improve, but the lack of touch on short throws scares me, that is not easy to “develop” in the pros. By this time, you would hope he has some of that touch, you see the guys like Mayfield, Wilson, Murray, etc they come into the league looking like a QB, they may have a physical limitation, but they play QB. Allen is the opposite, you are counting on a great athlete learning to play the position, that’s rarely the guy that makes it big in the NFL.

End of the day, this season and probably next, lies squarely on what Josh allen can do, nothing more, nothing less. If he fails, so does the team. If he makes huge strides, they are set.

Down and Out · May 26, 2019

You really can’t blame him, 52% is pretty awful. He’s been a gamer, but is he a QB or a giant Tyrod? That’s to be seen. I think he can improve, but the lack of touch on short throws scares me, that is not easy to “develop” in the pros. By this time, you would hope he has some of that touch, you see the guys like Mayfield, Wilson, Murray, etc they come into the league looking like a QB, they may have a physical limitation, but they play QB. Allen is the opposite, you are counting on a great athlete learning to play the position, that’s rarely the guy that makes it big in the NFL.

End of the day, this season and probably next, lies squarely on what Josh allen can do, nothing more, nothing less. If he fails, so does the team. If he makes huge strides, they are set.

I see no reason to believe that Allen will fall flat like some pundits and fans pontificate. Allen played that way out of necessity ( no oline and no hands from wrs most of the year. With the talent added to the oline and reciever group, I'm pretty sure Allen is gonna have a huge season for us. Folks also need to understand that Allen's style of play is that of a gunslinger, which is what is needed imo, but he IS gonna have some games where he will try and do too much, I suspect year 3 is where he will put it all together assuming the team, OC and playbooks don't change drastically.

nealpellecchia@yahoo · May 26, 2019

He was a runner in college too I read. He needs to master short passing to move the chains. Throwing fast balls on cold days is unadvisable too.

CBennett · May 26, 2019

I have heard BOTH sides...some say he wont be good...many say he will be....if you take the positive guys VS the negative guys id say its 70% pro Allen 30% think Bills will be looking for a QB again in 2-3 years. At the end of last year I saw enough that while im NOT SOLD yet..im positive he will take a step forward.

junglesouljah · May 26, 2019

What does ESPN NFL analyst and Head of Football Outsiders know about the development of a QB? Schatz doesn't think Allen can improve as a player? Schatz is biased as he basses everything on stats without context. The fact Allen survived last year with no running game, with piss poor pass protection literally running for his life showed toughness and leadership and a will to win.

This analysis by Schatz is a complete joke. He is making the assumption that Allen can't improve because of his confirmation bias that he doesn't like Josh Allen. Schatz would rather stubbornly stick to his guns about his evaluation about Allen, than admit the young QB can improve.

Mainejays · May 26, 2019

Who?

junglesouljah · May 26, 2019

You really can’t blame him, 52% is pretty awful. He’s been a gamer, but is he a QB or a giant Tyrod? That’s to be seen. I think he can improve, but the lack of touch on short throws scares me, that is not easy to “develop” in the pros. By this time, you would hope he has some of that touch, you see the guys like Mayfield, Wilson, Murray, etc they come into the league looking like a QB, they may have a physical limitation, but they play QB. Allen is the opposite, you are counting on a great athlete learning to play the position, that’s rarely the guy that makes it big in the NFL.

End of the day, this season and probably next, lies squarely on what Josh Allen can do, nothing more, nothing less. If he fails, so does the team. If he makes huge strides, they are set.

Stats without context are meaningless. The OL lost LT Cordy Glenn, LG Richie Incognito, C Eric Woods, that may affect the ability of the OLto dominate the line of scrimmage. BTW it must be a coincidence that there will be new starters at LG, C, RT and RT.

Allen is capable of making short throws, he just has to fix minor footwork issues. I would rather have a QB like Josh Allen than a QB like Blake Bortles or Josh Allen.

junglesouljah · May 26, 2019

Who?

Aaron Schatz is ESPN NFL analyst and Head of Football Outsiders. Football Outsiders is similar to Pro Football Focus, a football analytics company.

You can't base the evaluation of players solely on analytics. Schatz doesn't like Allen because he doesn't have good stats. I wonder how many of Allen's games Schatz watched of Allen? Mel Kiper gave his analysis of Allen but watched every game Allen played and made points of his untimely penalties and dropped passes hurt the Bills offense last year. The box score scouting that Schatz does is not innovative or insightfull.

junglesouljah · May 26, 2019

I have heard BOTH sides...some say he wont be good...many say he will be....if you take the positive guys VS the negative guys id say its 70% pro Allen 30% think Bills will be looking for a QB again in 2-3 years. At the end of last year I saw enough that while im NOT SOLD yet..im positive he will take a step forward.

Josh Allen is a work in progress, he is not a finished product, that was his scouting report. Allen is an ascending player who improved throughout the season having his best game the last game of the season versus Miami.

He needs to take the next step in his development this year, which I believe he is capable of making. I'm not going to do lazy analysis like Schatz because Allen struggled to play behind a terrible OL.

“I’m just not a Josh Allen believer. I do not think there is an NFL starting quarterback in there waiting to come out and develop. And if I’m wrong then, you know, they’ve done a lot to give him help, but if I’m right then they’re still in trouble.”

To paraphrase Schatz he hates Allen and doesn't think he will develop as a QB, but if he does get better he gets no credit because Buffalo surrounded him with better players. Wow, very insightful analysis by Schatz.

Meathead · May 26, 2019

i would tell you my opinion but i have to go take a wicked schatz

Topas · May 26, 2019

The guys pissing on Schatz have obviously never read footballoutsiders. He is the first to say that stats without context mean nothing and that this is only an addition to watching tape. You can say because he is a statistician he only believes numbers but that is flat out wrong.

His argument is that history has proven that QBs in the Pros do not improve significantly with regard to accuracy and that QBs below 60% do not become Franchise QBs. Can Allen be an exception. Yes. Is it likely? No.

Let’s see ...

And don’t give me Favre. He played in another time with regard to completion percentage ...

scotsho · May 26, 2019

He'll end up being one of those QBs that you can't quite cut bait from. A Bortles or a Tannehill, somebody who will show flashes, but then goes into big funks. The rookie deal won't be quite enough, he'll get a bridge deal to prolong the inevitable.

And I am Mr Homer, I want him to succeed.

Meathead · May 26, 2019

quite frankly, most of last season, from a passing perspective, i didnt see an nfl starting qb either. his biggest problems by far last season were a) struggling with the shorter game, and 4) running too much. he also struggled trying (not trying?) to take some fire off the ball when he didnt need it

but c'mon, he was a rookie. all the rookies struggled with compl% except mayfield, who had easily the best ball catchers of any starting rookie

i also have to admit im deeply skeptical of his ability to ever turn into a 65% passer, but he might not have to. 60% would be the bare minimum he would need to make his style a winning style. i think he can make that just by improving shorter ball accuracy and not throwing shorter balls like they need to go through a brick wall. every point above that just brings him closer and closer to franchise status

65% would only be around 20th in the league, but with his style it would be fantastic. combining his long ball prowess, his excellent scrambling and throwing while running ability, and (hopefully) a more limited but timely use of his terrific running skills, would likely make him easily franchise material. i seriously doubt he makes that pcent this season, but by year 3/4 he should be getting pretty darn close

hes a long ball and scrambling specialist. that CAN work, but you have to also be able to hit the routine bread & butter plays that keep drives alive. if you can get four yards on a first down pass, take it. your first down conversion chances go way up after that - especially compared to a longer ball attempt that falls incomplete

fortunately, it looks like the coaches are pounding that into him in the off-season. from mini-camp reports and the little bit of video i saw, they clearly were having him throw a ton of the screens, swings, short slants, etc. i expect we'll see that all through camp as the coaches try to force him to improve by instruction and sheer repetition

lets put it this way: if he cant get to 60% this season then the chances they are looking for a new qb in two years skyrocket. so be happy when he completes 3-4 yard passes in the preseason. the more of those, the better

212frawk · May 26, 2019

What does ESPN NFL analyst and Head of Football Outsiders know about the development of a QB? Schatz doesn't think Allen can improve as a player? Schatz is biased as he basses everything on stats without context. The fact Allen survived last year with no running game, with piss poor pass protection literally running for his life showed toughness and leadership and a will to win.

This analysis by Schatz is a complete joke. He is making the assumption that Allen can't improve because of his confirmation bias that he doesn't like Josh Allen. Schatz would rather stubbornly stick to his guns about his evaluation about Allen, than admit the young QB can improve.

holy shit this is the homeriest homerism out of you yet. Yea, Schatz is just some rando thats throwing darts against the wall

Methods To Our Madness

THE ULTRA-SHORT VERSION:

DVOA measures a team's efficiency by comparing success on every single play to a league average based on situation and opponent.

THE SHORT VERSION:

DVOA is a method of evaluating teams, units, or players. It takes every single play during the NFL season and compares each one to a league-average baseline based on situation. DVOA measures not just yardage, but yardage towards a first down: Five yards on third-and-4 are worth more than five yards on first-and-10 and much more than five yards on third-and-12. Red zone plays are worth more than other plays. Performance is also adjusted for the quality of the opponent. DVOA is a percentage, so a team with a DVOA of 10.0% is 10 percent better than the average team, and a quarterback with a DVOA of -20.0% is 20 percent worse than the average quarterback. Because DVOA measures scoring, defenses are better when they are negative. For more detail, read below.

Please feel free to contact us with questions and comments about our original statistics using the contact form.

DVOA EXPLAINED

DYAR EXPLAINED

SPECIAL TEAMS STATS

DRIVE STATS

ADJUSTED LINE YARDS and other offensive/defensive line stats

DVOA EXPLAINED

One running back runs for three yards. Another running back runs for three yards. Which is the better run? This sounds like a stupid question, but it isn’t. In fact, this question is at the heart of nearly all of the analysis on Football Outsiders.

Several factors can differentiate one three-yard run from another. What is the down and distance? Is it third-and-2 or second-and-15? Where on the field is the ball? Does the player get only three yards because he hits the goal line and scores? Is the player’s team up by two touchdowns in the fourth quarter, and thus running out the clock; or down by two touchdowns, and thus facing a defense that is playing purely against the pass? Is the running back playing against the porous defense of the Raiders, or the stalwart defense of the Bears?

Conventional NFL statistics value plays based solely on their net yardage. The NFL determines the best players by adding up all their yards no matter what situations they came in or how many plays it took to get them. Now, why would they do that? Football has one objective -- to get to the end zone -- and two ways to achieve that -- by gaining yards and achieving first downs. These two goals need to be balanced to determine a player’s value or a team’s performance. All the yards in the world won’t help a team win if they all come in six-yard chunks on third-and-10.

The popularity of fantasy football only exacerbates the problem. Fans have gotten used to judging players based on how much they help fantasy teams win and lose, not how much they help real teams win and lose. Typical fantasy scoring further skews things by counting the yard between the one and the goal line as 61 times more important than all the other yards on the field (each yard worth 0.1 points, a touchdown worth 6). Let’s say Larry Fitzgerald catches a pass on third-and-15 and goes 50 yards but gets tackled two yards from the goal line, and then Andre Ellington takes the ball on first-and-goal from the two-yard line and plunges in for the score. Has Ellington done something special? Not really. When an offense gets the ball on first-and-goal at the two-yard line, they're expected to score a touchdown five out of six times. Ellington is getting credit for the work done by the passing game.

Doing a better job of distributing credit for scoring points and winning games is the goal of DVOA, or Defense-adjusted Value Over Average. DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.

We then expand upon that basic idea with a more complicated system of “success points,” improved over the past few years with a lot of mathematics and a bit of trial and error. A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point. Interceptions occurring on fourth down during the last two minutes of a game incur no penalty whatsoever, but all others average -6 points, with an adjustment for the length of the pass and the location of the interception (since an interception tipped at the line is more likely to produce a long return than an interception on a 40-yard pass). A fumble is worth anywhere from -1.7 to -4.0 points depending on how often a fumble in that situation is lost to the defense -- no matter who actually recovers the fumble. Red zone plays get a bonus: 20 percent for team offense, five percent for team defense, and 10 percent for individual players. There is a bonus given for a touchdown, which acknowledges that the goal line is significantly more difficult to cross than the previous 99 yards (although this bonus is nowhere near as large as the one used in fantasy football).

(Our system is a bit more complex than the one in Hidden Game thanks to our subsequent research, which added larger penalties for turnovers, the fractional points, and a slightly higher baseline for success on first down. The reason why all fumbles are counted, no matter whether they are recovered by the offense or defense, is explained in FO Basics.)

Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players, adjusted for a number of variables. These include down and distance, field location, time remaining in game, and the team’s lead or deficit in the game score. Teams are always compared to the overall offensive average, as the team made its own choice whether to pass or rush. When it comes to individual players, however, rushing plays are compared to other rushing plays, passing plays to other passing plays, tight ends to tight ends, wideouts to wideouts, and so on.

Going back to our example of the three-yard rush, if Player A gains three yards under a set of circumstances in which the average NFL running back gains only one yard, then Player A has a certain amount of value above others at his position. Likewise, if Player B gains three yards on a play on which, under similar circumstances, an average NFL back gains four yards, that Player B has negative value relative to others at his position. Once we make all our adjustments, we can evaluate the difference between this player’s rate of success and the expected success rate of an average running back in the same situation (or between the opposing defense and the average defense in the same situation, etc.). Add up every play by a certain team or player, divide by the total of the various baselines* for success in all those situations, and you get VOA, or Value Over Average.

The biggest variable in football is the fact that each team plays a different schedule against teams of disparate quality. By adjusting each play based on the opposing defense’s average success in stopping that type of play over the course of a season, we get DVOA, or Defense-adjusted Value Over Average. Rushing and passing plays are adjusted based on down and location on the field; passing plays are also adjusted based on how the defense performs against passes to running backs, tight ends, or wide receivers. Defenses are adjusted based on the average success of the offenses they are facing. (Yes, technically the defensive stats are actually “offense-adjusted.” If it seems weird, think of the “D” in “DVOA” as standing for “opponent-Dependent” or something.)

The final step in calculating DVOA involves normalizing each year's ratings. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all time. Therefore, we need to ensure that DVOA in a given season isn't skewed by league environment.

For teams, DVOA is normalized so that league averages for offense and defense are 0%. (However, because pass plays are more efficient than run plays, league averages for team passing and team rushing are not zero.) For players, DVOA is normalized separately for individual passing, individual rushing, and the three individual receiving groups (wide receivers, tight ends, and running backs) so that the league average for each is 0%.

Of course, one of the hardest parts of understanding a new statistic is interpreting its scale. To use DVOA, you have to know what numbers represent good performance and what numbers represent bad performance. We’ve made that easy. In all cases, 0% represents league-average. A positive DVOA represents a situation that favors the offense, while a negative DVOA represents a situation that favors the defense. This is why the best offenses have positive DVOA ratings (last year, Green Bay led the league at +24.7%) and the best defenses have negative DVOA ratings (with Seattle number one in 2014 at -16.8%). In most years, the best and worst offenses tend to rate around ± 30%, while the best and worst defenses tend to rate around ± 25%. For starting players, the scale tends to reach roughly ± 40% for passing and receiving, and ± 30% for rushing. As you might imagine, some players with fewer attempts will surpass both extremes.

DVOA has three main advantages over more traditional ways to judge NFL performance. First, by subtracting defense DVOA from offense DVOA (and adding in special teams DVOA, which is described below), we can create a set of team rankings that's based on play-by-play efficiency rather than total yards. Because DVOA does a better job of explaining past wins and predicting future wins than total yards, it gives a more accurate picture of how much better (or worse) a team really is relative to the rest of the league.

Because it compares each play only to plays with similar circumstances, this advantage also applies vis-a-vis situational team rankings. The list of top DVOA offenses on third down, for example, is more accurate than the conventional NFL conversion statistic because it takes into account that converting third-and-long is more difficult than converting third-and-short, and that a turnover is worse than an incomplete pass because it eliminates the opportunity to move the other team back with a punt on fourth down. The same could be said about plays on fourth down or in the red zone.

Second, unlike formulas based on comparing drives rather than individual plays, DVOA can be separated into a myriad of splits (e.g., by down, by week, by distance needed for a first down, etc.). Therefore, we're able to break teams and players down to find strengths and weaknesses in a variety of situations. All Pittsburgh third downs can be compared to how an average team does on third down. Josh McCown and Mike Glennon can each be compared to how an average quarterback performs in the red zone, or with a lead, or in the second half of the game. This doesn't just give us a better idea of which team or player is better. More importantly, it helps us understand why they're better, and therefore allows us to offer prescriptions for improvement in the future.

Finally, a third advantage of DVOA is that normalization makes our comparisons of current teams and players to past teams and players (going back to 1989) more accurate than those based on traditional statistics like wins or total yards, as well as those based on more sophisticated metrics that aren't normalized (e.g., expected points added, passer rating differential, etc.). For instance, which Denver Broncos team had the better offense: the 2013 edition with Peyton Manning, or the 1998 club led by Terrell Davis? Going by total yardage (7,317 vs. 6,092) or even yards per play (6.3 vs. 5.9), it's not even a contest. The 2013 team were clearly better. However, this ignores the fact that the average NFL offense was much more pass-oriented, and thus more efficient, in 2013 than in 1998. If we take the difference in offensive environment into account by using DVOA, it turns out that the 1998 Broncos offense was slightly better relative to the rest of the league (34.5% to 33.5%).

*It should be noted that certain plays are included in DVOA for offense but not for defense. Other plays are included for both, but scored differently. This leads to separate baselines on each side of the ball. For instance

Only four total penalties are included. Two penalties count as pass plays on both sides of the ball: intentional grounding and defensive pass interference. The other two penalties are included for offense only: false starts and delay of game. Because the inclusion of these penalties means a group of negative plays that don’t count as either passes or runs, the league averages for pass offense and run offense are higher than the league averages for pass defense and run defense.

Aborted snaps and incomplete backwards lateral passes are only penalized on offense, not rewarded on defense.

Adjustments for playing from behind or with a lead in the fourth quarter are different for offense and defense, as are adjustments for the final two minutes of the first half when the offense is not near field-goal range.

Offense gets a slight penalty and defense gets a slight bonus for games indoors.

DYAR EXPLAINED

After using DVOA for a few months, we came across a strange phenomenon: well-regarded players, particularly those known for their durability, had DVOA ratings that came out around average. The reason is that DVOA, by virtue of being a percentage or rate statistic, doesn’t take into account the cumulative value of having a player producing at a league-average level over the course of an above-average number of plays. By definition, an average level of performance is better than that provided by half of the league and the ability to maintain that level of performance while carrying a heavy workload is very valuable indeed. In addition, a player who is involved in a high number of plays can draw the defense’s attention away from other parts of the offense, and, if that player is a running back, he can take time off the clock with repeated runs.

Let’s say you have a running back who carries the ball 300 times in a season. What would happen if you were to remove this player from his team’s offense? What would happen to those 300 plays? Those plays don’t disappear with the player, though some might be lost to the defense because of the associated loss of first downs. Rather those plays would have to be distributed among the remaining players in the offense, with the bulk of them being given to a replacement running back. This is where we arrive at the concept of replacement level, borrowed from our partners at Baseball Prospectus. When a player is removed from an offense, he is usually not replaced by a player of similar ability. Nearly every starting player in the NFL is a starter because he is better than the alternative. Those 300 plays will typically be given to a significantly worse player, someone who is the backup because he doesn’t have as much experience and/or talent. A player’s true value can then be measured by the level of performance he provides above that replacement level baseline, totaled over all of his run or pass attempts.

Of course, the real replacement player is different for each team in the NFL. In 2013, the second-string running back in Minnesota (Jerick McKinnon) had a higher DVOA than the alleged starter (Matt Asiata). Sometimes a player like Ryan Grant or Danny Woodhead will be cut by one team and turn into a star for another. On other teams, the drop from the starter to the backup can be even greater than the general drop to replacement level. (The Indianapolis Colts of 2011--the dark year between the Manning and Luck eras--will be the hallmark example of this until the end of time.) The choice to start an inferior player or to employ a sub-replacement level backup, however, falls to the team, not the starter being evaluated. Thus, we generalize replacement level for the league as a whole, as the ultimate goal is to evaluate players independent of the quality of their teammates.

Our estimates of replacement level were re-done during the 2008 season and are computed differently for each position. For quarterbacks, we analyzed situations where two or more quarterbacks had played meaningful snaps for a team in the same season, then compared the overall DVOA of the original starters to the overall DVOA of the replacements. We did not include situations where the backup was actually a top prospect waiting his turn on the bench, since a first-round pick is by no means a "replacement-level" player.

At other positions, there is no easy way to separate players into "starters" and "replacements," since unlike at quarterback, being the starter doesn't make you the only guy who gets in the game. Instead, we used a simpler method, ranking players at each position in each season by attempts. The players who made up the final 10 percent of passes or runs were split out as "replacement players" and then compared to the players making up the other 90 percent of plays at that position. This took care of the fact that not every non-starter at running back or wide receiver is a freely available talent. (Think of Jonathan Stewart or Randall Cobb, for example.)

As noted earlier, the challenge of any new stat is to present it on a scale that’s meaningful to those attempting to use it. Saying that Tony Romo's passes were worth 40 success value points over replacement in 2014 has very little value without a context to tell us if 40 is good total or a bad one. Therefore, we translate these success values into a number called "Defense-adjusted Yards Above Replacement, or DYAR. Thus, Romo was fifth among quarterbacks with 1,187 passing DYAR. It is our estimate that a generic replacement-level quarterback, throwing in the same situations as Romo, would have been worth 1,187 fewer yards. Note that this doesn’t mean the replacement level quarterback would have gained exactly 1,187 fewer yards. First downs, touchdowns, and turnovers all have an estimated yardage value in this system, so what we are saying is that a generic replacement-level quarterback would have fewer yards and touchdowns (and more turnovers) that would total up to be equivalent to the value of 1,187 yards.

(Note: Prior to the 2008 season, DYAR was translated in terms of points rather than yardage, and old articles will refer to these stats as "DPAR" instead.)

HOW CAN A 16-GAME SEASON BE SIGNIFICANT?

Football statistics can't be analyzed in the same way baseball statistics are. After all, there are only 16 games in a season. Baseball has over ten times more, and even the NBA and NHL offer over five times more. The more games, the more events to analyze, and the more events to analyze, the more statistical significance.

That is true, but the trick is to consider each play in an NFL game as a separate event. For example, Drew Brees played only 16 games in 2014, but in those 16 games he had 692 passing plays (including sacks) and 16 rushing plays (including scrambles) for a total of 708 events. Ian Kinsler in 2014 played in 161 games and had 726 plate appearances. For the most part, a quarterback who plays a full season will have almost the same number of plays as a baseball hitter who plays in most of his team's games.

A running back will have fewer plays than a quarterback, and wide receivers and tight ends will have even fewer. But there should still be enough plays with most starting running backs and receivers to allow for analysis with some significance. As an example, DeMarco Murray ran the ball 392 times in 2014, and was the target of 64 passes (including incompletes), for a total of 456 plays. In general, a starting running back will have 375-450 plays over 16 games. Receivers are used a bit less, and therefore their stats are likely not as accurate. In general, starting wide receivers have 75-150 pass targets over a full season.

ISSUES WITH DVOA/DYAR

DVOA is limited by what’s included in the official NFL play-by-play or tracked by the Football Outsiders game charting project. Because we need to have the entire play-by-play of a season in order to compute DVOA and DYAR, these metrics are not yet ready to compare players of today to players throughout the league’s history. As of this writing, we have processed 26 seasons, 1989 through 2014, and we add seasons at a rate of roughly two per year (the most recent season, plus one season back into history.)

Football is a game in which nearly every action requires the work of two or more teammates -- in fact, usually 11 teammates all working in unison. Unfortunately, when it comes to individual player ratings, we are still far from the point at which we can determine the value of a player independent from the performance of his teammates. That means that when we say, "In 2014, Marshawn Lynch had a DVOA of 23.1%, what we are really saying is “In 2014, Marshawn Lynch, playing in Darrell Bevell’s offensive system with the Seattle offensive line blocking for him and Russell Wilson selling the keeper when necessary, had a DVOA of 23.1%."

With fewer situations to measure, the numbers spread out a bit more, so you'll see more extreme DVOA ratings for part-time players and for measurements of teams in more specific situations (for example, passing on third downs). The charts listing players in order of DVOA have cut-offs for number of attempts, because players with just a handful of plays end up with absurd VOA and DVOA numbers. (In 2014, for instance, Johnny Manziel had a -73.2% passing DVOA in 38 plays.)

Passing statistics include sacks as well as fumbles on aborted snaps. Receiving statistics include all passes intended for the receiver in question, including those that are incomplete or intercepted. At some point, we hope to be able to determine just how much impact different receivers have on completes vs. incomplete passes, but various regression analyses make it clear that both quarterback and receiver have an impact on whether a pass is complete or not. The word passes refers to both complete and incomplete pass attempts.

Unless we say otherwise, all references to third down also include the handful of rushing and passing plays that take place on fourth down (primarily fourth-and-1).

DVOA FOR SPECIAL TEAMS

The problem with a system based on measuring both yardage and yardage towards a first down, of course, is what to do with plays that don't have the possibility of a first down. Special teams are an important part of football and we needed a way to add that performance to the team DVOA ranking. Our special teams metric includes five separate measurements: field goals (and extra points), net punting, punt returns, net kickoffs, and kick returns.

The foundation of most of these special teams ratings is the concept that each yard line has a different value based on how the likelihood of scoring changes with better field position. In Hidden Game, the authors suggested that the value of field position for the offense existed on a straight line with your own goal line being worth -2 points, the 50-yard line 2 points, and the opposing goal line 6 points. (-2 points isn't just the value of a safety; it also reflects the fact that when you are backed up in your own zone, you are likely going to see your drive stall, and you'll need to punt and give the ball to the other team in good field position. Thus, the defense is more likely to score next.) We use a more refined set of values based on our research, but the idea is the same.

Our special teams ratings compare each kick or punt to the league average for based on the point value of field position at the position of each kick, catch, and return. We've determined a league average for how far a kick goes based on the yard line from where the kick occurs (almost always the 35-yard line for kickoffs, variable for punts) and a league average for how far a return goes based on both the yard line where the ball is caught and the distance that it traveled in the air.

The kicking or punting team is rated based on net points compared to average, taking into account both the kick and the return if there is one. Because the average return is always positive, punts that are not returnable (touchbacks, out of bounds, fair catches, and punts downed by the coverage unit) will rate higher than punts of the same distance which are returnable. (This is also true of touchbacks on kickoffs.) There are also separate individual ratings for kickers and punters that are based only on distance and whether the kick is returnable, otherwise assuming an average return in order to judge the kicker separate from the coverage.

For the return team, the rating is based on how many points the return is worth compared to average, based on the location of the catch and the distance the ball traveled in the air. Return teams are not judged on the distance of kicks, nor are they judged on kicks that cannot be returned. As explained below, blocked kicks are so rare as to be statistically insignificant as predictors for future performance and are thus ignored.

Field goal kicking is measured differently. Measuring kickers by field goal percentage is a bit absurd, as it assumes that all field goals are of equal difficulty. In our metric, each field goal is compared to the average number of points scored on all field goal attempts from that distance over the past 15 years. The value of a field goal increases as distance from the goal line increases.

Kickoffs, punts, and field goals are then adjusted based on weather and altitude. It will surprise no one to learn that it is easier to kick the ball in Denver or a dome than it is to kick the ball in Buffalo in December. Because we do not yet have enough data to tailor our adjustments specifically to each stadium, each one is assigned to one of four categories: Cold, Warm, Dome, and Denver. There is also an additional adjustment dropping the value of field goals in Florida (because the warm temperatures allow the ball to carry better) and raising the value of punts in San Francisco (because of those infamous winds).

The baselines for special teams are adjusted in each year for rule changes such as the introduction of the special-teams-use-only “k-ball” in 1999 as well as the move of the kickoff line from the 35 to the 30 in 1994 and then back to the 35 in 2011. Baselines have also been adjusted each year to make up for the gradual improvement of kickers over the last two decades.

Once we’ve totaled how many points above or below average can be attributed to special teams, we translate those points into DVOA so the ratings can be added to offense and defense to get total team DVOA. As a final step, we then normalize special teams DVOA to reflect the league environment in a given year. It should be noted, however, that we don’t yet have a method to perfectly normalize each year of special teams, so the league average for special teams deviates from 0.0% in certain years, although not by more than 0.5%.

There are three aspects of special teams that have an impact on wins and losses, but don’t show up in the standard special teams rating because a team has little or no influence on them. The first is the length of kick¬offs by the opposing team, with an asterisk. Obviously, there are no defenders standing on the 35-yard line, ready to block a kickoff after the whistle blows. However, over the past few years, some teams have delib¬erately kicked short in order to avoid certain top return men, such as Devin Hester and Josh Cribbs. The special teams formula now includes adjustments to give teams extra credit for field position on kick returns if kickers are deliberately trying to avoid a return.

The other two items that special teams have little control over are field goals against your team, and punt distance against your team. Research shows no indication that teams can influence the accuracy or strength of field-goal kickers and punters, except for blocks. As mentioned above, although blocked field goals and punts are definitely skillful plays, they are so rare that they have no correlation to how well teams have played in the past or will play in the future, thus they are included here as if they were any other missed field goal or botched punt, giving the defense no additional credit for their efforts. The value of these three elements is listed separately as “Hidden” value.

Special teams ratings also do not include two-point conversions or onside kick attempts, both of which, like blocks, are so infrequent as to be statistically insignificant in judging future performance.

ADJUSTED LINE YARDS EXPLAINED

One exception to the use of DVOA/DYAR, and the use of "play success" instead of raw yardage, is the rating system for offensive and defensive lines. Actually, these are only measures of running plays, and of course the defensive numbers don't measure just the defensive line, but the whole front seven against the run.

One of the most difficult goals of statistical analysis in football is somehow isolating how much responsibility for a play lies with each of the 22 men on the field. Nowhere is this as obvious as the running game, where one player runs while up to nine other players -- including wideouts, tight ends, and fullback -- block in different directions. None of the statistics we use for measuring rushing -- yards, touchdowns, yards per carry -- differentiate between the contribution of the running back and the contribution of the offensive line. Neither do our advanced metrics DVOA and DYAR.

We have enough data amassed that we can try to separate the effect that the running back has on a particular play from the effect of the offensive line (and other offensive blockers) and the effect of the defense. A team might have two running backs in its stable: RB A, who averages 3.0 yards per carry, and RB B, who averages 3.5 yards per carry. Who is the better back? Imagine that RB A doesn't just average 3.0 yards per carry, but gets exactly 3 yards on every single carry, while RB B has a highly variable yardage output: sometimes 5 yards, sometimes -2 yards, sometimes 20 yards. The difference in variability between the runners can be exploited to not only determine the difference between the runners, but the effect the offensive line has on every running play.

We know that at some point in every long running play, the running back has gotten past all of his offensive line blocks. From here on, the rest of the play is dependent on the runner's own speed and elusiveness, combined with the speed and tackling ability of the defensive players. If Frank Gore breaks through the line for 50 yards, avoiding tacklers all the way to the goal line, his offensive line has done a great job -- but they aren't responsible for most of that run. How much are they responsible for?

For each running back carry, we calculated the probability that the back involved would run for the specific yardage on that play, based on that back's average yardage per carry and the variability of their yardage on every play. We also calculated the probability that the offense would get the yardage based on the team's rushing average and variability without the back involved in the play, and the probability that the defense would give up the specific amount of yardage based on its average rushing yards allowed per carry and variability. For example, based on his rushing average and variability, the probability in 2004 that Tiki Barber would have a positive carry was 80% while the probability that Giants would have a positive carry without Barber running was only 73%.

Yardage ends up falling into roughly the following combinations: Losses, 0-4 yards, 5-10 yards, and 11+ yards. In general, the offensive line is 20% more responsible for lost yardage than it is for yardage gained up to four yards, but 50% less responsible for yardage gained from 5-10 yards, and not responsible for yardage past that. Thus, the creation of Adjusted Line Yards.

Adjusted Line Yards take every carry by a running back and apply those percentages. (We don’t include carries by receivers, which are usually based on deception rather than straight blocking, or carries by quarterbacks, which are almost always busted passing plays unless they involve someone like Colin Kaepernick or Cam Newton.) Those numbers are then adjusted based on down, distance, situation, opponent and whether or not a team is in the shotgun. (Because defenses are generally playing pass when the quarterback is in shotgun, the average running back carry from shotgun last year gained 5.36 yards, compared to just 4.16 yards on other carries.) The adjusted numbers are then normalized so that the league average for Adjusted Line Yards per carry is the same as the league average for RB yards per carry (in 2013, 4.10 yards).

Runs are listed by the NFL in seven different directions: left/right end, left/right tackle, left/right guard, and middle. Further research showed no statistically significant difference between how well a team performed on runs listed middle, left guard, and right guard, so we also list runs separated into five different directions. Note that there may not be a statistically significant difference between right tackle and middle/guard either, but until we can research further (and for the sake of symmetry) we do still split out runs behind the right tackle separately.

These splits allow us to evaluate subsections of a team’s offensive line, but not necessarily individual linemen, as we can’t account for blocking assignments. We don't know when a guard is pulling and when a guard is blocking straight ahead. We know that some runners are just inherently better going up the middle, and some are better going side to side, and we can't measure how much that impacts these numbers. We have no way of knowing the blocking contribution made by fullbacks, tight ends, or wide receivers.

Other numbers we use to measure the running game:

Open Field Yards gives the portion of the team's rushing average gained after the first 10 yards of each run. So for a 10-yard run, no yards are counted; for a 15-yard run, five yards are counted; for an 80-yard run, 70 yards are counted. This number gives you an idea of how much of a team's running game was based on the breakaway speed of the running backs. A team with a low ranking in Adjusted Line Yards but a high ranking in Open Field Yards is heavily dependent on its running back breaking long runs to make the running game work. This number is not adjusted in any way.

Second Level Yards gives the portion of the team's rushing average gained 5-10 yards past the line of scrimmage. So for a five-yard run, no yards are counted; for a 10-yard run, five yards are counted; for an 80-yard run, still only five yards are counted. This number represents the midpoint between the contributions of a team's offensive line and its running back. A low ranking in Open Field Yards coupled with a high ranking in Second Level Yards indicates that the team's offensive line is opening holes, but its backs aren't proficient open-field runners. This number is not adjusted in any way.

Power success measures the success of specific running plays rather than the distance. This number represents how often a running attempt on third or fourth down, with two yards or less to go, achieved a first down or touchdown. Since quarterback sneaks, unlike scrambles, are heavily dependent on the offensive line, this percentage does include runs by all players, not just running backs. This is the only stat given that includes quarterback runs. It is not adjusted based on game situation or opponent.

Stuffed measures the percentage of runs that are stopped at or before the line of scrimmage.

DRIVE STATS

The stats section of our website also features drive stats compiled by Jim Armstrong. These stats are computed from NFL Drive Charts and are not adjusted for strength of schedule or situation. Take-a-knee drives at the end of halves are discarded. Drive stats are generally self-explanatory, giving each team's total number of drives as well as average yards per drive, points per drive, touchdowns per drive, punts per drive, and turnovers per drive, interceptions per drive, and fumbles lost per drive. LOS/Drive represents average starting field position (line of scrimmage) per drive from the offensive point of view. Drive stats are given for offense and defense, with NET representing simply offense minus defense.

A NOTE ON PLAY-BY-PLAY DATA

Our data may differ slightly from official NFL numbers due to discrepancies in different play-by-play reports. In addition, we've adjusted clock plays, with kneels no longer counting as rush attempts and spikes no longer counting as pass attempts. We also count most aborted snaps as passing plays, not rushing plays, unless the play-by-play specifies that the play was an aborted handoff.

Data Provided By:Data Provided By Sports Info Solutions

HipKat · May 26, 2019

The problem is that where they were going to be before those moves is not as good as I think where Bills fans think they would be before those moves. So even after making a lot of good moves, I’m not that optimistic about them.”

Is this guy an idiot?? Why the HELL does he think they MADE all of those moves. We knew they were not good coming out of last season. No one that's a Bills fan thought any different and as for Allen - improvced as the season went on, that's what I wanted to see and that's grounds for being optimistic.

Figures - an espn schmuck

Bocephus · May 26, 2019

He might need to take a look at that Cover 1 article breaking down Allen's play before making these statements.

I do not see a NFL starting QB in Buffalo Bills’ Josh Allen, says ESPN’s Aaron Schatz

Many national NFL analysts are starting to come around on the Buffalo Bills. Don’t count Aaron Schatz, ESPN NFL analyst and Head of Football Outsiders, among them.

Schatz appeared on WGR-550′s Schopp and the Bulldog on Friday to discuss the NFL offseason. Along the way, Schatz was asked about the Bills and the sudden amount of optimism surrounding the team. When asked if he was among those who were optimistic about Buffalo, Schatz stated he was not despite the team making some good moves in free agency.

“I’ve noticed a lot of optimism about the moves that the Bills have made in the offseason," Schatz said. "That they’ve made a lot of good moves. And here’s the thing, I think they’ve made a lot of good moves. The problem is that where they were going to be before those moves is not as good as I think where Bills fans think they would be before those moves. So even after making a lot of good moves, I’m not that optimistic about them.”

Perhaps the biggest reason why Schatz does not believe in Buffalo is due to quarterback Josh Allen. Despite leading the Bills to five of their six victories last season, Schatz told Schopp and the Bulldog that he doesn’t view Allen as a NFL starting quarterback.

“I’m just not a Josh Allen believer. I do not think there is an NFL starting quarterback in there waiting to come out and develop. And if I’m wrong then, you know, they’ve done a lot to give him help, but if I’m right then they’re still in trouble.”

Schatz isn’t the first NFL analyst to be down on Allen and he certainly won’t be the last, but the quarterback improved significantly when he returned from an elbow injury. By the end of his rookie year, Allen passed for 2074 yards and added 631 yards on the ground in 12 games (11 starts). He finished the year with 18 total touchdowns and 12 interceptions.

In his second season, Allen will be expected to make strides on these numbers, as well as his completion percentage (52.8). Buffalo made several upgrades along the offensive line to protect the quarterback and his wide receivers are much improved from one season ago.

Fuck Schatz. Who?

Bocephus · May 26, 2019

holy shit this is the homeriest homerism out of you yet. Yea, Schatz is just some rando thats throwing darts against the wall

Methods To Our Madness
THE ULTRA-SHORT VERSION:

DVOA measures a team's efficiency by comparing success on every single play to a league average based on situation and opponent.
THE SHORT VERSION:

DVOA is a method of evaluating teams, units, or players. It takes every single play during the NFL season and compares each one to a league-average baseline based on situation. DVOA measures not just yardage, but yardage towards a first down: Five yards on third-and-4 are worth more than five yards on first-and-10 and much more than five yards on third-and-12. Red zone plays are worth more than other plays. Performance is also adjusted for the quality of the opponent. DVOA is a percentage, so a team with a DVOA of 10.0% is 10 percent better than the average team, and a quarterback with a DVOA of -20.0% is 20 percent worse than the average quarterback. Because DVOA measures scoring, defenses are better when they are negative. For more detail, read below.

Please feel free to contact us with questions and comments about our original statistics using the contact form.

DVOA EXPLAINED
DYAR EXPLAINED
SPECIAL TEAMS STATS
DRIVE STATS
ADJUSTED LINE YARDS and other offensive/defensive line stats

DVOA EXPLAINED

One running back runs for three yards. Another running back runs for three yards. Which is the better run? This sounds like a stupid question, but it isn’t. In fact, this question is at the heart of nearly all of the analysis on Football Outsiders.

Several factors can differentiate one three-yard run from another. What is the down and distance? Is it third-and-2 or second-and-15? Where on the field is the ball? Does the player get only three yards because he hits the goal line and scores? Is the player’s team up by two touchdowns in the fourth quarter, and thus running out the clock; or down by two touchdowns, and thus facing a defense that is playing purely against the pass? Is the running back playing against the porous defense of the Raiders, or the stalwart defense of the Bears?

Conventional NFL statistics value plays based solely on their net yardage. The NFL determines the best players by adding up all their yards no matter what situations they came in or how many plays it took to get them. Now, why would they do that? Football has one objective -- to get to the end zone -- and two ways to achieve that -- by gaining yards and achieving first downs. These two goals need to be balanced to determine a player’s value or a team’s performance. All the yards in the world won’t help a team win if they all come in six-yard chunks on third-and-10.

The popularity of fantasy football only exacerbates the problem. Fans have gotten used to judging players based on how much they help fantasy teams win and lose, not how much they help real teams win and lose. Typical fantasy scoring further skews things by counting the yard between the one and the goal line as 61 times more important than all the other yards on the field (each yard worth 0.1 points, a touchdown worth 6). Let’s say Larry Fitzgerald catches a pass on third-and-15 and goes 50 yards but gets tackled two yards from the goal line, and then Andre Ellington takes the ball on first-and-goal from the two-yard line and plunges in for the score. Has Ellington done something special? Not really. When an offense gets the ball on first-and-goal at the two-yard line, they're expected to score a touchdown five out of six times. Ellington is getting credit for the work done by the passing game.

Doing a better job of distributing credit for scoring points and winning games is the goal of DVOA, or Defense-adjusted Value Over Average. DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.

We then expand upon that basic idea with a more complicated system of “success points,” improved over the past few years with a lot of mathematics and a bit of trial and error. A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point. Interceptions occurring on fourth down during the last two minutes of a game incur no penalty whatsoever, but all others average -6 points, with an adjustment for the length of the pass and the location of the interception (since an interception tipped at the line is more likely to produce a long return than an interception on a 40-yard pass). A fumble is worth anywhere from -1.7 to -4.0 points depending on how often a fumble in that situation is lost to the defense -- no matter who actually recovers the fumble. Red zone plays get a bonus: 20 percent for team offense, five percent for team defense, and 10 percent for individual players. There is a bonus given for a touchdown, which acknowledges that the goal line is significantly more difficult to cross than the previous 99 yards (although this bonus is nowhere near as large as the one used in fantasy football).

(Our system is a bit more complex than the one in Hidden Game thanks to our subsequent research, which added larger penalties for turnovers, the fractional points, and a slightly higher baseline for success on first down. The reason why all fumbles are counted, no matter whether they are recovered by the offense or defense, is explained in FO Basics.)

Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players, adjusted for a number of variables. These include down and distance, field location, time remaining in game, and the team’s lead or deficit in the game score. Teams are always compared to the overall offensive average, as the team made its own choice whether to pass or rush. When it comes to individual players, however, rushing plays are compared to other rushing plays, passing plays to other passing plays, tight ends to tight ends, wideouts to wideouts, and so on.

Going back to our example of the three-yard rush, if Player A gains three yards under a set of circumstances in which the average NFL running back gains only one yard, then Player A has a certain amount of value above others at his position. Likewise, if Player B gains three yards on a play on which, under similar circumstances, an average NFL back gains four yards, that Player B has negative value relative to others at his position. Once we make all our adjustments, we can evaluate the difference between this player’s rate of success and the expected success rate of an average running back in the same situation (or between the opposing defense and the average defense in the same situation, etc.). Add up every play by a certain team or player, divide by the total of the various baselines* for success in all those situations, and you get VOA, or Value Over Average.

The biggest variable in football is the fact that each team plays a different schedule against teams of disparate quality. By adjusting each play based on the opposing defense’s average success in stopping that type of play over the course of a season, we get DVOA, or Defense-adjusted Value Over Average. Rushing and passing plays are adjusted based on down and location on the field; passing plays are also adjusted based on how the defense performs against passes to running backs, tight ends, or wide receivers. Defenses are adjusted based on the average success of the offenses they are facing. (Yes, technically the defensive stats are actually “offense-adjusted.” If it seems weird, think of the “D” in “DVOA” as standing for “opponent-Dependent” or something.)

The final step in calculating DVOA involves normalizing each year's ratings. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all time. Therefore, we need to ensure that DVOA in a given season isn't skewed by league environment.

For teams, DVOA is normalized so that league averages for offense and defense are 0%. (However, because pass plays are more efficient than run plays, league averages for team passing and team rushing are not zero.) For players, DVOA is normalized separately for individual passing, individual rushing, and the three individual receiving groups (wide receivers, tight ends, and running backs) so that the league average for each is 0%.

Of course, one of the hardest parts of understanding a new statistic is interpreting its scale. To use DVOA, you have to know what numbers represent good performance and what numbers represent bad performance. We’ve made that easy. In all cases, 0% represents league-average. A positive DVOA represents a situation that favors the offense, while a negative DVOA represents a situation that favors the defense. This is why the best offenses have positive DVOA ratings (last year, Green Bay led the league at +24.7%) and the best defenses have negative DVOA ratings (with Seattle number one in 2014 at -16.8%). In most years, the best and worst offenses tend to rate around ± 30%, while the best and worst defenses tend to rate around ± 25%. For starting players, the scale tends to reach roughly ± 40% for passing and receiving, and ± 30% for rushing. As you might imagine, some players with fewer attempts will surpass both extremes.

DVOA has three main advantages over more traditional ways to judge NFL performance. First, by subtracting defense DVOA from offense DVOA (and adding in special teams DVOA, which is described below), we can create a set of team rankings that's based on play-by-play efficiency rather than total yards. Because DVOA does a better job of explaining past wins and predicting future wins than total yards, it gives a more accurate picture of how much better (or worse) a team really is relative to the rest of the league.

Because it compares each play only to plays with similar circumstances, this advantage also applies vis-a-vis situational team rankings. The list of top DVOA offenses on third down, for example, is more accurate than the conventional NFL conversion statistic because it takes into account that converting third-and-long is more difficult than converting third-and-short, and that a turnover is worse than an incomplete pass because it eliminates the opportunity to move the other team back with a punt on fourth down. The same could be said about plays on fourth down or in the red zone.

Second, unlike formulas based on comparing drives rather than individual plays, DVOA can be separated into a myriad of splits (e.g., by down, by week, by distance needed for a first down, etc.). Therefore, we're able to break teams and players down to find strengths and weaknesses in a variety of situations. All Pittsburgh third downs can be compared to how an average team does on third down. Josh McCown and Mike Glennon can each be compared to how an average quarterback performs in the red zone, or with a lead, or in the second half of the game. This doesn't just give us a better idea of which team or player is better. More importantly, it helps us understand why they're better, and therefore allows us to offer prescriptions for improvement in the future.

Finally, a third advantage of DVOA is that normalization makes our comparisons of current teams and players to past teams and players (going back to 1989) more accurate than those based on traditional statistics like wins or total yards, as well as those based on more sophisticated metrics that aren't normalized (e.g., expected points added, passer rating differential, etc.). For instance, which Denver Broncos team had the better offense: the 2013 edition with Peyton Manning, or the 1998 club led by Terrell Davis? Going by total yardage (7,317 vs. 6,092) or even yards per play (6.3 vs. 5.9), it's not even a contest. The 2013 team were clearly better. However, this ignores the fact that the average NFL offense was much more pass-oriented, and thus more efficient, in 2013 than in 1998. If we take the difference in offensive environment into account by using DVOA, it turns out that the 1998 Broncos offense was slightly better relative to the rest of the league (34.5% to 33.5%).

*It should be noted that certain plays are included in DVOA for offense but not for defense. Other plays are included for both, but scored differently. This leads to separate baselines on each side of the ball. For instance

Only four total penalties are included. Two penalties count as pass plays on both sides of the ball: intentional grounding and defensive pass interference. The other two penalties are included for offense only: false starts and delay of game. Because the inclusion of these penalties means a group of negative plays that don’t count as either passes or runs, the league averages for pass offense and run offense are higher than the league averages for pass defense and run defense.
Aborted snaps and incomplete backwards lateral passes are only penalized on offense, not rewarded on defense.
Adjustments for playing from behind or with a lead in the fourth quarter are different for offense and defense, as are adjustments for the final two minutes of the first half when the offense is not near field-goal range.
Offense gets a slight penalty and defense gets a slight bonus for games indoors.

DYAR EXPLAINED

After using DVOA for a few months, we came across a strange phenomenon: well-regarded players, particularly those known for their durability, had DVOA ratings that came out around average. The reason is that DVOA, by virtue of being a percentage or rate statistic, doesn’t take into account the cumulative value of having a player producing at a league-average level over the course of an above-average number of plays. By definition, an average level of performance is better than that provided by half of the league and the ability to maintain that level of performance while carrying a heavy workload is very valuable indeed. In addition, a player who is involved in a high number of plays can draw the defense’s attention away from other parts of the offense, and, if that player is a running back, he can take time off the clock with repeated runs.

Let’s say you have a running back who carries the ball 300 times in a season. What would happen if you were to remove this player from his team’s offense? What would happen to those 300 plays? Those plays don’t disappear with the player, though some might be lost to the defense because of the associated loss of first downs. Rather those plays would have to be distributed among the remaining players in the offense, with the bulk of them being given to a replacement running back. This is where we arrive at the concept of replacement level, borrowed from our partners at Baseball Prospectus. When a player is removed from an offense, he is usually not replaced by a player of similar ability. Nearly every starting player in the NFL is a starter because he is better than the alternative. Those 300 plays will typically be given to a significantly worse player, someone who is the backup because he doesn’t have as much experience and/or talent. A player’s true value can then be measured by the level of performance he provides above that replacement level baseline, totaled over all of his run or pass attempts.

Of course, the real replacement player is different for each team in the NFL. In 2013, the second-string running back in Minnesota (Jerick McKinnon) had a higher DVOA than the alleged starter (Matt Asiata). Sometimes a player like Ryan Grant or Danny Woodhead will be cut by one team and turn into a star for another. On other teams, the drop from the starter to the backup can be even greater than the general drop to replacement level. (The Indianapolis Colts of 2011--the dark year between the Manning and Luck eras--will be the hallmark example of this until the end of time.) The choice to start an inferior player or to employ a sub-replacement level backup, however, falls to the team, not the starter being evaluated. Thus, we generalize replacement level for the league as a whole, as the ultimate goal is to evaluate players independent of the quality of their teammates.

Our estimates of replacement level were re-done during the 2008 season and are computed differently for each position. For quarterbacks, we analyzed situations where two or more quarterbacks had played meaningful snaps for a team in the same season, then compared the overall DVOA of the original starters to the overall DVOA of the replacements. We did not include situations where the backup was actually a top prospect waiting his turn on the bench, since a first-round pick is by no means a "replacement-level" player.

At other positions, there is no easy way to separate players into "starters" and "replacements," since unlike at quarterback, being the starter doesn't make you the only guy who gets in the game. Instead, we used a simpler method, ranking players at each position in each season by attempts. The players who made up the final 10 percent of passes or runs were split out as "replacement players" and then compared to the players making up the other 90 percent of plays at that position. This took care of the fact that not every non-starter at running back or wide receiver is a freely available talent. (Think of Jonathan Stewart or Randall Cobb, for example.)

As noted earlier, the challenge of any new stat is to present it on a scale that’s meaningful to those attempting to use it. Saying that Tony Romo's passes were worth 40 success value points over replacement in 2014 has very little value without a context to tell us if 40 is good total or a bad one. Therefore, we translate these success values into a number called "Defense-adjusted Yards Above Replacement, or DYAR. Thus, Romo was fifth among quarterbacks with 1,187 passing DYAR. It is our estimate that a generic replacement-level quarterback, throwing in the same situations as Romo, would have been worth 1,187 fewer yards. Note that this doesn’t mean the replacement level quarterback would have gained exactly 1,187 fewer yards. First downs, touchdowns, and turnovers all have an estimated yardage value in this system, so what we are saying is that a generic replacement-level quarterback would have fewer yards and touchdowns (and more turnovers) that would total up to be equivalent to the value of 1,187 yards.

(Note: Prior to the 2008 season, DYAR was translated in terms of points rather than yardage, and old articles will refer to these stats as "DPAR" instead.)
HOW CAN A 16-GAME SEASON BE SIGNIFICANT?

Football statistics can't be analyzed in the same way baseball statistics are. After all, there are only 16 games in a season. Baseball has over ten times more, and even the NBA and NHL offer over five times more. The more games, the more events to analyze, and the more events to analyze, the more statistical significance.

That is true, but the trick is to consider each play in an NFL game as a separate event. For example, Drew Brees played only 16 games in 2014, but in those 16 games he had 692 passing plays (including sacks) and 16 rushing plays (including scrambles) for a total of 708 events. Ian Kinsler in 2014 played in 161 games and had 726 plate appearances. For the most part, a quarterback who plays a full season will have almost the same number of plays as a baseball hitter who plays in most of his team's games.

A running back will have fewer plays than a quarterback, and wide receivers and tight ends will have even fewer. But there should still be enough plays with most starting running backs and receivers to allow for analysis with some significance. As an example, DeMarco Murray ran the ball 392 times in 2014, and was the target of 64 passes (including incompletes), for a total of 456 plays. In general, a starting running back will have 375-450 plays over 16 games. Receivers are used a bit less, and therefore their stats are likely not as accurate. In general, starting wide receivers have 75-150 pass targets over a full season.
ISSUES WITH DVOA/DYAR

DVOA is limited by what’s included in the official NFL play-by-play or tracked by the Football Outsiders game charting project. Because we need to have the entire play-by-play of a season in order to compute DVOA and DYAR, these metrics are not yet ready to compare players of today to players throughout the league’s history. As of this writing, we have processed 26 seasons, 1989 through 2014, and we add seasons at a rate of roughly two per year (the most recent season, plus one season back into history.)

Football is a game in which nearly every action requires the work of two or more teammates -- in fact, usually 11 teammates all working in unison. Unfortunately, when it comes to individual player ratings, we are still far from the point at which we can determine the value of a player independent from the performance of his teammates. That means that when we say, "In 2014, Marshawn Lynch had a DVOA of 23.1%, what we are really saying is “In 2014, Marshawn Lynch, playing in Darrell Bevell’s offensive system with the Seattle offensive line blocking for him and Russell Wilson selling the keeper when necessary, had a DVOA of 23.1%."

With fewer situations to measure, the numbers spread out a bit more, so you'll see more extreme DVOA ratings for part-time players and for measurements of teams in more specific situations (for example, passing on third downs). The charts listing players in order of DVOA have cut-offs for number of attempts, because players with just a handful of plays end up with absurd VOA and DVOA numbers. (In 2014, for instance, Johnny Manziel had a -73.2% passing DVOA in 38 plays.)

Passing statistics include sacks as well as fumbles on aborted snaps. Receiving statistics include all passes intended for the receiver in question, including those that are incomplete or intercepted. At some point, we hope to be able to determine just how much impact different receivers have on completes vs. incomplete passes, but various regression analyses make it clear that both quarterback and receiver have an impact on whether a pass is complete or not. The word passes refers to both complete and incomplete pass attempts.

Unless we say otherwise, all references to third down also include the handful of rushing and passing plays that take place on fourth down (primarily fourth-and-1).
DVOA FOR SPECIAL TEAMS

The problem with a system based on measuring both yardage and yardage towards a first down, of course, is what to do with plays that don't have the possibility of a first down. Special teams are an important part of football and we needed a way to add that performance to the team DVOA ranking. Our special teams metric includes five separate measurements: field goals (and extra points), net punting, punt returns, net kickoffs, and kick returns.

The foundation of most of these special teams ratings is the concept that each yard line has a different value based on how the likelihood of scoring changes with better field position. In Hidden Game, the authors suggested that the value of field position for the offense existed on a straight line with your own goal line being worth -2 points, the 50-yard line 2 points, and the opposing goal line 6 points. (-2 points isn't just the value of a safety; it also reflects the fact that when you are backed up in your own zone, you are likely going to see your drive stall, and you'll need to punt and give the ball to the other team in good field position. Thus, the defense is more likely to score next.) We use a more refined set of values based on our research, but the idea is the same.

Our special teams ratings compare each kick or punt to the league average for based on the point value of field position at the position of each kick, catch, and return. We've determined a league average for how far a kick goes based on the yard line from where the kick occurs (almost always the 35-yard line for kickoffs, variable for punts) and a league average for how far a return goes based on both the yard line where the ball is caught and the distance that it traveled in the air.

The kicking or punting team is rated based on net points compared to average, taking into account both the kick and the return if there is one. Because the average return is always positive, punts that are not returnable (touchbacks, out of bounds, fair catches, and punts downed by the coverage unit) will rate higher than punts of the same distance which are returnable. (This is also true of touchbacks on kickoffs.) There are also separate individual ratings for kickers and punters that are based only on distance and whether the kick is returnable, otherwise assuming an average return in order to judge the kicker separate from the coverage.

For the return team, the rating is based on how many points the return is worth compared to average, based on the location of the catch and the distance the ball traveled in the air. Return teams are not judged on the distance of kicks, nor are they judged on kicks that cannot be returned. As explained below, blocked kicks are so rare as to be statistically insignificant as predictors for future performance and are thus ignored.

Field goal kicking is measured differently. Measuring kickers by field goal percentage is a bit absurd, as it assumes that all field goals are of equal difficulty. In our metric, each field goal is compared to the average number of points scored on all field goal attempts from that distance over the past 15 years. The value of a field goal increases as distance from the goal line increases.

Kickoffs, punts, and field goals are then adjusted based on weather and altitude. It will surprise no one to learn that it is easier to kick the ball in Denver or a dome than it is to kick the ball in Buffalo in December. Because we do not yet have enough data to tailor our adjustments specifically to each stadium, each one is assigned to one of four categories: Cold, Warm, Dome, and Denver. There is also an additional adjustment dropping the value of field goals in Florida (because the warm temperatures allow the ball to carry better) and raising the value of punts in San Francisco (because of those infamous winds).

The baselines for special teams are adjusted in each year for rule changes such as the introduction of the special-teams-use-only “k-ball” in 1999 as well as the move of the kickoff line from the 35 to the 30 in 1994 and then back to the 35 in 2011. Baselines have also been adjusted each year to make up for the gradual improvement of kickers over the last two decades.

Once we’ve totaled how many points above or below average can be attributed to special teams, we translate those points into DVOA so the ratings can be added to offense and defense to get total team DVOA. As a final step, we then normalize special teams DVOA to reflect the league environment in a given year. It should be noted, however, that we don’t yet have a method to perfectly normalize each year of special teams, so the league average for special teams deviates from 0.0% in certain years, although not by more than 0.5%.

There are three aspects of special teams that have an impact on wins and losses, but don’t show up in the standard special teams rating because a team has little or no influence on them. The first is the length of kick¬offs by the opposing team, with an asterisk. Obviously, there are no defenders standing on the 35-yard line, ready to block a kickoff after the whistle blows. However, over the past few years, some teams have delib¬erately kicked short in order to avoid certain top return men, such as Devin Hester and Josh Cribbs. The special teams formula now includes adjustments to give teams extra credit for field position on kick returns if kickers are deliberately trying to avoid a return.

The other two items that special teams have little control over are field goals against your team, and punt distance against your team. Research shows no indication that teams can influence the accuracy or strength of field-goal kickers and punters, except for blocks. As mentioned above, although blocked field goals and punts are definitely skillful plays, they are so rare that they have no correlation to how well teams have played in the past or will play in the future, thus they are included here as if they were any other missed field goal or botched punt, giving the defense no additional credit for their efforts. The value of these three elements is listed separately as “Hidden” value.

Special teams ratings also do not include two-point conversions or onside kick attempts, both of which, like blocks, are so infrequent as to be statistically insignificant in judging future performance.
ADJUSTED LINE YARDS EXPLAINED

One exception to the use of DVOA/DYAR, and the use of "play success" instead of raw yardage, is the rating system for offensive and defensive lines. Actually, these are only measures of running plays, and of course the defensive numbers don't measure just the defensive line, but the whole front seven against the run.

One of the most difficult goals of statistical analysis in football is somehow isolating how much responsibility for a play lies with each of the 22 men on the field. Nowhere is this as obvious as the running game, where one player runs while up to nine other players -- including wideouts, tight ends, and fullback -- block in different directions. None of the statistics we use for measuring rushing -- yards, touchdowns, yards per carry -- differentiate between the contribution of the running back and the contribution of the offensive line. Neither do our advanced metrics DVOA and DYAR.

We have enough data amassed that we can try to separate the effect that the running back has on a particular play from the effect of the offensive line (and other offensive blockers) and the effect of the defense. A team might have two running backs in its stable: RB A, who averages 3.0 yards per carry, and RB B, who averages 3.5 yards per carry. Who is the better back? Imagine that RB A doesn't just average 3.0 yards per carry, but gets exactly 3 yards on every single carry, while RB B has a highly variable yardage output: sometimes 5 yards, sometimes -2 yards, sometimes 20 yards. The difference in variability between the runners can be exploited to not only determine the difference between the runners, but the effect the offensive line has on every running play.

We know that at some point in every long running play, the running back has gotten past all of his offensive line blocks. From here on, the rest of the play is dependent on the runner's own speed and elusiveness, combined with the speed and tackling ability of the defensive players. If Frank Gore breaks through the line for 50 yards, avoiding tacklers all the way to the goal line, his offensive line has done a great job -- but they aren't responsible for most of that run. How much are they responsible for?

For each running back carry, we calculated the probability that the back involved would run for the specific yardage on that play, based on that back's average yardage per carry and the variability of their yardage on every play. We also calculated the probability that the offense would get the yardage based on the team's rushing average and variability without the back involved in the play, and the probability that the defense would give up the specific amount of yardage based on its average rushing yards allowed per carry and variability. For example, based on his rushing average and variability, the probability in 2004 that Tiki Barber would have a positive carry was 80% while the probability that Giants would have a positive carry without Barber running was only 73%.

Yardage ends up falling into roughly the following combinations: Losses, 0-4 yards, 5-10 yards, and 11+ yards. In general, the offensive line is 20% more responsible for lost yardage than it is for yardage gained up to four yards, but 50% less responsible for yardage gained from 5-10 yards, and not responsible for yardage past that. Thus, the creation of Adjusted Line Yards.

Adjusted Line Yards take every carry by a running back and apply those percentages. (We don’t include carries by receivers, which are usually based on deception rather than straight blocking, or carries by quarterbacks, which are almost always busted passing plays unless they involve someone like Colin Kaepernick or Cam Newton.) Those numbers are then adjusted based on down, distance, situation, opponent and whether or not a team is in the shotgun. (Because defenses are generally playing pass when the quarterback is in shotgun, the average running back carry from shotgun last year gained 5.36 yards, compared to just 4.16 yards on other carries.) The adjusted numbers are then normalized so that the league average for Adjusted Line Yards per carry is the same as the league average for RB yards per carry (in 2013, 4.10 yards).

Runs are listed by the NFL in seven different directions: left/right end, left/right tackle, left/right guard, and middle. Further research showed no statistically significant difference between how well a team performed on runs listed middle, left guard, and right guard, so we also list runs separated into five different directions. Note that there may not be a statistically significant difference between right tackle and middle/guard either, but until we can research further (and for the sake of symmetry) we do still split out runs behind the right tackle separately.

These splits allow us to evaluate subsections of a team’s offensive line, but not necessarily individual linemen, as we can’t account for blocking assignments. We don't know when a guard is pulling and when a guard is blocking straight ahead. We know that some runners are just inherently better going up the middle, and some are better going side to side, and we can't measure how much that impacts these numbers. We have no way of knowing the blocking contribution made by fullbacks, tight ends, or wide receivers.

Other numbers we use to measure the running game:

Open Field Yards gives the portion of the team's rushing average gained after the first 10 yards of each run. So for a 10-yard run, no yards are counted; for a 15-yard run, five yards are counted; for an 80-yard run, 70 yards are counted. This number gives you an idea of how much of a team's running game was based on the breakaway speed of the running backs. A team with a low ranking in Adjusted Line Yards but a high ranking in Open Field Yards is heavily dependent on its running back breaking long runs to make the running game work. This number is not adjusted in any way.
Second Level Yards gives the portion of the team's rushing average gained 5-10 yards past the line of scrimmage. So for a five-yard run, no yards are counted; for a 10-yard run, five yards are counted; for an 80-yard run, still only five yards are counted. This number represents the midpoint between the contributions of a team's offensive line and its running back. A low ranking in Open Field Yards coupled with a high ranking in Second Level Yards indicates that the team's offensive line is opening holes, but its backs aren't proficient open-field runners. This number is not adjusted in any way.

Power success measures the success of specific running plays rather than the distance. This number represents how often a running attempt on third or fourth down, with two yards or less to go, achieved a first down or touchdown. Since quarterback sneaks, unlike scrambles, are heavily dependent on the offensive line, this percentage does include runs by all players, not just running backs. This is the only stat given that includes quarterback runs. It is not adjusted based on game situation or opponent.
Stuffed measures the percentage of runs that are stopped at or before the line of scrimmage.

DRIVE STATS

The stats section of our website also features drive stats compiled by Jim Armstrong. These stats are computed from NFL Drive Charts and are not adjusted for strength of schedule or situation. Take-a-knee drives at the end of halves are discarded. Drive stats are generally self-explanatory, giving each team's total number of drives as well as average yards per drive, points per drive, touchdowns per drive, punts per drive, and turnovers per drive, interceptions per drive, and fumbles lost per drive. LOS/Drive represents average starting field position (line of scrimmage) per drive from the offensive point of view. Drive stats are given for offense and defense, with NET representing simply offense minus defense.
A NOTE ON PLAY-BY-PLAY DATA

Our data may differ slightly from official NFL numbers due to discrepancies in different play-by-play reports. In addition, we've adjusted clock plays, with kneels no longer counting as rush attempts and spikes no longer counting as pass attempts. We also count most aborted snaps as passing plays, not rushing plays, unless the play-by-play specifies that the play was an aborted handoff.

Data Provided By:Data Provided By Sports Info Solutions

Frawk.. That was painful. Really? LOL

junglesouljah · May 26, 2019

The guys pissing on Schatz have obviously never read footballoutsiders. He is the first to say that stats without context mean nothing and that this is only an addition to watching tape. You can say because he is a statistician he only believes numbers but that is flat out wrong.
His argument is that history has proven that QBs in the Pros do not improve significantly with regard to accuracy and that QBs below 60% do not become Franchise QBs. Can Allen be an exception. Yes. Is it likely? No.
Let’s see ...

And don’t give me Favre. He played in another time with regard to completion percentage ...

There are a lot of factors that affect the completion percentage including playcalling. If Daboll called 15 screen passes a game Allen could artificially inflate his completion % by a substantial amount like 15 % or more. Another factor that affects the completion percentage is having a clean pocket to read the defense, go through your progressions, and find the open WR. It's kind of hard to do that when playing behind a sub-par OL. The talent at the WR position will also affect your completion % because it affects the WR's ability to get open and to consistently catch the ball. Lastly, a lack of a running game hurt the offense because it can't be balanced and DL can pin their ears back and go after the QB knowing that the offense is not running the ball.

This was lazy analysis and just a poor attent to cover his ass when Josh Allen has a good season. Schatz can go back to his statement that Allen was only successful because Buffalo upgraded the talent around him. But I'm sure Schatz will not bring up the fact that Tom Brady threw to Randy Moss, Rob Gronkowski, Wes Welker, Julian Edelman, Aaron Hernandez, Donte Stallworth, Troy Brown, etc.

There is a long list of QB's who there under 60% completion rate in college and became good QB's in the NFL. There is also a long list of QB who threw over 60% completion rate and were terrible QB's in the NFL. Yes ideally you want a QB with good stats, who has started 4 years at a major program but there are a very limited amount of those types of prospect and those prospects don't always succeed in the NFL.

I could care less if the analytical "draft experts" like Football Outsider and Pro Football Focus don't like Allen. Allen showed toughness coming back from his elbow injury, showed leadership, improved each game he played and had his best game of the year on the last game versus Miami. It's laughable that the analytical "draft experts" can't look at the game tape of Allen improving last year behind a terrible OL and can't project him to take the next step behind a talented rebuild OL and the talent added to the skill positions on offense via the draft and free agency.

Stat can be manipulated by included or excluding data, stats can lie or tell half-truths. If you watched every game Josh Allen played last year and didn't see progress and only point at the stats without the context of why he struggled then you don't you have confirmation bias about Allen.

junglesouljah · May 26, 2019

Is this guy an idiot?? Why the HELL does he think they MADE all of those moves. We knew they were not good coming out of last season. No one that's a Bills fan thought any different and as for Allen - improvced as the season went on, that's what I wanted to see and that's grounds for being optimistic.

Figures - an espn schmuck

Exactly. New England did the same thing surrounded Tom Brady with talent when they were developing him. Brady didn't become the QB he is today with out being groomed and developed by Belichick and throwing to talented WR's to decrease his learning curve.

junglesouljah · May 26, 2019

holy shit this is the homeriest homerism out of you yet. Yea, Schatz is just some rando thats throwing darts against the wall

Methods To Our Madness
THE ULTRA-SHORT VERSION:

DVOA measures a team's efficiency by comparing success on every single play to a league average based on situation and opponent.
THE SHORT VERSION:

DVOA is a method of evaluating teams, units, or players. It takes every single play during the NFL season and compares each one to a league-average baseline based on situation. DVOA measures not just yardage, but yardage towards a first down: Five yards on third-and-4 are worth more than five yards on first-and-10 and much more than five yards on third-and-12. Red zone plays are worth more than other plays. Performance is also adjusted for the quality of the opponent. DVOA is a percentage, so a team with a DVOA of 10.0% is 10 percent better than the average team, and a quarterback with a DVOA of -20.0% is 20 percent worse than the average quarterback. Because DVOA measures scoring, defenses are better when they are negative. For more detail, read below.

Please feel free to contact us with questions and comments about our original statistics using the contact form.

DVOA EXPLAINED
DYAR EXPLAINED
SPECIAL TEAMS STATS
DRIVE STATS
ADJUSTED LINE YARDS and other offensive/defensive line stats

DVOA EXPLAINED

One running back runs for three yards. Another running back runs for three yards. Which is the better run? This sounds like a stupid question, but it isn’t. In fact, this question is at the heart of nearly all of the analysis on Football Outsiders.

Several factors can differentiate one three-yard run from another. What is the down and distance? Is it third-and-2 or second-and-15? Where on the field is the ball? Does the player get only three yards because he hits the goal line and scores? Is the player’s team up by two touchdowns in the fourth quarter, and thus running out the clock; or down by two touchdowns, and thus facing a defense that is playing purely against the pass? Is the running back playing against the porous defense of the Raiders, or the stalwart defense of the Bears?

Conventional NFL statistics value plays based solely on their net yardage. The NFL determines the best players by adding up all their yards no matter what situations they came in or how many plays it took to get them. Now, why would they do that? Football has one objective -- to get to the end zone -- and two ways to achieve that -- by gaining yards and achieving first downs. These two goals need to be balanced to determine a player’s value or a team’s performance. All the yards in the world won’t help a team win if they all come in six-yard chunks on third-and-10.

The popularity of fantasy football only exacerbates the problem. Fans have gotten used to judging players based on how much they help fantasy teams win and lose, not how much they help real teams win and lose. Typical fantasy scoring further skews things by counting the yard between the one and the goal line as 61 times more important than all the other yards on the field (each yard worth 0.1 points, a touchdown worth 6). Let’s say Larry Fitzgerald catches a pass on third-and-15 and goes 50 yards but gets tackled two yards from the goal line, and then Andre Ellington takes the ball on first-and-goal from the two-yard line and plunges in for the score. Has Ellington done something special? Not really. When an offense gets the ball on first-and-goal at the two-yard line, they're expected to score a touchdown five out of six times. Ellington is getting credit for the work done by the passing game.

Doing a better job of distributing credit for scoring points and winning games is the goal of DVOA, or Defense-adjusted Value Over Average. DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.

We then expand upon that basic idea with a more complicated system of “success points,” improved over the past few years with a lot of mathematics and a bit of trial and error. A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point. Interceptions occurring on fourth down during the last two minutes of a game incur no penalty whatsoever, but all others average -6 points, with an adjustment for the length of the pass and the location of the interception (since an interception tipped at the line is more likely to produce a long return than an interception on a 40-yard pass). A fumble is worth anywhere from -1.7 to -4.0 points depending on how often a fumble in that situation is lost to the defense -- no matter who actually recovers the fumble. Red zone plays get a bonus: 20 percent for team offense, five percent for team defense, and 10 percent for individual players. There is a bonus given for a touchdown, which acknowledges that the goal line is significantly more difficult to cross than the previous 99 yards (although this bonus is nowhere near as large as the one used in fantasy football).

(Our system is a bit more complex than the one in Hidden Game thanks to our subsequent research, which added larger penalties for turnovers, the fractional points, and a slightly higher baseline for success on first down. The reason why all fumbles are counted, no matter whether they are recovered by the offense or defense, is explained in FO Basics.)

Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players, adjusted for a number of variables. These include down and distance, field location, time remaining in game, and the team’s lead or deficit in the game score. Teams are always compared to the overall offensive average, as the team made its own choice whether to pass or rush. When it comes to individual players, however, rushing plays are compared to other rushing plays, passing plays to other passing plays, tight ends to tight ends, wideouts to wideouts, and so on.

Going back to our example of the three-yard rush, if Player A gains three yards under a set of circumstances in which the average NFL running back gains only one yard, then Player A has a certain amount of value above others at his position. Likewise, if Player B gains three yards on a play on which, under similar circumstances, an average NFL back gains four yards, that Player B has negative value relative to others at his position. Once we make all our adjustments, we can evaluate the difference between this player’s rate of success and the expected success rate of an average running back in the same situation (or between the opposing defense and the average defense in the same situation, etc.). Add up every play by a certain team or player, divide by the total of the various baselines* for success in all those situations, and you get VOA, or Value Over Average.

The biggest variable in football is the fact that each team plays a different schedule against teams of disparate quality. By adjusting each play based on the opposing defense’s average success in stopping that type of play over the course of a season, we get DVOA, or Defense-adjusted Value Over Average. Rushing and passing plays are adjusted based on down and location on the field; passing plays are also adjusted based on how the defense performs against passes to running backs, tight ends, or wide receivers. Defenses are adjusted based on the average success of the offenses they are facing. (Yes, technically the defensive stats are actually “offense-adjusted.” If it seems weird, think of the “D” in “DVOA” as standing for “opponent-Dependent” or something.)

The final step in calculating DVOA involves normalizing each year's ratings. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all time. Therefore, we need to ensure that DVOA in a given season isn't skewed by league environment.

For teams, DVOA is normalized so that league averages for offense and defense are 0%. (However, because pass plays are more efficient than run plays, league averages for team passing and team rushing are not zero.) For players, DVOA is normalized separately for individual passing, individual rushing, and the three individual receiving groups (wide receivers, tight ends, and running backs) so that the league average for each is 0%.

Of course, one of the hardest parts of understanding a new statistic is interpreting its scale. To use DVOA, you have to know what numbers represent good performance and what numbers represent bad performance. We’ve made that easy. In all cases, 0% represents league-average. A positive DVOA represents a situation that favors the offense, while a negative DVOA represents a situation that favors the defense. This is why the best offenses have positive DVOA ratings (last year, Green Bay led the league at +24.7%) and the best defenses have negative DVOA ratings (with Seattle number one in 2014 at -16.8%). In most years, the best and worst offenses tend to rate around ± 30%, while the best and worst defenses tend to rate around ± 25%. For starting players, the scale tends to reach roughly ± 40% for passing and receiving, and ± 30% for rushing. As you might imagine, some players with fewer attempts will surpass both extremes.

DVOA has three main advantages over more traditional ways to judge NFL performance. First, by subtracting defense DVOA from offense DVOA (and adding in special teams DVOA, which is described below), we can create a set of team rankings that's based on play-by-play efficiency rather than total yards. Because DVOA does a better job of explaining past wins and predicting future wins than total yards, it gives a more accurate picture of how much better (or worse) a team really is relative to the rest of the league.

Because it compares each play only to plays with similar circumstances, this advantage also applies vis-a-vis situational team rankings. The list of top DVOA offenses on third down, for example, is more accurate than the conventional NFL conversion statistic because it takes into account that converting third-and-long is more difficult than converting third-and-short, and that a turnover is worse than an incomplete pass because it eliminates the opportunity to move the other team back with a punt on fourth down. The same could be said about plays on fourth down or in the red zone.

Second, unlike formulas based on comparing drives rather than individual plays, DVOA can be separated into a myriad of splits (e.g., by down, by week, by distance needed for a first down, etc.). Therefore, we're able to break teams and players down to find strengths and weaknesses in a variety of situations. All Pittsburgh third downs can be compared to how an average team does on third down. Josh McCown and Mike Glennon can each be compared to how an average quarterback performs in the red zone, or with a lead, or in the second half of the game. This doesn't just give us a better idea of which team or player is better. More importantly, it helps us understand why they're better, and therefore allows us to offer prescriptions for improvement in the future.

Finally, a third advantage of DVOA is that normalization makes our comparisons of current teams and players to past teams and players (going back to 1989) more accurate than those based on traditional statistics like wins or total yards, as well as those based on more sophisticated metrics that aren't normalized (e.g., expected points added, passer rating differential, etc.). For instance, which Denver Broncos team had the better offense: the 2013 edition with Peyton Manning, or the 1998 club led by Terrell Davis? Going by total yardage (7,317 vs. 6,092) or even yards per play (6.3 vs. 5.9), it's not even a contest. The 2013 team were clearly better. However, this ignores the fact that the average NFL offense was much more pass-oriented, and thus more efficient, in 2013 than in 1998. If we take the difference in offensive environment into account by using DVOA, it turns out that the 1998 Broncos offense was slightly better relative to the rest of the league (34.5% to 33.5%).

*It should be noted that certain plays are included in DVOA for offense but not for defense. Other plays are included for both, but scored differently. This leads to separate baselines on each side of the ball. For instance

Only four total penalties are included. Two penalties count as pass plays on both sides of the ball: intentional grounding and defensive pass interference. The other two penalties are included for offense only: false starts and delay of game. Because the inclusion of these penalties means a group of negative plays that don’t count as either passes or runs, the league averages for pass offense and run offense are higher than the league averages for pass defense and run defense.
Aborted snaps and incomplete backwards lateral passes are only penalized on offense, not rewarded on defense.
Adjustments for playing from behind or with a lead in the fourth quarter are different for offense and defense, as are adjustments for the final two minutes of the first half when the offense is not near field-goal range.
Offense gets a slight penalty and defense gets a slight bonus for games indoors.

DYAR EXPLAINED

After using DVOA for a few months, we came across a strange phenomenon: well-regarded players, particularly those known for their durability, had DVOA ratings that came out around average. The reason is that DVOA, by virtue of being a percentage or rate statistic, doesn’t take into account the cumulative value of having a player producing at a league-average level over the course of an above-average number of plays. By definition, an average level of performance is better than that provided by half of the league and the ability to maintain that level of performance while carrying a heavy workload is very valuable indeed. In addition, a player who is involved in a high number of plays can draw the defense’s attention away from other parts of the offense, and, if that player is a running back, he can take time off the clock with repeated runs.

Let’s say you have a running back who carries the ball 300 times in a season. What would happen if you were to remove this player from his team’s offense? What would happen to those 300 plays? Those plays don’t disappear with the player, though some might be lost to the defense because of the associated loss of first downs. Rather those plays would have to be distributed among the remaining players in the offense, with the bulk of them being given to a replacement running back. This is where we arrive at the concept of replacement level, borrowed from our partners at Baseball Prospectus. When a player is removed from an offense, he is usually not replaced by a player of similar ability. Nearly every starting player in the NFL is a starter because he is better than the alternative. Those 300 plays will typically be given to a significantly worse player, someone who is the backup because he doesn’t have as much experience and/or talent. A player’s true value can then be measured by the level of performance he provides above that replacement level baseline, totaled over all of his run or pass attempts.

Of course, the real replacement player is different for each team in the NFL. In 2013, the second-string running back in Minnesota (Jerick McKinnon) had a higher DVOA than the alleged starter (Matt Asiata). Sometimes a player like Ryan Grant or Danny Woodhead will be cut by one team and turn into a star for another. On other teams, the drop from the starter to the backup can be even greater than the general drop to replacement level. (The Indianapolis Colts of 2011--the dark year between the Manning and Luck eras--will be the hallmark example of this until the end of time.) The choice to start an inferior player or to employ a sub-replacement level backup, however, falls to the team, not the starter being evaluated. Thus, we generalize replacement level for the league as a whole, as the ultimate goal is to evaluate players independent of the quality of their teammates.

Our estimates of replacement level were re-done during the 2008 season and are computed differently for each position. For quarterbacks, we analyzed situations where two or more quarterbacks had played meaningful snaps for a team in the same season, then compared the overall DVOA of the original starters to the overall DVOA of the replacements. We did not include situations where the backup was actually a top prospect waiting his turn on the bench, since a first-round pick is by no means a "replacement-level" player.

At other positions, there is no easy way to separate players into "starters" and "replacements," since unlike at quarterback, being the starter doesn't make you the only guy who gets in the game. Instead, we used a simpler method, ranking players at each position in each season by attempts. The players who made up the final 10 percent of passes or runs were split out as "replacement players" and then compared to the players making up the other 90 percent of plays at that position. This took care of the fact that not every non-starter at running back or wide receiver is a freely available talent. (Think of Jonathan Stewart or Randall Cobb, for example.)

As noted earlier, the challenge of any new stat is to present it on a scale that’s meaningful to those attempting to use it. Saying that Tony Romo's passes were worth 40 success value points over replacement in 2014 has very little value without a context to tell us if 40 is good total or a bad one. Therefore, we translate these success values into a number called "Defense-adjusted Yards Above Replacement, or DYAR. Thus, Romo was fifth among quarterbacks with 1,187 passing DYAR. It is our estimate that a generic replacement-level quarterback, throwing in the same situations as Romo, would have been worth 1,187 fewer yards. Note that this doesn’t mean the replacement level quarterback would have gained exactly 1,187 fewer yards. First downs, touchdowns, and turnovers all have an estimated yardage value in this system, so what we are saying is that a generic replacement-level quarterback would have fewer yards and touchdowns (and more turnovers) that would total up to be equivalent to the value of 1,187 yards.

(Note: Prior to the 2008 season, DYAR was translated in terms of points rather than yardage, and old articles will refer to these stats as "DPAR" instead.)
HOW CAN A 16-GAME SEASON BE SIGNIFICANT?

Football statistics can't be analyzed in the same way baseball statistics are. After all, there are only 16 games in a season. Baseball has over ten times more, and even the NBA and NHL offer over five times more. The more games, the more events to analyze, and the more events to analyze, the more statistical significance.

That is true, but the trick is to consider each play in an NFL game as a separate event. For example, Drew Brees played only 16 games in 2014, but in those 16 games he had 692 passing plays (including sacks) and 16 rushing plays (including scrambles) for a total of 708 events. Ian Kinsler in 2014 played in 161 games and had 726 plate appearances. For the most part, a quarterback who plays a full season will have almost the same number of plays as a baseball hitter who plays in most of his team's games.

A running back will have fewer plays than a quarterback, and wide receivers and tight ends will have even fewer. But there should still be enough plays with most starting running backs and receivers to allow for analysis with some significance. As an example, DeMarco Murray ran the ball 392 times in 2014, and was the target of 64 passes (including incompletes), for a total of 456 plays. In general, a starting running back will have 375-450 plays over 16 games. Receivers are used a bit less, and therefore their stats are likely not as accurate. In general, starting wide receivers have 75-150 pass targets over a full season.
ISSUES WITH DVOA/DYAR

DVOA is limited by what’s included in the official NFL play-by-play or tracked by the Football Outsiders game charting project. Because we need to have the entire play-by-play of a season in order to compute DVOA and DYAR, these metrics are not yet ready to compare players of today to players throughout the league’s history. As of this writing, we have processed 26 seasons, 1989 through 2014, and we add seasons at a rate of roughly two per year (the most recent season, plus one season back into history.)

Football is a game in which nearly every action requires the work of two or more teammates -- in fact, usually 11 teammates all working in unison. Unfortunately, when it comes to individual player ratings, we are still far from the point at which we can determine the value of a player independent from the performance of his teammates. That means that when we say, "In 2014, Marshawn Lynch had a DVOA of 23.1%, what we are really saying is “In 2014, Marshawn Lynch, playing in Darrell Bevell’s offensive system with the Seattle offensive line blocking for him and Russell Wilson selling the keeper when necessary, had a DVOA of 23.1%."

With fewer situations to measure, the numbers spread out a bit more, so you'll see more extreme DVOA ratings for part-time players and for measurements of teams in more specific situations (for example, passing on third downs). The charts listing players in order of DVOA have cut-offs for number of attempts, because players with just a handful of plays end up with absurd VOA and DVOA numbers. (In 2014, for instance, Johnny Manziel had a -73.2% passing DVOA in 38 plays.)

Passing statistics include sacks as well as fumbles on aborted snaps. Receiving statistics include all passes intended for the receiver in question, including those that are incomplete or intercepted. At some point, we hope to be able to determine just how much impact different receivers have on completes vs. incomplete passes, but various regression analyses make it clear that both quarterback and receiver have an impact on whether a pass is complete or not. The word passes refers to both complete and incomplete pass attempts.

Unless we say otherwise, all references to third down also include the handful of rushing and passing plays that take place on fourth down (primarily fourth-and-1).
DVOA FOR SPECIAL TEAMS

The problem with a system based on measuring both yardage and yardage towards a first down, of course, is what to do with plays that don't have the possibility of a first down. Special teams are an important part of football and we needed a way to add that performance to the team DVOA ranking. Our special teams metric includes five separate measurements: field goals (and extra points), net punting, punt returns, net kickoffs, and kick returns.

The foundation of most of these special teams ratings is the concept that each yard line has a different value based on how the likelihood of scoring changes with better field position. In Hidden Game, the authors suggested that the value of field position for the offense existed on a straight line with your own goal line being worth -2 points, the 50-yard line 2 points, and the opposing goal line 6 points. (-2 points isn't just the value of a safety; it also reflects the fact that when you are backed up in your own zone, you are likely going to see your drive stall, and you'll need to punt and give the ball to the other team in good field position. Thus, the defense is more likely to score next.) We use a more refined set of values based on our research, but the idea is the same.

Our special teams ratings compare each kick or punt to the league average for based on the point value of field position at the position of each kick, catch, and return. We've determined a league average for how far a kick goes based on the yard line from where the kick occurs (almost always the 35-yard line for kickoffs, variable for punts) and a league average for how far a return goes based on both the yard line where the ball is caught and the distance that it traveled in the air.

The kicking or punting team is rated based on net points compared to average, taking into account both the kick and the return if there is one. Because the average return is always positive, punts that are not returnable (touchbacks, out of bounds, fair catches, and punts downed by the coverage unit) will rate higher than punts of the same distance which are returnable. (This is also true of touchbacks on kickoffs.) There are also separate individual ratings for kickers and punters that are based only on distance and whether the kick is returnable, otherwise assuming an average return in order to judge the kicker separate from the coverage.

For the return team, the rating is based on how many points the return is worth compared to average, based on the location of the catch and the distance the ball traveled in the air. Return teams are not judged on the distance of kicks, nor are they judged on kicks that cannot be returned. As explained below, blocked kicks are so rare as to be statistically insignificant as predictors for future performance and are thus ignored.

Field goal kicking is measured differently. Measuring kickers by field goal percentage is a bit absurd, as it assumes that all field goals are of equal difficulty. In our metric, each field goal is compared to the average number of points scored on all field goal attempts from that distance over the past 15 years. The value of a field goal increases as distance from the goal line increases.

Kickoffs, punts, and field goals are then adjusted based on weather and altitude. It will surprise no one to learn that it is easier to kick the ball in Denver or a dome than it is to kick the ball in Buffalo in December. Because we do not yet have enough data to tailor our adjustments specifically to each stadium, each one is assigned to one of four categories: Cold, Warm, Dome, and Denver. There is also an additional adjustment dropping the value of field goals in Florida (because the warm temperatures allow the ball to carry better) and raising the value of punts in San Francisco (because of those infamous winds).

The baselines for special teams are adjusted in each year for rule changes such as the introduction of the special-teams-use-only “k-ball” in 1999 as well as the move of the kickoff line from the 35 to the 30 in 1994 and then back to the 35 in 2011. Baselines have also been adjusted each year to make up for the gradual improvement of kickers over the last two decades.

Once we’ve totaled how many points above or below average can be attributed to special teams, we translate those points into DVOA so the ratings can be added to offense and defense to get total team DVOA. As a final step, we then normalize special teams DVOA to reflect the league environment in a given year. It should be noted, however, that we don’t yet have a method to perfectly normalize each year of special teams, so the league average for special teams deviates from 0.0% in certain years, although not by more than 0.5%.

There are three aspects of special teams that have an impact on wins and losses, but don’t show up in the standard special teams rating because a team has little or no influence on them. The first is the length of kick¬offs by the opposing team, with an asterisk. Obviously, there are no defenders standing on the 35-yard line, ready to block a kickoff after the whistle blows. However, over the past few years, some teams have delib¬erately kicked short in order to avoid certain top return men, such as Devin Hester and Josh Cribbs. The special teams formula now includes adjustments to give teams extra credit for field position on kick returns if kickers are deliberately trying to avoid a return.

The other two items that special teams have little control over are field goals against your team, and punt distance against your team. Research shows no indication that teams can influence the accuracy or strength of field-goal kickers and punters, except for blocks. As mentioned above, although blocked field goals and punts are definitely skillful plays, they are so rare that they have no correlation to how well teams have played in the past or will play in the future, thus they are included here as if they were any other missed field goal or botched punt, giving the defense no additional credit for their efforts. The value of these three elements is listed separately as “Hidden” value.

Special teams ratings also do not include two-point conversions or onside kick attempts, both of which, like blocks, are so infrequent as to be statistically insignificant in judging future performance.
ADJUSTED LINE YARDS EXPLAINED

One exception to the use of DVOA/DYAR, and the use of "play success" instead of raw yardage, is the rating system for offensive and defensive lines. Actually, these are only measures of running plays, and of course the defensive numbers don't measure just the defensive line, but the whole front seven against the run.

One of the most difficult goals of statistical analysis in football is somehow isolating how much responsibility for a play lies with each of the 22 men on the field. Nowhere is this as obvious as the running game, where one player runs while up to nine other players -- including wideouts, tight ends, and fullback -- block in different directions. None of the statistics we use for measuring rushing -- yards, touchdowns, yards per carry -- differentiate between the contribution of the running back and the contribution of the offensive line. Neither do our advanced metrics DVOA and DYAR.

We have enough data amassed that we can try to separate the effect that the running back has on a particular play from the effect of the offensive line (and other offensive blockers) and the effect of the defense. A team might have two running backs in its stable: RB A, who averages 3.0 yards per carry, and RB B, who averages 3.5 yards per carry. Who is the better back? Imagine that RB A doesn't just average 3.0 yards per carry, but gets exactly 3 yards on every single carry, while RB B has a highly variable yardage output: sometimes 5 yards, sometimes -2 yards, sometimes 20 yards. The difference in variability between the runners can be exploited to not only determine the difference between the runners, but the effect the offensive line has on every running play.

We know that at some point in every long running play, the running back has gotten past all of his offensive line blocks. From here on, the rest of the play is dependent on the runner's own speed and elusiveness, combined with the speed and tackling ability of the defensive players. If Frank Gore breaks through the line for 50 yards, avoiding tacklers all the way to the goal line, his offensive line has done a great job -- but they aren't responsible for most of that run. How much are they responsible for?

For each running back carry, we calculated the probability that the back involved would run for the specific yardage on that play, based on that back's average yardage per carry and the variability of their yardage on every play. We also calculated the probability that the offense would get the yardage based on the team's rushing average and variability without the back involved in the play, and the probability that the defense would give up the specific amount of yardage based on its average rushing yards allowed per carry and variability. For example, based on his rushing average and variability, the probability in 2004 that Tiki Barber would have a positive carry was 80% while the probability that Giants would have a positive carry without Barber running was only 73%.

Yardage ends up falling into roughly the following combinations: Losses, 0-4 yards, 5-10 yards, and 11+ yards. In general, the offensive line is 20% more responsible for lost yardage than it is for yardage gained up to four yards, but 50% less responsible for yardage gained from 5-10 yards, and not responsible for yardage past that. Thus, the creation of Adjusted Line Yards.

Adjusted Line Yards take every carry by a running back and apply those percentages. (We don’t include carries by receivers, which are usually based on deception rather than straight blocking, or carries by quarterbacks, which are almost always busted passing plays unless they involve someone like Colin Kaepernick or Cam Newton.) Those numbers are then adjusted based on down, distance, situation, opponent and whether or not a team is in the shotgun. (Because defenses are generally playing pass when the quarterback is in shotgun, the average running back carry from shotgun last year gained 5.36 yards, compared to just 4.16 yards on other carries.) The adjusted numbers are then normalized so that the league average for Adjusted Line Yards per carry is the same as the league average for RB yards per carry (in 2013, 4.10 yards).

Runs are listed by the NFL in seven different directions: left/right end, left/right tackle, left/right guard, and middle. Further research showed no statistically significant difference between how well a team performed on runs listed middle, left guard, and right guard, so we also list runs separated into five different directions. Note that there may not be a statistically significant difference between right tackle and middle/guard either, but until we can research further (and for the sake of symmetry) we do still split out runs behind the right tackle separately.

These splits allow us to evaluate subsections of a team’s offensive line, but not necessarily individual linemen, as we can’t account for blocking assignments. We don't know when a guard is pulling and when a guard is blocking straight ahead. We know that some runners are just inherently better going up the middle, and some are better going side to side, and we can't measure how much that impacts these numbers. We have no way of knowing the blocking contribution made by fullbacks, tight ends, or wide receivers.

Other numbers we use to measure the running game:

Open Field Yards gives the portion of the team's rushing average gained after the first 10 yards of each run. So for a 10-yard run, no yards are counted; for a 15-yard run, five yards are counted; for an 80-yard run, 70 yards are counted. This number gives you an idea of how much of a team's running game was based on the breakaway speed of the running backs. A team with a low ranking in Adjusted Line Yards but a high ranking in Open Field Yards is heavily dependent on its running back breaking long runs to make the running game work. This number is not adjusted in any way.
Second Level Yards gives the portion of the team's rushing average gained 5-10 yards past the line of scrimmage. So for a five-yard run, no yards are counted; for a 10-yard run, five yards are counted; for an 80-yard run, still only five yards are counted. This number represents the midpoint between the contributions of a team's offensive line and its running back. A low ranking in Open Field Yards coupled with a high ranking in Second Level Yards indicates that the team's offensive line is opening holes, but its backs aren't proficient open-field runners. This number is not adjusted in any way.

Power success measures the success of specific running plays rather than the distance. This number represents how often a running attempt on third or fourth down, with two yards or less to go, achieved a first down or touchdown. Since quarterback sneaks, unlike scrambles, are heavily dependent on the offensive line, this percentage does include runs by all players, not just running backs. This is the only stat given that includes quarterback runs. It is not adjusted based on game situation or opponent.
Stuffed measures the percentage of runs that are stopped at or before the line of scrimmage.

DRIVE STATS

The stats section of our website also features drive stats compiled by Jim Armstrong. These stats are computed from NFL Drive Charts and are not adjusted for strength of schedule or situation. Take-a-knee drives at the end of halves are discarded. Drive stats are generally self-explanatory, giving each team's total number of drives as well as average yards per drive, points per drive, touchdowns per drive, punts per drive, and turnovers per drive, interceptions per drive, and fumbles lost per drive. LOS/Drive represents average starting field position (line of scrimmage) per drive from the offensive point of view. Drive stats are given for offense and defense, with NET representing simply offense minus defense.
A NOTE ON PLAY-BY-PLAY DATA

Our data may differ slightly from official NFL numbers due to discrepancies in different play-by-play reports. In addition, we've adjusted clock plays, with kneels no longer counting as rush attempts and spikes no longer counting as pass attempts. We also count most aborted snaps as passing plays, not rushing plays, unless the play-by-play specifies that the play was an aborted handoff.

Data Provided By:Data Provided By Sports Info Solutions

I"m sure the analytics guys are never wrong. Or maybe they do mental gymnastics like Schatz and give lazy pathetic analysis like:

“I’m just not a Josh Allen believer. I do not think there is an NFL starting quarterback in there waiting to come out and develop. And if I’m wrong then, you know, they’ve done a lot to give him help, but if I’m right then they’re still in trouble.”

Schatz is a joke by making that qualifying statement. He says Allen is not a starting NFL QB but look at the stats on the last game Allen played: 17/26 65.4% 224 yards 3 TD's , 1 int, 95 rushing yards on 9 attempts and 2 rushing TD's. So he had a QB rating of 114.9 and accounted for 5 total TD's. That looks likes like an NFL starting QB despite what Schatz says.

Bocephus · May 26, 2019

holy shit this is the homeriest homerism out of you yet. Yea, Schatz is just some rando thats throwing darts against the wall

Methods To Our Madness
THE ULTRA-SHORT VERSION:

DVOA measures a team's efficiency by comparing success on every single play to a league average based on situation and opponent.
THE SHORT VERSION:

DVOA is a method of evaluating teams, units, or players. It takes every single play during the NFL season and compares each one to a league-average baseline based on situation. DVOA measures not just yardage, but yardage towards a first down: Five yards on third-and-4 are worth more than five yards on first-and-10 and much more than five yards on third-and-12. Red zone plays are worth more than other plays. Performance is also adjusted for the quality of the opponent. DVOA is a percentage, so a team with a DVOA of 10.0% is 10 percent better than the average team, and a quarterback with a DVOA of -20.0% is 20 percent worse than the average quarterback. Because DVOA measures scoring, defenses are better when they are negative. For more detail, read below.

Please feel free to contact us with questions and comments about our original statistics using the contact form.

DVOA EXPLAINED
DYAR EXPLAINED
SPECIAL TEAMS STATS
DRIVE STATS
ADJUSTED LINE YARDS and other offensive/defensive line stats

DVOA EXPLAINED

One running back runs for three yards. Another running back runs for three yards. Which is the better run? This sounds like a stupid question, but it isn’t. In fact, this question is at the heart of nearly all of the analysis on Football Outsiders.

Several factors can differentiate one three-yard run from another. What is the down and distance? Is it third-and-2 or second-and-15? Where on the field is the ball? Does the player get only three yards because he hits the goal line and scores? Is the player’s team up by two touchdowns in the fourth quarter, and thus running out the clock; or down by two touchdowns, and thus facing a defense that is playing purely against the pass? Is the running back playing against the porous defense of the Raiders, or the stalwart defense of the Bears?

Conventional NFL statistics value plays based solely on their net yardage. The NFL determines the best players by adding up all their yards no matter what situations they came in or how many plays it took to get them. Now, why would they do that? Football has one objective -- to get to the end zone -- and two ways to achieve that -- by gaining yards and achieving first downs. These two goals need to be balanced to determine a player’s value or a team’s performance. All the yards in the world won’t help a team win if they all come in six-yard chunks on third-and-10.

The popularity of fantasy football only exacerbates the problem. Fans have gotten used to judging players based on how much they help fantasy teams win and lose, not how much they help real teams win and lose. Typical fantasy scoring further skews things by counting the yard between the one and the goal line as 61 times more important than all the other yards on the field (each yard worth 0.1 points, a touchdown worth 6). Let’s say Larry Fitzgerald catches a pass on third-and-15 and goes 50 yards but gets tackled two yards from the goal line, and then Andre Ellington takes the ball on first-and-goal from the two-yard line and plunges in for the score. Has Ellington done something special? Not really. When an offense gets the ball on first-and-goal at the two-yard line, they're expected to score a touchdown five out of six times. Ellington is getting credit for the work done by the passing game.

Doing a better job of distributing credit for scoring points and winning games is the goal of DVOA, or Defense-adjusted Value Over Average. DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.

We then expand upon that basic idea with a more complicated system of “success points,” improved over the past few years with a lot of mathematics and a bit of trial and error. A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point. Interceptions occurring on fourth down during the last two minutes of a game incur no penalty whatsoever, but all others average -6 points, with an adjustment for the length of the pass and the location of the interception (since an interception tipped at the line is more likely to produce a long return than an interception on a 40-yard pass). A fumble is worth anywhere from -1.7 to -4.0 points depending on how often a fumble in that situation is lost to the defense -- no matter who actually recovers the fumble. Red zone plays get a bonus: 20 percent for team offense, five percent for team defense, and 10 percent for individual players. There is a bonus given for a touchdown, which acknowledges that the goal line is significantly more difficult to cross than the previous 99 yards (although this bonus is nowhere near as large as the one used in fantasy football).

(Our system is a bit more complex than the one in Hidden Game thanks to our subsequent research, which added larger penalties for turnovers, the fractional points, and a slightly higher baseline for success on first down. The reason why all fumbles are counted, no matter whether they are recovered by the offense or defense, is explained in FO Basics.)

Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players, adjusted for a number of variables. These include down and distance, field location, time remaining in game, and the team’s lead or deficit in the game score. Teams are always compared to the overall offensive average, as the team made its own choice whether to pass or rush. When it comes to individual players, however, rushing plays are compared to other rushing plays, passing plays to other passing plays, tight ends to tight ends, wideouts to wideouts, and so on.

Going back to our example of the three-yard rush, if Player A gains three yards under a set of circumstances in which the average NFL running back gains only one yard, then Player A has a certain amount of value above others at his position. Likewise, if Player B gains three yards on a play on which, under similar circumstances, an average NFL back gains four yards, that Player B has negative value relative to others at his position. Once we make all our adjustments, we can evaluate the difference between this player’s rate of success and the expected success rate of an average running back in the same situation (or between the opposing defense and the average defense in the same situation, etc.). Add up every play by a certain team or player, divide by the total of the various baselines* for success in all those situations, and you get VOA, or Value Over Average.

The biggest variable in football is the fact that each team plays a different schedule against teams of disparate quality. By adjusting each play based on the opposing defense’s average success in stopping that type of play over the course of a season, we get DVOA, or Defense-adjusted Value Over Average. Rushing and passing plays are adjusted based on down and location on the field; passing plays are also adjusted based on how the defense performs against passes to running backs, tight ends, or wide receivers. Defenses are adjusted based on the average success of the offenses they are facing. (Yes, technically the defensive stats are actually “offense-adjusted.” If it seems weird, think of the “D” in “DVOA” as standing for “opponent-Dependent” or something.)

The final step in calculating DVOA involves normalizing each year's ratings. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all time. Therefore, we need to ensure that DVOA in a given season isn't skewed by league environment.

For teams, DVOA is normalized so that league averages for offense and defense are 0%. (However, because pass plays are more efficient than run plays, league averages for team passing and team rushing are not zero.) For players, DVOA is normalized separately for individual passing, individual rushing, and the three individual receiving groups (wide receivers, tight ends, and running backs) so that the league average for each is 0%.

Of course, one of the hardest parts of understanding a new statistic is interpreting its scale. To use DVOA, you have to know what numbers represent good performance and what numbers represent bad performance. We’ve made that easy. In all cases, 0% represents league-average. A positive DVOA represents a situation that favors the offense, while a negative DVOA represents a situation that favors the defense. This is why the best offenses have positive DVOA ratings (last year, Green Bay led the league at +24.7%) and the best defenses have negative DVOA ratings (with Seattle number one in 2014 at -16.8%). In most years, the best and worst offenses tend to rate around ± 30%, while the best and worst defenses tend to rate around ± 25%. For starting players, the scale tends to reach roughly ± 40% for passing and receiving, and ± 30% for rushing. As you might imagine, some players with fewer attempts will surpass both extremes.

DVOA has three main advantages over more traditional ways to judge NFL performance. First, by subtracting defense DVOA from offense DVOA (and adding in special teams DVOA, which is described below), we can create a set of team rankings that's based on play-by-play efficiency rather than total yards. Because DVOA does a better job of explaining past wins and predicting future wins than total yards, it gives a more accurate picture of how much better (or worse) a team really is relative to the rest of the league.

Because it compares each play only to plays with similar circumstances, this advantage also applies vis-a-vis situational team rankings. The list of top DVOA offenses on third down, for example, is more accurate than the conventional NFL conversion statistic because it takes into account that converting third-and-long is more difficult than converting third-and-short, and that a turnover is worse than an incomplete pass because it eliminates the opportunity to move the other team back with a punt on fourth down. The same could be said about plays on fourth down or in the red zone.

Second, unlike formulas based on comparing drives rather than individual plays, DVOA can be separated into a myriad of splits (e.g., by down, by week, by distance needed for a first down, etc.). Therefore, we're able to break teams and players down to find strengths and weaknesses in a variety of situations. All Pittsburgh third downs can be compared to how an average team does on third down. Josh McCown and Mike Glennon can each be compared to how an average quarterback performs in the red zone, or with a lead, or in the second half of the game. This doesn't just give us a better idea of which team or player is better. More importantly, it helps us understand why they're better, and therefore allows us to offer prescriptions for improvement in the future.

Finally, a third advantage of DVOA is that normalization makes our comparisons of current teams and players to past teams and players (going back to 1989) more accurate than those based on traditional statistics like wins or total yards, as well as those based on more sophisticated metrics that aren't normalized (e.g., expected points added, passer rating differential, etc.). For instance, which Denver Broncos team had the better offense: the 2013 edition with Peyton Manning, or the 1998 club led by Terrell Davis? Going by total yardage (7,317 vs. 6,092) or even yards per play (6.3 vs. 5.9), it's not even a contest. The 2013 team were clearly better. However, this ignores the fact that the average NFL offense was much more pass-oriented, and thus more efficient, in 2013 than in 1998. If we take the difference in offensive environment into account by using DVOA, it turns out that the 1998 Broncos offense was slightly better relative to the rest of the league (34.5% to 33.5%).

*It should be noted that certain plays are included in DVOA for offense but not for defense. Other plays are included for both, but scored differently. This leads to separate baselines on each side of the ball. For instance

Only four total penalties are included. Two penalties count as pass plays on both sides of the ball: intentional grounding and defensive pass interference. The other two penalties are included for offense only: false starts and delay of game. Because the inclusion of these penalties means a group of negative plays that don’t count as either passes or runs, the league averages for pass offense and run offense are higher than the league averages for pass defense and run defense.
Aborted snaps and incomplete backwards lateral passes are only penalized on offense, not rewarded on defense.
Adjustments for playing from behind or with a lead in the fourth quarter are different for offense and defense, as are adjustments for the final two minutes of the first half when the offense is not near field-goal range.
Offense gets a slight penalty and defense gets a slight bonus for games indoors.

DYAR EXPLAINED

After using DVOA for a few months, we came across a strange phenomenon: well-regarded players, particularly those known for their durability, had DVOA ratings that came out around average. The reason is that DVOA, by virtue of being a percentage or rate statistic, doesn’t take into account the cumulative value of having a player producing at a league-average level over the course of an above-average number of plays. By definition, an average level of performance is better than that provided by half of the league and the ability to maintain that level of performance while carrying a heavy workload is very valuable indeed. In addition, a player who is involved in a high number of plays can draw the defense’s attention away from other parts of the offense, and, if that player is a running back, he can take time off the clock with repeated runs.

Let’s say you have a running back who carries the ball 300 times in a season. What would happen if you were to remove this player from his team’s offense? What would happen to those 300 plays? Those plays don’t disappear with the player, though some might be lost to the defense because of the associated loss of first downs. Rather those plays would have to be distributed among the remaining players in the offense, with the bulk of them being given to a replacement running back. This is where we arrive at the concept of replacement level, borrowed from our partners at Baseball Prospectus. When a player is removed from an offense, he is usually not replaced by a player of similar ability. Nearly every starting player in the NFL is a starter because he is better than the alternative. Those 300 plays will typically be given to a significantly worse player, someone who is the backup because he doesn’t have as much experience and/or talent. A player’s true value can then be measured by the level of performance he provides above that replacement level baseline, totaled over all of his run or pass attempts.

Of course, the real replacement player is different for each team in the NFL. In 2013, the second-string running back in Minnesota (Jerick McKinnon) had a higher DVOA than the alleged starter (Matt Asiata). Sometimes a player like Ryan Grant or Danny Woodhead will be cut by one team and turn into a star for another. On other teams, the drop from the starter to the backup can be even greater than the general drop to replacement level. (The Indianapolis Colts of 2011--the dark year between the Manning and Luck eras--will be the hallmark example of this until the end of time.) The choice to start an inferior player or to employ a sub-replacement level backup, however, falls to the team, not the starter being evaluated. Thus, we generalize replacement level for the league as a whole, as the ultimate goal is to evaluate players independent of the quality of their teammates.

Our estimates of replacement level were re-done during the 2008 season and are computed differently for each position. For quarterbacks, we analyzed situations where two or more quarterbacks had played meaningful snaps for a team in the same season, then compared the overall DVOA of the original starters to the overall DVOA of the replacements. We did not include situations where the backup was actually a top prospect waiting his turn on the bench, since a first-round pick is by no means a "replacement-level" player.

At other positions, there is no easy way to separate players into "starters" and "replacements," since unlike at quarterback, being the starter doesn't make you the only guy who gets in the game. Instead, we used a simpler method, ranking players at each position in each season by attempts. The players who made up the final 10 percent of passes or runs were split out as "replacement players" and then compared to the players making up the other 90 percent of plays at that position. This took care of the fact that not every non-starter at running back or wide receiver is a freely available talent. (Think of Jonathan Stewart or Randall Cobb, for example.)

As noted earlier, the challenge of any new stat is to present it on a scale that’s meaningful to those attempting to use it. Saying that Tony Romo's passes were worth 40 success value points over replacement in 2014 has very little value without a context to tell us if 40 is good total or a bad one. Therefore, we translate these success values into a number called "Defense-adjusted Yards Above Replacement, or DYAR. Thus, Romo was fifth among quarterbacks with 1,187 passing DYAR. It is our estimate that a generic replacement-level quarterback, throwing in the same situations as Romo, would have been worth 1,187 fewer yards. Note that this doesn’t mean the replacement level quarterback would have gained exactly 1,187 fewer yards. First downs, touchdowns, and turnovers all have an estimated yardage value in this system, so what we are saying is that a generic replacement-level quarterback would have fewer yards and touchdowns (and more turnovers) that would total up to be equivalent to the value of 1,187 yards.

(Note: Prior to the 2008 season, DYAR was translated in terms of points rather than yardage, and old articles will refer to these stats as "DPAR" instead.)
HOW CAN A 16-GAME SEASON BE SIGNIFICANT?

Football statistics can't be analyzed in the same way baseball statistics are. After all, there are only 16 games in a season. Baseball has over ten times more, and even the NBA and NHL offer over five times more. The more games, the more events to analyze, and the more events to analyze, the more statistical significance.

That is true, but the trick is to consider each play in an NFL game as a separate event. For example, Drew Brees played only 16 games in 2014, but in those 16 games he had 692 passing plays (including sacks) and 16 rushing plays (including scrambles) for a total of 708 events. Ian Kinsler in 2014 played in 161 games and had 726 plate appearances. For the most part, a quarterback who plays a full season will have almost the same number of plays as a baseball hitter who plays in most of his team's games.

A running back will have fewer plays than a quarterback, and wide receivers and tight ends will have even fewer. But there should still be enough plays with most starting running backs and receivers to allow for analysis with some significance. As an example, DeMarco Murray ran the ball 392 times in 2014, and was the target of 64 passes (including incompletes), for a total of 456 plays. In general, a starting running back will have 375-450 plays over 16 games. Receivers are used a bit less, and therefore their stats are likely not as accurate. In general, starting wide receivers have 75-150 pass targets over a full season.
ISSUES WITH DVOA/DYAR

DVOA is limited by what’s included in the official NFL play-by-play or tracked by the Football Outsiders game charting project. Because we need to have the entire play-by-play of a season in order to compute DVOA and DYAR, these metrics are not yet ready to compare players of today to players throughout the league’s history. As of this writing, we have processed 26 seasons, 1989 through 2014, and we add seasons at a rate of roughly two per year (the most recent season, plus one season back into history.)

Football is a game in which nearly every action requires the work of two or more teammates -- in fact, usually 11 teammates all working in unison. Unfortunately, when it comes to individual player ratings, we are still far from the point at which we can determine the value of a player independent from the performance of his teammates. That means that when we say, "In 2014, Marshawn Lynch had a DVOA of 23.1%, what we are really saying is “In 2014, Marshawn Lynch, playing in Darrell Bevell’s offensive system with the Seattle offensive line blocking for him and Russell Wilson selling the keeper when necessary, had a DVOA of 23.1%."

With fewer situations to measure, the numbers spread out a bit more, so you'll see more extreme DVOA ratings for part-time players and for measurements of teams in more specific situations (for example, passing on third downs). The charts listing players in order of DVOA have cut-offs for number of attempts, because players with just a handful of plays end up with absurd VOA and DVOA numbers. (In 2014, for instance, Johnny Manziel had a -73.2% passing DVOA in 38 plays.)

Passing statistics include sacks as well as fumbles on aborted snaps. Receiving statistics include all passes intended for the receiver in question, including those that are incomplete or intercepted. At some point, we hope to be able to determine just how much impact different receivers have on completes vs. incomplete passes, but various regression analyses make it clear that both quarterback and receiver have an impact on whether a pass is complete or not. The word passes refers to both complete and incomplete pass attempts.

Unless we say otherwise, all references to third down also include the handful of rushing and passing plays that take place on fourth down (primarily fourth-and-1).
DVOA FOR SPECIAL TEAMS

The problem with a system based on measuring both yardage and yardage towards a first down, of course, is what to do with plays that don't have the possibility of a first down. Special teams are an important part of football and we needed a way to add that performance to the team DVOA ranking. Our special teams metric includes five separate measurements: field goals (and extra points), net punting, punt returns, net kickoffs, and kick returns.

The foundation of most of these special teams ratings is the concept that each yard line has a different value based on how the likelihood of scoring changes with better field position. In Hidden Game, the authors suggested that the value of field position for the offense existed on a straight line with your own goal line being worth -2 points, the 50-yard line 2 points, and the opposing goal line 6 points. (-2 points isn't just the value of a safety; it also reflects the fact that when you are backed up in your own zone, you are likely going to see your drive stall, and you'll need to punt and give the ball to the other team in good field position. Thus, the defense is more likely to score next.) We use a more refined set of values based on our research, but the idea is the same.

Our special teams ratings compare each kick or punt to the league average for based on the point value of field position at the position of each kick, catch, and return. We've determined a league average for how far a kick goes based on the yard line from where the kick occurs (almost always the 35-yard line for kickoffs, variable for punts) and a league average for how far a return goes based on both the yard line where the ball is caught and the distance that it traveled in the air.

The kicking or punting team is rated based on net points compared to average, taking into account both the kick and the return if there is one. Because the average return is always positive, punts that are not returnable (touchbacks, out of bounds, fair catches, and punts downed by the coverage unit) will rate higher than punts of the same distance which are returnable. (This is also true of touchbacks on kickoffs.) There are also separate individual ratings for kickers and punters that are based only on distance and whether the kick is returnable, otherwise assuming an average return in order to judge the kicker separate from the coverage.

For the return team, the rating is based on how many points the return is worth compared to average, based on the location of the catch and the distance the ball traveled in the air. Return teams are not judged on the distance of kicks, nor are they judged on kicks that cannot be returned. As explained below, blocked kicks are so rare as to be statistically insignificant as predictors for future performance and are thus ignored.

Field goal kicking is measured differently. Measuring kickers by field goal percentage is a bit absurd, as it assumes that all field goals are of equal difficulty. In our metric, each field goal is compared to the average number of points scored on all field goal attempts from that distance over the past 15 years. The value of a field goal increases as distance from the goal line increases.

Kickoffs, punts, and field goals are then adjusted based on weather and altitude. It will surprise no one to learn that it is easier to kick the ball in Denver or a dome than it is to kick the ball in Buffalo in December. Because we do not yet have enough data to tailor our adjustments specifically to each stadium, each one is assigned to one of four categories: Cold, Warm, Dome, and Denver. There is also an additional adjustment dropping the value of field goals in Florida (because the warm temperatures allow the ball to carry better) and raising the value of punts in San Francisco (because of those infamous winds).

The baselines for special teams are adjusted in each year for rule changes such as the introduction of the special-teams-use-only “k-ball” in 1999 as well as the move of the kickoff line from the 35 to the 30 in 1994 and then back to the 35 in 2011. Baselines have also been adjusted each year to make up for the gradual improvement of kickers over the last two decades.

Once we’ve totaled how many points above or below average can be attributed to special teams, we translate those points into DVOA so the ratings can be added to offense and defense to get total team DVOA. As a final step, we then normalize special teams DVOA to reflect the league environment in a given year. It should be noted, however, that we don’t yet have a method to perfectly normalize each year of special teams, so the league average for special teams deviates from 0.0% in certain years, although not by more than 0.5%.

There are three aspects of special teams that have an impact on wins and losses, but don’t show up in the standard special teams rating because a team has little or no influence on them. The first is the length of kick¬offs by the opposing team, with an asterisk. Obviously, there are no defenders standing on the 35-yard line, ready to block a kickoff after the whistle blows. However, over the past few years, some teams have delib¬erately kicked short in order to avoid certain top return men, such as Devin Hester and Josh Cribbs. The special teams formula now includes adjustments to give teams extra credit for field position on kick returns if kickers are deliberately trying to avoid a return.

The other two items that special teams have little control over are field goals against your team, and punt distance against your team. Research shows no indication that teams can influence the accuracy or strength of field-goal kickers and punters, except for blocks. As mentioned above, although blocked field goals and punts are definitely skillful plays, they are so rare that they have no correlation to how well teams have played in the past or will play in the future, thus they are included here as if they were any other missed field goal or botched punt, giving the defense no additional credit for their efforts. The value of these three elements is listed separately as “Hidden” value.

Special teams ratings also do not include two-point conversions or onside kick attempts, both of which, like blocks, are so infrequent as to be statistically insignificant in judging future performance.
ADJUSTED LINE YARDS EXPLAINED

One exception to the use of DVOA/DYAR, and the use of "play success" instead of raw yardage, is the rating system for offensive and defensive lines. Actually, these are only measures of running plays, and of course the defensive numbers don't measure just the defensive line, but the whole front seven against the run.

One of the most difficult goals of statistical analysis in football is somehow isolating how much responsibility for a play lies with each of the 22 men on the field. Nowhere is this as obvious as the running game, where one player runs while up to nine other players -- including wideouts, tight ends, and fullback -- block in different directions. None of the statistics we use for measuring rushing -- yards, touchdowns, yards per carry -- differentiate between the contribution of the running back and the contribution of the offensive line. Neither do our advanced metrics DVOA and DYAR.

We have enough data amassed that we can try to separate the effect that the running back has on a particular play from the effect of the offensive line (and other offensive blockers) and the effect of the defense. A team might have two running backs in its stable: RB A, who averages 3.0 yards per carry, and RB B, who averages 3.5 yards per carry. Who is the better back? Imagine that RB A doesn't just average 3.0 yards per carry, but gets exactly 3 yards on every single carry, while RB B has a highly variable yardage output: sometimes 5 yards, sometimes -2 yards, sometimes 20 yards. The difference in variability between the runners can be exploited to not only determine the difference between the runners, but the effect the offensive line has on every running play.

We know that at some point in every long running play, the running back has gotten past all of his offensive line blocks. From here on, the rest of the play is dependent on the runner's own speed and elusiveness, combined with the speed and tackling ability of the defensive players. If Frank Gore breaks through the line for 50 yards, avoiding tacklers all the way to the goal line, his offensive line has done a great job -- but they aren't responsible for most of that run. How much are they responsible for?

For each running back carry, we calculated the probability that the back involved would run for the specific yardage on that play, based on that back's average yardage per carry and the variability of their yardage on every play. We also calculated the probability that the offense would get the yardage based on the team's rushing average and variability without the back involved in the play, and the probability that the defense would give up the specific amount of yardage based on its average rushing yards allowed per carry and variability. For example, based on his rushing average and variability, the probability in 2004 that Tiki Barber would have a positive carry was 80% while the probability that Giants would have a positive carry without Barber running was only 73%.

Yardage ends up falling into roughly the following combinations: Losses, 0-4 yards, 5-10 yards, and 11+ yards. In general, the offensive line is 20% more responsible for lost yardage than it is for yardage gained up to four yards, but 50% less responsible for yardage gained from 5-10 yards, and not responsible for yardage past that. Thus, the creation of Adjusted Line Yards.

Adjusted Line Yards take every carry by a running back and apply those percentages. (We don’t include carries by receivers, which are usually based on deception rather than straight blocking, or carries by quarterbacks, which are almost always busted passing plays unless they involve someone like Colin Kaepernick or Cam Newton.) Those numbers are then adjusted based on down, distance, situation, opponent and whether or not a team is in the shotgun. (Because defenses are generally playing pass when the quarterback is in shotgun, the average running back carry from shotgun last year gained 5.36 yards, compared to just 4.16 yards on other carries.) The adjusted numbers are then normalized so that the league average for Adjusted Line Yards per carry is the same as the league average for RB yards per carry (in 2013, 4.10 yards).

Runs are listed by the NFL in seven different directions: left/right end, left/right tackle, left/right guard, and middle. Further research showed no statistically significant difference between how well a team performed on runs listed middle, left guard, and right guard, so we also list runs separated into five different directions. Note that there may not be a statistically significant difference between right tackle and middle/guard either, but until we can research further (and for the sake of symmetry) we do still split out runs behind the right tackle separately.

These splits allow us to evaluate subsections of a team’s offensive line, but not necessarily individual linemen, as we can’t account for blocking assignments. We don't know when a guard is pulling and when a guard is blocking straight ahead. We know that some runners are just inherently better going up the middle, and some are better going side to side, and we can't measure how much that impacts these numbers. We have no way of knowing the blocking contribution made by fullbacks, tight ends, or wide receivers.

Other numbers we use to measure the running game:

Open Field Yards gives the portion of the team's rushing average gained after the first 10 yards of each run. So for a 10-yard run, no yards are counted; for a 15-yard run, five yards are counted; for an 80-yard run, 70 yards are counted. This number gives you an idea of how much of a team's running game was based on the breakaway speed of the running backs. A team with a low ranking in Adjusted Line Yards but a high ranking in Open Field Yards is heavily dependent on its running back breaking long runs to make the running game work. This number is not adjusted in any way.
Second Level Yards gives the portion of the team's rushing average gained 5-10 yards past the line of scrimmage. So for a five-yard run, no yards are counted; for a 10-yard run, five yards are counted; for an 80-yard run, still only five yards are counted. This number represents the midpoint between the contributions of a team's offensive line and its running back. A low ranking in Open Field Yards coupled with a high ranking in Second Level Yards indicates that the team's offensive line is opening holes, but its backs aren't proficient open-field runners. This number is not adjusted in any way.

Power success measures the success of specific running plays rather than the distance. This number represents how often a running attempt on third or fourth down, with two yards or less to go, achieved a first down or touchdown. Since quarterback sneaks, unlike scrambles, are heavily dependent on the offensive line, this percentage does include runs by all players, not just running backs. This is the only stat given that includes quarterback runs. It is not adjusted based on game situation or opponent.
Stuffed measures the percentage of runs that are stopped at or before the line of scrimmage.

DRIVE STATS

The stats section of our website also features drive stats compiled by Jim Armstrong. These stats are computed from NFL Drive Charts and are not adjusted for strength of schedule or situation. Take-a-knee drives at the end of halves are discarded. Drive stats are generally self-explanatory, giving each team's total number of drives as well as average yards per drive, points per drive, touchdowns per drive, punts per drive, and turnovers per drive, interceptions per drive, and fumbles lost per drive. LOS/Drive represents average starting field position (line of scrimmage) per drive from the offensive point of view. Drive stats are given for offense and defense, with NET representing simply offense minus defense.
A NOTE ON PLAY-BY-PLAY DATA

Our data may differ slightly from official NFL numbers due to discrepancies in different play-by-play reports. In addition, we've adjusted clock plays, with kneels no longer counting as rush attempts and spikes no longer counting as pass attempts. We also count most aborted snaps as passing plays, not rushing plays, unless the play-by-play specifies that the play was an aborted handoff.

Data Provided By:Data Provided By Sports Info Solutions

Homers, huh. So you've got to be insanely critical of everything the Bills do to be correct, right? LOL. So funny.

BertSquirtgum · May 26, 2019

Here, we have a know nothing FAGGOT.

junglesouljah · May 26, 2019

What bothers me a bit is that he can blame the data if he's wrong. For example: https://t.co/RQcg1oiOj5
— Ryan Talbot (@RyanTalbotBills) May 25, 2019

This is hilarious when he is wrong he makes an excuse and blames "an error in our BackCAST system" So Schatz was wrong about Alvin Kamara using his own stats and wasn't even man enough to admit it. Sorry, I don't see Analytic experts like Schatz being the authority on prospect evaluation.

As long as we're at it, the biggest error in the history of SackSEER was Jason Pierre-Paul, the biggest error in Playmaker Score was Stephen Hill, and the biggest error in QBASE is probably Johnny Manziel or Brian Brohm.
— Aaron Schatz 🏈 (@FO_ASchatz) May 23, 2019

Maybe some people shouldn't blindly believe everything the analytics experts tell them.

[video=youtube;lQ11wnEPG-Q]

LOL, Schatz thinks Jimmy Garoppolo is overrated.

SackMan518 · May 26, 2019

I mean guys, his name is Schatz. That's the past tense of dropping a deuce somewhere.

I do not see a NFL starting QB in Buffalo Bills’ Josh Allen, says ESPN’s Aaron Schatz

Recommended Posts

SackMan518 3,812

Share this post

Link to post

Share on other sites

DC2007 230

Share this post

Link to post

Share on other sites

Down and Out 846

Share this post

Link to post

Share on other sites

nealpellecchia@yahoo 1,132

Share this post

Link to post

Share on other sites

CBennett 15,993

Share this post

Link to post

Share on other sites

junglesouljah 58,498

Share this post

Link to post

Share on other sites

Mainejays 1,483

Share this post

Link to post

Share on other sites

junglesouljah 58,498

Share this post

Link to post

Share on other sites

junglesouljah 58,498

Share this post

Link to post

Share on other sites

junglesouljah 58,498

Share this post

Link to post

Share on other sites

Meathead 71,416

Share this post

Link to post

Share on other sites

Topas 5,351

Share this post

Link to post

Share on other sites

scotsho 1,981

Share this post

Link to post

Share on other sites

Meathead 71,416

Share this post

Link to post

Share on other sites

212frawk 7,794

Share this post

Link to post

Share on other sites

HipKat 971

Share this post

Link to post

Share on other sites

Bocephus 546

Share this post

Link to post

Share on other sites

Bocephus 546

Share this post

Link to post

Share on other sites

junglesouljah 58,498

Share this post

Link to post

Share on other sites

junglesouljah 58,498

Share this post