Power Rankings FAQ

In February 2009, Derby News Network debuted a new feature: DNN Power Rankings. Each month, we'll bring to you our considered estimation of who, currently, are the very best teams in modern roller derby. Below, find answers to the most likely questions about this feature.

UPDATE: Please consider the first Power Rankings article of 2010 to be an addendum to this FAQ.

How do you determine the DNN Power Rankings?

DNN Power Rankings are determined by the consensus of Derby News Network principals Gnosis, Justice Feelgood Marshall, and Hurt Reynolds. And by consensus, we mean each of us cooks up our own take on the rankings, then we compare them, marvel at the parts where our conclusions were identical, accuse each other of possible mental disorders where we differ, and eventually find common ground.

Through it all, our guiding principle is simply this: If two given teams meet today, which one do we think is more likely to win?

Are these rankings official?

What do you mean by "official," exactly? Despite some incipient organizational structure, modern roller derby is still a very ad-hoc, DIY creature. We are all still pretty much making it up as we go along -- and that is a good thing.

If you're asking if these rankings have any particular standing with the sport's main governing body, the Womens Flat Track Derby Association, the answer is no -- WFTDA has its own official ranking methodology. We're doing this on our own prerogative, to meet needs that we feel are currently unserved. Read on...

Aren't there already other ranking systems?

Absolutely. The two main other ranking systems in place are:

1) WFTDA rankings. These are determined on a quarterly basis by a vote of WFTDA member leagues, one vote per league. Starting with the 2008 fourth-quarter rankings, each league votes only to rank teams within its own region -- WFTDA no longer ranks teams quarterly on a national basis. Leagues that have not met minimum participation requirements may lose eligibility for ranking in the WFTDA system.

2) Flat Track Stats. These are calculated continuously, based on a carefully crafted algorithm which is routinely tweaked as shortcomings in the system manifest themselves. Flat Track Stats is driven purely by math, removing the possibility of judgment bias but increasing the difficulty of determining which bouts are most indicative of a team's actual skill.

So, what makes you think there's some need for another set of rankings?

Each of the existing systems serves a vital purpose, and will continue to do so. WFTDA's organic rankings primarily serve to determine seeding into the organization's regional tournaments. Flat Track Stats takes a purely analytical approach, which reveals trends that can otherwise be pretty opaque within a wall of derby data.

That said, they leave some very interesting ground uncovered. WFTDA's rankings, for well-considered operational reasons, tend not to include all teams (except for the ranking period immediately preceding regional tournaments). The lack of a nationwide comparison and relative infrequency of updates also leave many skaters and fans wanting.

Flat Track Stats, for all its analytical rigor, struggles (like any purely mathematical approach must) with the challenge of attempting to simulate reasoned judgment through programming. Ask any NCAA football fan how they feel about computerized rankings, and you're guaranteed to get an earful. It's virtually impossible to distill into code the human instinct for which bouts "matter," as the FTS designers well recognize and discuss in depth on their dev blog.

Wow, you sure do hate those ranking systems, don't you?

Uh, no, not hardly! Both do exactly what they are intended to do, and each does it very well.

We cannot emphasize enough how much respect and admiration we have for both WFTDA and for Flat Track Stats. In WFTDA's brief existence, the organization has established standard track dimensions (PDF), spearheaded intercity bouting, developed a widely-used standardized ruleset, instituted safety and training standards (PDF), and generated a wide variety of best practices (PDF) without which the sport likely could not have thrived so quickly and so globally.

Flat Track Stats is one of our very favorite derby sites, and without their innovative tools -- especially the Rank Wrench -- the prospect of even trying to assemble our own rankings kinda makes us want to blow our brains out. We owe a great debt of gratitude to both Flat Track Stats and WFTDA, and we think you do too.

What we hope to provide with DNN Power Rankings is a frequently-updated, well-considered analysis of which teams are the most highly capable competitors in modern roller derby -- all other considerations aside.

What will you be using these rankings for?

Backdated to Friday, January 30th 2009, and going forward, we'll prepend each ranked team's name with the team's DNN Power Ranking that was in effect at the time of the bout in all previews, recaps, and sitewide for bout and score listings. For bouts prior to that date, we'll still show the WFTDA ranking in effect at that time.

What teams are eligible for a DNN Power Ranking?

If your team plays roller derby as a sport, you're eligible. Here's what we mean by "as a sport":

• You play by a written ruleset, which is strictly and consistently administered by your officials
• Scoring occurs only during the normal flow of jams (i.e. a competition can't have a chance of being decided by points earned through armwrestling, pillow fights, and so on)
• Severe violations of the rules, specifically including fighting, always result
  in serious consequences for the culprits, such as expulsion or suspension
• Your team has a stable roster that changes infrequently

Wow, that's pretty uptight. How'd you get that no-fun so far up your asses?

We want to be clear about this: we're not telling you how to play. We're not telling you that everything you do on skates has to meet these standards. Some of the most entertaining and enjoyable bout-type activities don't qualify, and we're at the front of the line to thoroughly enjoy and appreciate all the most irreverent and hilarious parts of the sport and the community. We're not telling you that you're "wrong" if you do things differently -- who the hell are we to tell anyone what they should be doing?

All we're saying is, when we set our rankings, we're going to pay attention to the bouts that follow these guidelines, and we're going to largely disregard the rest.

Is this only for WFTDA members? Do we have to play by WFTDA rules?

In principle, DNN Power Rankings are not limited to WFTDA members, or to bouts played by the WFTDA ruleset. In practice, however, the WFTDA ruleset is used for the preponderance of bouts, and as a result the teams with the broadest range of competitive experience tend to use them. Furthermore, the oldest, most seasoned teams tend already to be WFTDA members.

Any team we see consistently holding its own against other ranked teams, regardless of ruleset, will be considered for ranking.

How can my team get ranked?

Well, the most straightforward way to demonstrate that you deserve a spot on the DNN Power Rankings is by defeating a team that's currently ranked. Beyond that, make sure we get the final score! If we don't know about a bout result, we can't factor it into our consideration.

When will DNN Power Rankings be posted?

We'll endeavor to post updated rankings in the first week of each month.

You didn't rank the East Jesus Roller Broadz. I think you're retarded.

That's not a question.

Ok, why didn't you rank the East Jesus Roller Broadz? They beat the West Jesus Immaculate Contraptions, but West Jesus made the rankings and East Jesus didn't!

Probably because we're looking at a much wider data set than just that one bout. Say, for example, that East Jesus beat West Jesus by half a dozen points last month, but two months earlier, West Jesus put a seventy-point pounding on South Bunghole, while the month prior to that South Bunghole clobbered East Jesus by a similar margin. (You might want to draw a picture).

In a case like this, we're inclined, based on the full range of data, to look on the East vs. West bout as an outlier, at least until we see more data points -- East outperforming West either directly or indirectly -- before we're ready to start moving them around.

The oft-overlooked truth is that any linear ranking system is an attempt to map a non-linear data set to a linear scale. In the example above, WJ > SB > EJ does not jibe with EJ > WJ. This is where we put our heads together and apply some human judgment, and this unavoidably involves some subjectivity. If you're hating on subjectivity, you're in luck! See above, "Aren't there already other ranking systems?"

I still think you're retarded.

Fair enough. That's why there's a comments section on each Power Rankings post. Also, before you call us retarded again, please do take heed of the Site Code of Conduct. We welcome reasoned, vigorous, but polite and respectful dialogue throughout our site.

UPDATE: Please consider the first Power Rankings article of 2010 to be an addendum to this FAQ.