Crossfit Open 2014: Stats, Ranks, and Data-Nerding

The CrossFit community is a welcoming community, but it is a welcoming community built upon the competitive nature of a very particular type of person. With every WOD there is competition with yourself (i.e. setting new a PR), competition with the other athletes in your gym (I’m looking at you Lance), and once a year there is a competition among all CrossFit’ers in the form of the CrossFit Games.

I am a member of CrossFit 206 in Seattle. The coaches at 206 have built an environment that is inclusive; 206 caters to both the most intense of athletes (cough, cough Evan) and to those that are just going to have a good time. It is a great blend of people and I am thoroughly grateful. With the start of the Games though I have begun to wonder where in the spectrum of gyms 206 falls. With the wealth of data made available through the Games website it is just too tempting for me not to nerd out and answer that seemingly simple question.1

I am going to preface everything to come with (1) I am not actually learning about the gym per se, but rather about the team the gym puts forth. It is not a minor distinction so I want to make it explicit. (2) For tractability reasons I have only included CrossFit’ers in the USA.

I think the natural starting place is to look at the number of reps that athletes were able to complete in a rather brutal 10 minutes. reps_wod1

The histogram of the performances of athletes is rather striking. The pink-ish bars are the number of women that did a certain number of reps, and the green-ish bars are the equivalent for men. Those distinct peaks in the histogram correspond to the power snatches portion of the workout.

The histogram of all athlete’s performance doesn’t do justice to the distinctness of those peaks. If I zoom in on just the central three peaks I get this: reps_wod1_zoom_not_marked

The leading edge of the peak corresponds to the transition between double-unders and power snatches. It makes sense that with a few seconds left an athlete wouldn’t have time to get set for a snatch and just give up after the double-unders. What strikes me is the dramatic drop off of athletes that finished with 13 or 14 snatches relative to those who finished with between 5 and 12. I’ve marked those drop offs with arrows on this plot. reps_wod1_zoom

The dramatic drop off could be attributed to a couple of things. The first explanation is that athletes who would naturally finish at 13 or 14 kicked in the “after-burners” in order to complete the snatches; I find this credible, though I know at 9:55 in the workout I had absolutely nothing left. What doesn’t jibe with this explanation is why those in reps 5 through 12 didn’t also kick in the after-burners, which would fill in the gap and leave a smoother transition. The second explanation is a little more nefarious in that the judges were a little generous with the clock when the athletes needed to get just that one more rep. Unfortunately I find this explanation both credible and quite likely. Whatever the reason for these striking patterns in these data I find it neat, and I mean what is a second or two among friends? Honestly for 99 percent of people we’re just doing this to have fun, so who cares about a little fudging.

But back to the main question, how does 206 stack up to other gyms? Since the Open is by definition open to athletes of all shapes, sizes, ages, and genders it doesn’t really make sense to compare a gym of mostly 20-something college kids to a gym of equally capable but older athletes. The Open addresses this by creating “Masters” categories and breaking out rankings by gender. I want to take that same idea and go just a bit further. For all the data nerd-ing to follow I am going to normalize2 an athlete’s performance against that of their peers, with a peer defined as an athlete of the same age and gender.

For benchmarking a gym I think the most obvious measure is the average performance of their athletes once age and gender has been accounted for. So what is the average performance of 206’s athletes relative to the average performance of all other gyms? CrossFit 206 is 2,536 out of 3,648 gyms3 competing in the Open; conditional on the inclusive environment 206 has developed I think that is pretty darn good. In case you’re curious where your gym falls in the spectrum the full list can be found here. (link)

Despite CrossFit’s “hardcore” reputation, performance isn’t everything. To me a good mix of people is just as, if not more important than how we stack up in a once a year competition. I like the term “inclusive” but you can use just about any word you want to describe the intangible feel of a gym. Almost by definition the intangible feel of a gym is difficult to quantify but I am going to measure it as (1) the mix of men and women and (2) the dispersion in age of athletes. Once again I feel compelled to caveat these two measures with the fact that an athlete must choose to compete in the Open (i.e. it is not random) and I have a strong suspicion that the choice to compete is related to the age and the expected performance of an athlete.

Despite this limitation I think I find some neat results. The average gym has 32 athletes competing and of those 41 percent of those are women. There is a wide range in the gender mix but very few teams are dominated by either men or women. female_fraction

Taking this one step further, when the women at a gym perform better are the men likely to perform better also? The answer is yes, there is a positive relationship between how the women do in a gym and how the men do. female_male_performance

This is reassuring. My take is that good coaching techniques lead to better outcomes for both men and women. Isn’t this the point of CrossFit? To master fundamental, all purpose exercises that are sport and gender independent?

Now that I’m thinking about performance, I wonder if gender mix affects a gym’s performance? The simple answer is there seems to be a positive relationship between the fraction of women at the gym and the gym’s overall performance, but only to a certain extent. To answer I had to turn to a regression5 based approach. Here is a picture of the gym’s expected performance as a function of the fraction of women in the gym. female_fraction_expected_peformance

I caution against drawing too strong of conclusions, or any conclusions for that matter, from this simple plot. It seems though that gyms that are dominated by either gender do worse than those who have a good mix. I am open to speculation about this from the CrossFit community.

Age dispersion is another dimension of inclusive-ness in my book. I have had my ass handed to me by AARP members as well as kids too young to have a drivers license. CrossFit is an equal opportunity ego-shatterer. A far as gyms go the average age is 32 avg_age_hist

but what I find more amazing is the dispersion in age of the gyms. The average gym has 31 years between its oldest competing athlete and its youngest competing athlete. age_spread_hist

Does this age dispersion have an effect on performance? Do the old-timers impart wisdom on the young bucks? age_spread_expected_performance

Unfortunately the answer is probably not. There doesn’t seem to be a strong relationship between the age dispersion and performance.

So far I have been looking at average gym performance, but as I mentioned earlier we also compete as individuals. I am not going to highlight my spectacularly mediocre performance in 14.1, but I have put together a list of personal rankings. By normalizing by age and gender I think a fair comparison can be made between a high school young’n, a kick-ass mother of two, and a cubicle bound older gent. Here is the full list, but as a heads up, it is a large file. (link) Under this ranking system Pam Kusar (age 53) of CrossFit Akron is number 1 when she completed 352 reps. That is stellar!

This little data-nerding exercise was fun. It satisfied my curiosity. I am not going to read too much into my performance or my gym’s performance on one workout in the Open. To me CrossFit is about having a good time and that’s all I care about.

1 Dear CrossFit Games website, this is the day and age of open data and mashups. Making it easier to collect and build on the wealth of data provided by the games is mutually beneficial.

2 Normalize means I take an athlete’s performance minus the average peer athlete’s performance divided by the standard deviation of their peer group. I require a gym to have at least ten athletes in the Open in order to get into my analysis. I remove athletes who signed up but did not compete in 14.1. Additionally I require ten people to be in a peer age group in order for the athlete to be in the sample.

3 For this analysis a gym must have ten members that competed in the first round of the Open.

4 I measure dispersion as the difference in age between the youngest and oldest member of the gym.

5 I first run a regression of the gym’s performance on female fraction, female fraction squared, average age, average age squared, age spread, and age spread squared. The squared terms are included because I am worried about non-linearities. I then plotted the expected gym performance on the y-axis and female fraction on the x-axis. I then fit a loess regression to show a trend.