Searching for Sports' Holy Grail

Last summer, a trainer for an English Premier League club, who asked to remain anonymous, was looking for a way to upgrade his club's injury prevention system prior to the season. All the information the training staff collected through regular examinations was scattered among the staff, organized into an Excel spreadsheet and then analyzed by an intern, which took too long. Players were already practicing by the time the staff could determine whether they were fit to be on the field.

The trainer then recalled a company he knew from his days working as a physio in rugby: Kitman Labs, a relatively new player in the injury prevention field. He was vaguely aware there were other companies out there, but none offering what Kitman could: a personalized injury alert system for each athlete. His team signed up with Kitman.

Now, every morning, the first thing players do when they arrive at the facility at 9 AM is check in via the Kitman system. At one of two Microsoft Kinect stations—the same type used with Xbox gaming consoles—players rotate their pelvises and stretch their hamstrings, with the camera logging imperceptible changes in their motion and flexibility. Then, on one of three tablet stations, they fill out a short survey about their sleep, diet, soreness, and overall well-being. Within seconds, the trainers have the day's information available at their fingertips through Kitman's online dashboard, which highlights potential problem areas. That is, within seconds, the trainers know who's stiff, who's sore, and who's good to go.

This EPL club is one of many teams around the world to sign up with Kitman in the past few years, including the Los Angeles Dodgers, Tampa Bay Rays, and Miami Dolphins. The goal is to stem the tide of rising injury rates across sports.

Over the last decade, injuries have only gotten more frequent while salaries continue to rise. In the NFL, Football Outsiders' found that injury totals have increased league-wide four years in a row to 74.3 adjusted games lost in 2014 (weighting for starters and injury reports), a 10 percent increase over the previous year. A Fangraphs post from last year showed MLB's total days lost to injuries has increased in the last decade. Similarly, a UEFA study of top-tier soccer teams found the average team suffered 3.4 injuries per 1,000 training hours; that number jumps to 23.2 injuries per 1,000 hours when looking only at matches. The stakes are high, and yet despite advances in exercise regimens and dieting, athletes continue to get injured at record rates.

Naturally, teams want to reverse these trends, and Big Data is making big promises. Professional teams across the globe are attempting to optimize everything from on-field strategy to roster moves to ticket prices. It was only a matter of time until the focus turned to injury prevention, which writer Zach Lowe called the "Holy Grail of sports science and analytics."

Kitman Labs is just one of several companies in this growing industry seeking to profit from the so-called Holy Grail; nearly every pro team and many college programs are utilizing professional injury prevention technology in one way or another.

Yet, as companies and teams rush to collect more and more data—a worrisome trend for players' unions who are concerned about the amount of information collected—players themselves continue to get injured at high rates. The question is no longer whether injury prevention systems are possible, but to what extent they actually work.

In 2008, Stephen Smith, the founder of Kitman Labs, was working as a trainer for Leinster Rugby in Dublin. The team collected massive amounts of data during training for injury prevention purposes, largely via a GPS that measured players' accelerations, decelerations, and overall load, similar to the SportVU technology installed in NBA arenas last season. But all this information was simply too much. Smith called it "paralysis by analysis," meaning they spent all their time looking at the data without coming to any actionable conclusions, a familiar problem for any big data team.

While at Leinster, Smith experienced first-hand the minefield of injury prevention, a science riddled with false positives and confirmation bias. For example, Smith would look at the data for a player with a classic fatigue injury, such as a strained muscle, to try and pinpoint how it could have been prevented. He might see the player executed more accelerations and decelerations than normal—often referred to as the player's workload or "load" for short—in the prior training sessions, adding stress and fatigue to the muscle which he eventually strained. Problem solved, right?

But Smith observed that other players executed the same number, or even many more, of such movements, rendering the conclusion moot. Ultimately, Smith determined they still didn't have enough information despite all the data they were collecting. "The more and more we started to dig into this, the more we realized that it's a hugely multifactorial issue, and looking at one particular data point and trying to base all your decisions off that is a really bad idea."

While still at Leinster, Smith got his Master's in 2011 working on this very issue. Over a three year period, he studied 64 factors that, combined, might affect injury onset. His findings: none of them directly correlated with injuries. Smith concluded that humans are simply far too unique to create a one-size-fits-all injury predictor.

This realization led Smith to found Kitman Labs and its profiler system, the element that Smith believes sets Kitman apart from its competitors such as Catapult, an Australian injury prevention company founded in 2006 and perhaps the injury prevention industry's most prominent practitioner. Catapult boasts dozens of North American pro sports clients across all the major sports, although it doesn't use as many data points as Kitman, particularly the self-reported measures. But does it matter?

Teams, such as our EPL example, are tasked with figuring this out every day. Once all the team's players check in, the medical staff surveys the results. Kitman's system automatically flags potential problems. An alert system will tell the training staff if a player's hamstring is less flexible than it was a week, two weeks, or even a few days prior, if he hasn't slept well for a few consecutive nights, or if he has reported muscle soreness.

This is where the system's role ends and the trainers take over. "It doesn't tell you what their problem is, but it just highlights that there may be a problem," the EPL trainer says. He then approaches the player, examines them, and asks follow-up questions. It could be that the player's hamstring flexibility was below average because he's stiff from overexertion and should rest. Or, the player may not even notice it and is able to fully perform. Perhaps the player isn't sleeping well because he's constantly sore, or maybe his infant has been waking up screaming at 3 AM. It's up to the trainer to find out.

In difficult cases—say, a player's hamstring flexibility is a little low, he's played a bit more than average recently, but he isn't sore or noticing any ill effects—the trainer has to make a call based on what amounts to little more than a best guess. What happens next could change a team's season. If the player pulls his hamstring that practice, the trainer will second-guess his decision all season. But if everything goes fine, he may never think about that decision again.

This highlights a fundamental truth about injury prevention: it is all about managing risk, not predicting outcomes. Even the most advanced technology available such as Kitman's algorithms "are not a crystal ball," as Smith put it. Even with all the new data, it still comes down to the same thing it always has: qualified trainers making sound decisions.

"I think, ultimately, data plays a role, but it's not the numbers that are so important," says Tim Gabbett, a sports scientist who has consulted with several teams around the world and published over 150 peer-reviewed articles. "It's actually how you analyze those numbers, how you interpret those numbers, and then, how you act on them, and that comes down to good people."

The EPL trainer loves Kitman's product, but he's not sure how much impact it's had on preventing injuries. Rosters change year-to-year, and different players have different propensity towards injuries. Plus, the team put other injury prevention measures in place over the summer, which may be helping as well.

"When you talk about preventing an injury, it's hard to prove you actually prevented it because the injury didn't happen," Boden Westover, Catapult's Director of Marketing told me. "So those predictive analytic things are always difficult to confirm and say, yeah, we prevented that injury."

To try and get a best estimate, Catapult and Kitman compare their customers' results on a year-to-year basis. By this measure, their customers are quite pleased. Smith told me their customers see a 20 to 33 percent reduction in injury rates and a 10 percent increase in player availability. Likewise, when the Florida State football team won the National Championship in 2014, head coach Jimbo Fisher credited Catapult with the team's 88 percent reduction in soft tissue injuries.

Regardless of how much can be attributed to their programs, companies like Catapult and Kitman have made a difference. Since previous injury prevention "systems" boiled down to gut feel and asking players how they're feeling, it's hardly a surprise that intelligent, data-driven approaches lead to fewer injuries. In fact, most of the injury prevention experts I spoke with, including those from the companies themselves, say a large part of the job is simply installing a sensible data management system. Gabbett told me one team he worked with recently was keeping all their data in PDF files.

The EPL trainer said the biggest gain from the system comes from having quickly available, easy to digest, and actionable data within minutes. Trainers can decide what to do before the practice even starts, a huge advantage over Excel spreadsheets. "We think it helps in our management of players, that's for sure."

While more and better information on players is certainly a good thing, athletes and their unions are growing concerned about where this trend may lead. Nearly everyone I spoke with in the injury prevention industry told me that, eventually, leagues will want to get all their teams signed up with the same company, with the data then equally disseminated to the teams in some form.

In America, this would have to be collectively bargained with player unions. Of course, athletes want to be as healthy as possible for as long as possible, but not at the expense of surrendering vital information, such as submitting to DNA tests and predictive information that may affect hiring and salary decisions. Even with data Kitman collects, it's easy to see a misguided GM using the self-reported sleep and diet data to assess a player's character or make some other bizarre interpretation.

Either the league or NBPA can opt out of the current collective bargaining agreement in 2016, and player health tracking could be a topic of discussion in any negotiations. "We are monitoring teams' use of wearable technology and collecting feedback from our players about the potential benefits and drawbacks in using this kind of analytic equipment," NBPA Executive Director Michele Roberts told VICE Sports in a written statement. "We're paying keen attention to the data collected and how that data is being used by teams to evaluate players, and want to ensure best practices are in place as more teams use this technology."

The NFL CBA, which expires in 2021, stipulates that the league can make players wear sensors or "non-obtrusive tracking devices for purposes of collecting information regarding the performance of NFL games, including players' performances and movements, as well as medical and other player safety-related data." However, the NFL Players' Association must consent to any sensors used for "health and medical purposes." Teams like the Miami Dolphins using Kitman Labs' services must fill out a consent form for the NFLPA's approval.

For now, Smith downplays the unions' concerns, saying the data Kitman Labs collects wouldn't be useful in contract negotiations or other long-term considerations. The data "is only relevant today."

Still, it's easy to envision a future where coaches or training staff will want mid-practice and game alerts about which players are at risk of injury. In fact, this might not be far away. Kerry MacDonald, a researcher for the University of Calgary Sport Injury Prevention Research Centre, says he has similar tools available to him already when assessing volleyball players. "If they see athletes that are, say, underperforming or their jump values are decreasing and they can see fatigue setting in, substitutions are going to be made, adjustments are going to be made, and that can all happen real-time in game."

No matter what technological advances are around the corner, Gabbett is clear about one thing: there will always be injuries. "The only way you can know for sure that a player is going to break," he says, "is when he breaks."