"It’s very human to think, and to report, that you behaved a little bit better than you really did."

From the February issue of Drovers Cattle/Network.

Southwest Airlines passengers consistently praise the company for serving good food on its flights — but in fact, the airline doesn’t serve any food at all. Does this have implications for a majority of our dietary data, making that data, which relies on self-reporting, suspect? Some researchers say yes, including David Allison, director of the Nutrition Obesity Research Center at the University of Alabama at Birmingham, who relayed this story at a conference. After all, if people don’t accurately recall if they ate, should we trust their memories of what they ate? 

Research into the accuracy of self-reported data gives weight to this suspicion. One reason is the simple fallibility of memory. Self-reported diet data is usually gathered via surveys that might ask questions such as “How much (of whatever particular food) did you eat in the past six months?” It’s easy to imagine that people might not remember correctly.

The tendency to exaggerate desirable behavior — a pattern researchers have long understood — is another problem. It’s very human to think, and to report, that you behaved a little bit better than you really did. (Such outcomes appear consistently in other areas when data is self-reported. For example, studies have shown high levels of inaccuracy with people reporting desirable behaviors such as washing of hands and cutting boards during cooking; many study participants reported practicing these behaviors, even while they were observed failing to practice them. Data on self-reported exercise habits have similarly proved unreliable.) And finally, there’s also the possibility that participants might be purposefully dishonest in their self-reporting.

These problems are not new. Back in 1991, researchers looked at dozens of dietary studies from various countries and found that the majority of reported energy-intake results combined with metabolic-rate measures were scientifically implausible.

But researchers who want data, and lots of it, about what we eat have had to rely on this self-reported information. Now Allison and others are suggesting this data is actually so flawed, it shouldn’t be used at all. In a paper published in the January issue of the International Journal of Obesity, Allison and his fellow authors write, “[The data] are so poor...that they no longer have a justifiable place in scientific research.”

Allison continues, “Using these self-reported values, researchers had concluded for years that individuals with more body mass were eating less food than thinner individuals, which is counterintuitive. Indeed, when using objective, scientific measurements instead of self-reported measurements, it turns out that larger people on average eat more. Using the flawed self-reported values led to incorrect conclusions about physiology and the etiology of obesity, despite the large quantities of flawed data collected.”

But these data underlie not only most obesity research, but public health policies and dietary recommendations (like those suggesting limiting beef consumption) as well. These policies may be wrongheaded, it seems, because they are based on inaccurate data.

Instead of self-reported data, researchers suggest the need to develop more objective measures to track what we are really eating. Some options might include photography — taking photos of our meals — as well as wearable chewing and swallowing monitors and contemporaneous food records kept on cell phones. Maybe such techniques, or some combination of them, will help researchers get more accurate answers about what we’re really eating, and what it means for our health.