No pixies were harmed in the making of this post

Dear all,

The title of today's blog (while my sample is incubating) is based on this comment. Those who know me knows that I adore open source and file sharing and all that good stuff and, given my field, it comes as no surprise that I continue to follow PLoS in my spare time (or in this case, because the paper assigned to me about miRNA screening is so dull that I have failed to finish reading the first page over three times already).

For the most part, PLoS and its associated journals, publish decent quality stuff. However, there are occasionally articles like this that leaves me feeling pretty indignant as a scientist. (Go on, click the link: you don't need a subscription to read it and the people did a pretty good job of summarizing the research in the paragraphs that has a blue bar on the left.)

As a scientist, this is how this paper, which claims that middle-aged women who drink a small amount of alcohol are more likely to "age successfully", leaves me cold:

1) The people studied are registered nurses from 1976. Though they do try to account for things like age, education, and marriage, that is in no way enough to act as adequate control for population stratification between people who drink and people who don't. Social background and ethnicity do play a major role, and before anyone calls me a racist, I would like to point out that this is biomedically relevant fact: there's a higher occurrence of lactose intolerance in Asians, people with blue eyes are more susceptible to certain kinds of eye disease, that kind of things. Our skin color and hair type and what have you has a genetic background that provides us with unique advantages and disadvantages, so unless the authors of the paper provides proof that there is no significant different between the population characteristics in the control group vs the test group, I will take ANY conclusion they draw with a rather large chunk of metaphorical salt.

2) MATH. Math. Math. (Okay, so it's statistics -- don't hit me.) They failed to provide details to their method of analysis, which in itself isn't too surprising. The lack of detail is an unfortunate common theme. However, how they set up the data collection and analysis doesn't make much sense to me and probably would have given my PI fits had he seen it (he's into the whole accurate choice of tests thing). The numbers they're crunching, possibly because of the way their survey was set up (maybe stats for biological rigor wasn't invented in the 70s? I for one find that ridiculous, given some of the papers I've seen from the 70s) (it was a classics paper), which means that all the numbers they draw their conclusions on are percentages. (This can be okay only if they ran out the tests right and had really nice data).

This is misleading in two ways: A) percentages of 1% is all very well, but 1 out of a study of a hundred could be an anomaly, where as 1% of the world population is actually quite a significant amount -- I get back to this point later. B) Percentages are not normally distributed numerical data. The tests you can run on them to test for significance are thus less robust than anything you can do had they just changed a few things and made the data categorical instead.

It's like...if you can have more robust data, why wouldn't you?

3) The actual numbers. So they claimed that there is a significant difference between people who "aged successfully" (more on this later) who drinks and those who don't. The differences between the % of people who aged "successfully" and those who didn't, who drank certain amounts of alcohol (oh God oh God why did they arrange the data this way??) has a range of 0.1% to 3%, with their claim that the 2.2% for the 5-15gram/day category is significant. If we set 2.2% as the value we use as a cut off to say that a difference is significant (after all, that's what this paper is claiming), then in their paper's table what, here's what else is "significant:

a) Smoking: "successful" agers have 2.5% more people who'd never smoked, and 6.3% fewer people who are currently smoking.

b)Aspirin using: "successful" group has 5% more people who'd never used aspirin, 6.3% more people who used 1-2 tablet (low amount) per week. (There were fewer people from this group who uses high amount aspirin but it's just below the 2.2% cut off).

c) Education: "successful" agers have 5.4% fewer people who are registered nurses, 2.7% more people who have bachelors, and 2.7% more people who masters. 

d) Marriage: "successful" agers have 2.6% fewer people who are widowed

e) Family history: "successful" agers have 2.3% fewer people with family history of heart disease, 3% fewer people with family history of diabetes

f) Medical history: "successful" agers have 10.7% fewer people with history of hypertension and 3.5% fewer people with history of high cholesterol. 

This is not even getting into the other part, with the amount of exercise per week, where the data is actually very quantitative and I think shows a significant difference where the "successful" agers exercise more per week. 

And while each one of those bulletins may contribute only a small percentage, it's still easy to see, with the exception of aspirin use (yes it's a blood thinner and can indicate heart problems and such but it's also used for headaches and so I'm wary of trying to interpret that), that all of the data points toward known traits: people who don't smoke, have higher education, are happily married, don't have family history of heart disease / diabetes and don't have current medical history of hypertension and high collection are likely to lead longer, happier lives. Duh. 

This is in no way an accurate way to analyze the data, because each point in a)-f) will have different amount of effect on an individual (for instance, marriage and hypertension would not, by most people, be considered with equal weight in terms of longevity. Which side weights more depends on the individual), but just for fun, I added up all the percentages, since they all point in the same direction, and in total the "successful" agers have, barring point b) and c) for registered nurses (who knows if this increases or decreases one's life span?), there's a total of 36.3% worth of health point differences. The difference in alcohol consumption, again, is 0.1-3%.


You should never do what I just did with data and just randomly toss stuff out and add things together for anything you want to publish. You'll be laughed out of the field. My point in that exercise is that there's a lot of factors that the authors did not adequately take into account that could lead to the difference they see between drinking vs non-drinking and "successful" vs. not.

4) Last point: "successful" aging. What is this "success"? (This brings me flashbacks of 9th grade. Oi.) Well according to this paper:

"...we defined successful ageing as meeting all four of the following criteria: (1) no history of cancer (except nonmelanoma skin cancer), diabetes, myocardial infarction, coronary artery bypass graft surgery or percutaneous transluminal coronary angioplasty, congestive heart failure, stroke, kidney failure, chronic obstructive pulmonary disease, Parkinson's disease, multiple sclerosis, or amyotrophic lateral sclerosis; (2) no impairment of cognitive function (TICS score≥31); (3) no physical limitations (no limitations on moderate activities, and no more than moderate limitations on more demanding physical performance measures from the SF-36); and (4) good mental health status (a mental health index on the SF-36 higher than the median of 84 in our participants). Participants who survived to age 70 y, but did not meet the remaining criteria were defined as usual agers."

 Okay so in point 3)? Remember the 2.3% and 3% different in family history of diabetes and heart disease? Remember the 10.7% DIFFERENCE IN HYPERTENSION?? Also, exercise does improve your mental health; this is proven by far more robust papers than this.

So when I read:

These data suggest that regular, moderate consumption of alcohol at midlife may be related to a modest increase in overall health status among women who survive to older ages.

I can't help but headdesk.  Just a little.