What's catchier than "dandelion kids" and "orchid hypothesis"? Read the Science of Success in the December issue of Atlantic. Then read Genetic 'breakthroughs' in medicine are often nothing of the sort
Don't believe everything you read about genes and disease in prestigious journals like Science and Nature, say Marcus Munafò and Jonathan Flint. A lot of it is simply wrong.I don't have time for a longer post. I have looming deadlines at work and at home. But my money is on Enrico Fermi, Marcus Munafò and Jonathan Flint. ;-)
Discuss among yourself.
I think the are a bit hard on the statistical significance of the findings, be it genetics, medicine, or some other field. I will do enough measurements to make sure my results are statistically significant and then stop. This is due to time and budget constraints, not some surreptitious massaging of data.
ReplyDeleteI might have been a bit harsh on the Science of Success folks. But I was very irritable after reading Does exercise reduce your risk of cancer?.
ReplyDelete"scientists studied the health of a group of 2,560 middle-aged Finns over the course of about 17 years. The subjects, all men living in eastern Finland, kept diaries of their daily activities for a year and then went about them.
t the start of the study, none had cancer. By the end, 181 had died of the disease. Parsing the men’s activity levels, the researchers determined that, after controlling for cigarette smoking, fiber and fat intake, age, and other variables, the most physically active men were the least likely to develop cancer, particularly of the gastrointestinal tract or the lung. Even more striking, the intensity of the exercise was key. The more arduous it was, the more protective it proved. Jogging was the most strenuous activity studied, fishing among the least. The men who jogged or otherwise exercised fairly intensely for at least 30 minutes a day had a 50 percent reduction in the risk of dying prematurely from cancer"
Confusing correlation with causality is my pet peeve. Perhaps the people who were still able to jog into middle age were blessed with better health in the first place?
People with autoimmune problems that cause joint pain (and hence can't jog) also tend to experience high rates of certain types of cancer.
The authors projected their biases upon the data.
@Marianne has a good point. The Science of Success article is about a statistical analysis and most people would stop when they get a statistically significant result.
However, the opposing article points out the dishonesty in the field when people publish the statistically significant result w/o publishing all the ways they failed to get one. Perhaps we can blame it on journal page charges. ;-)
A very large fraction of working scientists, probably the majority of those working in biology and the social sciences, think that p-value means "the probability that my result is just statistical noise." This view is shared by a super majority of science journalists, as well. If you think that, then sure a p-value of 0.05 sounds really impressive! Why _wouldn't_ you publish something like that? I've had a number of competely unuseful and laborious exchanges of email with science journalists in which I've tried to present very reasonable, not-very-mathematical counterexamples in which one can get a p-value of .05 for a conclusion which is not only not 95% likely to be right, but is actually more than 95% likely to be merely statistical noise. I've gained no traction.
ReplyDeleteEric
I had a couple thoughts after reading the articles that are loosely related:
ReplyDelete1. I'm used to probabilities being expressed with a confidence interval, or more frequently with the lower bound of the confidence interval as in 95% probability with a 90% confidence level. It seems that the p-values ignore this last bit since they appear to be point estimates. So a p-value of 0.05 seems great but if the confidence level is only 60% then maybe not so much.
2. With regard to genetic makeup (or any life variable) and degree of pre-destination: I'm an adherent of the drift+diffusion theory. There's random elements (diffusion) but also directed elements (drift). Your path the the vector sum.