Donna Laframboise asks, "How many scientific papers just aren’t true? Enough that basing government policy on ‘peer-reviewed studies’ isn’t all it’s cracked up to be."
We’re continually assured that government policies are grounded in evidence, whether it’s an anti-bullying programme in Finland, an alcohol awareness initiative in Texas or climate change responses around the globe. Science itself, we’re told, is guiding our footsteps.
There’s just one problem: science is in deep trouble. Last year, Richard Horton, editor of the Lancet, referred to fears that ‘much of the scientific literature, perhaps half, may simply be untrue’ and that ‘science has taken a turn toward darkness.’
It’s a worrying thought. Government policies can’t be considered evidence-based if the evidence on which they depend hasn’t been independently verified, yet the vast majority of academic research is never put to this test. Instead, something called peer review takes place. When a research paper is submitted, journals invite a couple of people to evaluate it. Known as referees, these individuals recommend that the paper be published, modified, or rejected.
If it’s true that one gets what one pays for, let me point out that referees typically work for no payment. They lack both the time and the resources to perform anything other than a cursory overview. Nothing like an audit occurs. No one examines the raw data for accuracy or the computer code for errors. Peer review doesn’t guarantee that proper statistical analyses were employed, or that lab equipment was used properly. The peer review process itself is full of serious flaws, yet is treated as if it’s the handmaiden of objective truth.
And it shows. Referees at the most prestigious of journals have given the green light to research that was later found to be wholly fraudulent. Conversely, they’ve scoffed at work that went on to win Nobel prizes. Richard Smith, a former editor of the British Medical Journal, describes peer review as a roulette wheel, a lottery and a black box. He points out that an extensive body of research finds scant evidence that this vetting process accomplishes much at all. On the other hand, a mountain of scholarship has identified profound deficiencies.
. . .
Politicians and journalists have long found it convenient to regard peer-reviewed research as de facto sound science. Saying ‘Look at the studies!’ is a convenient way of avoiding argument ... We’ve long been assured that reports produced by the UN’s Intergovernmental Panel on Climate Change (IPCC) are authoritative because they rely entirely on peer-reviewed scientific literature. A 2010 InterAcademy Council investigation found this claim to be false, but that’s another story. Even if all IPCC source material did meet this threshold, the fact that one academic journal — and there are 25,000 of them — conducted an unspecified and unregulated peer review ritual is no warranty that a paper isn’t total nonsense.
If half of scientific literature ‘may simply be untrue’, then might it be that some of the climate research cited by the IPCC is also untrue? Even raising this question is often seen as being anti-scientific. But science is never settled. The history of scientific progress is the history of one set of assumptions being disproven, and another taking its place.
There's more at the link. Ms. Laframboise's full report may be read here (the link is to an Adobe Acrobat document in .PDF format).
This is precisely why I profoundly distrust any politician who tries to tell us that 'the science is settled'. All too often, it's far from settled. It may even be actively and deliberately fraudulent, producing results tailor-made to satisfy the objectives of those who've funded the research. Too many 'researchers' begin with a goal in mind, their conclusions already identified, and then seek evidence that will substantiate what they want to prove. Anything to the contrary is ignored or discarded, or flagrantly manipulated to achieve the desired result (as in this example, to cite just one - there are many more).
That's not research at all. It's pseudo-scientific sleight of hand. It's a shell game.
Peter
12 comments:
Look, I'm not contesting there's to much "book cooking" in the scientific community in order to push pet theories. However I prefer to rely on personal experience and good old common sense whenever possible to evaluate controversial statements. In the case of global warming I think there are so many statements supported by "scientific studies" that the "inconvenient truth" is you can't believe any of them. What I know is that I'm a 49 years guy living in central Europe and I remember the climate was different during my childhood and youth! I remember winters with at least 50cm snow levels were common and summer temperature rarely climbed over 35 degree Celsius. Nowadays we rarely get 10cm snow during the winter and temperature peaks over 40 degree Celsius are starting to become common.
I don't think cooking the books helps the climate change proponents, but ignoring the reality doesn't help the other side either. Making a political hoolabaloo out of the problem is the probably a stupid thing, redirecting the interest from the real problem to the political debate, but taking some positive action like trying to reduce the CO2 footprint shouldn't be demonized. I'm thinking on the reduction of the dependency on fossil fuels (which by the way would also cut into the financial strength of the supporters of "militant action" - see how politically correct I can be :-)), on new and cleaner technologies, on new industries for these technologies, spurring the limp economies.
Speaking from experience, it is ridiculously easy to get onto the review panel at some journals. In fact, many journals solicit reviewers, because to do a good review of even a single article takes quite a bit of time, so most people aren't all that willing to put the time and effort in for very many articles.
Unfortunately, that means that most reviewers for many articles aren't going to be reviewing something that is in their wheelhouse. The reviewers may be able to look for internal consistency within the article, how the statistics were compiled, and how the conclusions were obtained, and they may be able to point out problems in data collection in a general sense, but they may still miss major fallacies in the base premise.
Peter:
As you pointed out, anyone who says the science is settled is lying. There is no such thing.
The speed of light, sometimes called The universal speed limit? Scientists have made light all but stop in the laboratory, reducing the speed of light to a speed measurable in meters per YEAR. The gravitational constant? Turns out it's not.
The truest definition of a scientist is one who is willing to look at data that does not support their theory and, when validated and verified, junk their theory, or law, or understanding, and start over from scratch. Anyone unwilling is not a scientist.
There is no such thing as settled science. Everyone who tells you that is either stupid or lying to your face.
FormerFlyer
What C.G.R. is saying reveals the problem. The fact that we humans must be messing with environment "just feels right" and people like to believe it. Because at 49 years, you remember that the weather was different when you were a child, you think that the world's climate is/may be changing. That's not logical. Global climate and local weather are completely different things. Weather always changes, and the fact that you're human means your life cycle is too short to experience those variations. How do you know the winters you remember weren't unusually snowy compared to historical winters? How do you know that there isn't a completely "natural" (not man made) cycle lasting several decades that causes these changes? How do you really know that reducing CO2 does anything positive? The world has seen CO2 levels very different from today's over its history. If your plants could talk, they'd argue that CO2 is food, and they'd rather we not cut it back.
And even if the world is warmer today than when we ended the "Little Ice Age", why do we think that's a bad thing? Historically, cold is the killer, not warm.
These are the sort of questions the science is supposed to be answering, but isn't. The fact is that all of the predictions from climate science are based on computer models, which are simple simulations of very complex situations. Their record of predicting correctly is awful. Not one of them predicted the nearly 20 year long hiatus with no increase in global average temperatures we've been going through. All of the models assume positive feedback mechanisms that have never been shown to exist and all give a temperature vs. altitude distribution in the tropics that simply does not exist. Even the IPCC says the situation is too complex to model reasonably.
In short, there's no evidence that the global climate is really changing in any novel way.
I first became a "skeptic" (burn him!) when I read they were continually changing the temperature records. I can see filling in a missing data point or two, once, but going back to the records and changing them over and over just smells like cheating.
Speaking from personal email experience is no way to comment on climate. Climate is a centuries or millennia long average. The variations between decades are statistical blips. Yes, it was colder and snowier when I was a child. That was near the end of a decads-long drop in temperatures, which reversed itself and returned to the long term average. But no person can know that from personal experience.
Yes, the climate is slowly changing. That's normal, natural, and a good thing. Britain was once known for good weather and excellent grapes. There was once a civilization under what is now the North Sea. There used to be a mile if ice above where I'm sitting right now. Civilization exists because if the ending of the last ice age - which warming is still ongoing. Having ice at the north pole in the summer is an historic oddity, after all.
Yes, science has given itself a bad name and a black eye. That's what happens when you become more concerned with pleasing your masters and the people who pay the bills than you are with speaking the truth. Scientists are merely human, and they certainly aren't saints.
The Science is SETTLED, I tell you! It may not be denied, because PEER REVIEW!!!
Now, then, let me introduce you to the new and updated food pyramid, which is totally accurate and not a reflection of the donations from any large food product manufacturers, GMO-interest groups, ag-state lobbyists, or the pet theories of the First Lady....
Seriously, though, using the food pyramid to illustrate government science is something that resonates with many people, because it's something they can relate too.
Peer review was never meant to be a deep vetting of an article; it was supposed to establish that the jargon was used correctly and that the conclusion drawn weren't contradicted by the published numbers. Holding it out as making an article un-challengeable is a lie perpetrated by quacks and watermelons.
Rolf Please deposit all pyramid material in the nearest memory hole. It is Myplate. The definitive government guide to how you should eat!
We have always been at war with EastAsia.
Doubleplusgood!!!!!
Have you turned in a thoughcriminal today?
THOTpolice
As a person who has worked in academia, you are right to be suspicious. Students who can't remember to calibrate equipment, poorly written operating procedures, not consistently following those procedures, and too many other factors. Not like the Phd sat there with their student crew to cross check everything.
There are thousands of engineering and science journals. No one could possibly read all of that crap and believe the majority of it is useful.
greenland used to be green. maybe it will be green again.
I'm guessing that the scientific community is pretty much like any other community ...inundated by numerous accounts of favoritism.
You see it in the classroom; in the workplace; in the neighborhoods; in business and industry; most certainly in politics; most certainly in the media.
To most people the social and diplomatic aspects of interaction trumps the factual and down-to-earth transparency ones.
Ok, so from the general tenor of the comments it seems we more-or-less agree to be at least very suspicious on the "scientific facts" presented by the politicians. What seems to be into dispute is not if there is something true about climate change and if some kind of action is justified or not.
For you guys living in the US there may be no perceived difference in weather patterns, but for Europeans there are some differences over the last decades. I'm not saying it's not part of some normal cycle - I simply don't know. But hearing someone saying the CO2 emissions have no significant influence and we don't need to do anything about them leaves me underwhelmed with that person's intelligence...
Post a Comment