Author
|
Topic: The Broken Hockey Stick of Climate Science (Read 695 times) |
|
the.bricoleur
Archon
Posts: 341
Reputation: 8.46
Rate the.bricoleur
making sense of change
 
 
|
 |
The Broken Hockey Stick of Climate Science
« on: 2004-10-26 03:32:10 » |
 |
Global Warming Bombshell
A prime piece of evidence linking human activity to climate change turns out to be an artifact of poor mathematics.
By Richard Muller
Technology for Presidents
October 15, 2004
SOURCE: Technology Review
Progress in science is sometimes made by great discoveries. But science also advances when we learn that something we believed to be true isn’t. When solving a jigsaw puzzle, the solution can sometimes be stymied by the fact that a wrong piece has been wedged in a key place.
In the scientific and political debate over global warming, the latest wrong piece may be the “hockey stick,” the famous plot (shown below), published by University of Massachusetts geoscientist Michael Mann and colleagues. This plot purports to show that we are now experiencing the warmest climate in a millennium, and that the earth, after remaining cool for centuries during the medieval era, suddenly began to heat up about 100 years ago--just at the time that the burning of coal and oil led to an increase in atmospheric levels of carbon dioxide.
I talked about this at length in my December 2003 column. Unfortunately, discussion of this plot has been so polluted by political and activist frenzy that it is hard to dig into it to reach the science. My earlier column was largely a plea to let science proceed unmolested. Unfortunately, the very importance of the issue has made careful science difficult to pursue.
But now a shock: Canadian scientists Stephen McIntyre and Ross McKitrick have uncovered a fundamental mathematical flaw in the computer program that was used to produce the hockey stick. In his original publications of the stick, Mann purported to use a standard method known as principal component analysis, or PCA, to find the dominant features in a set of more than 70 different climate records.
But it wasn’t so. McIntyre and McKitrick obtained part of the program that Mann used, and they found serious problems. Not only does the program not do conventional PCA, but it handles data normalization in a way that can only be described as mistaken.
Now comes the real shocker. This improper normalization procedure tends to emphasize any data that do have the hockey stick shape, and to suppress all data that do not. To demonstrate this effect, McIntyre and McKitrick created some meaningless test data that had, on average, no trends. This method of generating random data is called “Monte Carlo” analysis, after the famous casino, and it is widely used in statistical analysis to test procedures. When McIntyre and McKitrick fed these random data into the Mann procedure, out popped a hockey stick shape!
That discovery hit me like a bombshell, and I suspect it is having the same effect on many others. Suddenly the hockey stick, the poster-child of the global warming community, turns out to be an artifact of poor mathematics. How could it happen? What is going on? Let me digress into a short technical discussion of how this incredible error took place.
In PCA and similar techniques, each of the (in this case, typically 70) different data sets have their averages subtracted (so they have a mean of zero), and then are multiplied by a number to make their average variation around that mean to be equal to one; in technical jargon, we say that each data set is normalized to zero mean and unit variance. In standard PCA, each data set is normalized over its complete data period; for key climate data sets that Mann used to create his hockey stick graph, this was the interval 1400-1980. But the computer program Mann used did not do that. Instead, it forced each data set to have zero mean for the time period 1902-1980, and to match the historical records for this interval. This is the time when the historical temperature is well known, so this procedure does guarantee the most accurate temperature scale. But it completely screws up PCA. PCA is mostly concerned with the data sets that have high variance, and the Mann normalization procedure tends to give very high variance to any data set with a hockey stick shape. (Such data sets have zero mean only over the 1902-1980 period, not over the longer 1400-1980 period.)
The net result: the “principal component” will have a hockey stick shape even if most of the data do not.
McIntyre and McKitrick sent their detailed analysis to Nature magazine for publication, and it was extensively refereed. But their paper was finally rejected. In frustration, McIntyre and McKitrick put the entire record of their submission and the referee reports on a Web page for all to see. If you look, you’ll see that McIntyre and McKitrick have found numerous other problems with the Mann analysis. I emphasize the bug in their PCA program simply because it is so blatant and so easy to understand. Apparently, Mann and his colleagues never tested their program with the standard Monte Carlo approach, or they would have discovered the error themselves. Other and different criticisms of the hockey stick are emerging (see, for example, the paper by Hans von Storch and colleagues in the September 30 issue of Science).
Some people may complain that McIntyre and McKitrick did not publish their results in a refereed journal. That is true--but not for lack of trying. Moreover, the paper was refereed--and even better, the referee reports are there for us to read. McIntyre and McKitrick’s only failure was in not convincing Nature that the paper was important enough to publish.
How does this bombshell affect what we think about global warming?
It certainly does not negate the threat of a long-term global temperature increase. In fact, McIntyre and McKitrick are careful to point out that it is hard to draw conclusions from these data, even with their corrections. Did medieval global warming take place? Last month the consensus was that it did not; now the correct answer is that nobody really knows. Uncovering errors in the Mann analysis doesn’t settle the debate; it just reopens it. We now know less about the history of climate, and its natural fluctuations over century-scale time frames, than we thought we knew.
If you are concerned about global warming (as I am) and think that human-created carbon dioxide may contribute (as I do), then you still should agree that we are much better off having broken the hockey stick. Misinformation can do real harm, because it distorts predictions. Suppose, for example, that future measurements in the years 2005-2015 show a clear and distinct global cooling trend. (It could happen.) If we mistakenly took the hockey stick seriously--that is, if we believed that natural fluctuations in climate are small--then we might conclude (mistakenly) that the cooling could not be just a random fluctuation on top of a long-term warming trend, since according to the hockey stick, such fluctuations are negligible. And that might lead in turn to the mistaken conclusion that global warming predictions are a lot of hooey. If, on the other hand, we reject the hockey stick, and recognize that natural fluctuations can be large, then we will not be misled by a few years of random cooling.
A phony hockey stick is more dangerous than a broken one--if we know it is broken. It is our responsibility as scientists to look at the data in an unbiased way, and draw whatever conclusions follow. When we discover a mistake, we admit it, learn from it, and perhaps discover once again the value of caution.
Richard A. Muller, a 1982 MacArthur Fellow, is a physics professor at the University of California, Berkeley, where he teaches a course called “Physics for Future Presidents.” Since 1972, he has been a Jason consultant on U.S. national security.
|
|
|
|
|
David Lucifer
Archon
Posts: 2642
Reputation: 8.93
Rate David Lucifer
Enlighten me.
|
|
Re:The Broken Hockey Stick of Climate Science
« Reply #1 on: 2004-11-06 12:47:01 » |
|
Thank you for posting this. It is well worth reading McIntyre and McKitrick's project update. I knew that the actual practice of science is messy and political, but this story reminds me of the quote attributed to Bismarck about law and sausages. It is difficult to not assume conspiracy here (instead of mere incompetence). I can only hope that M&M get this published in a well read journal elsewhere and give Nature a black eye in the process.
Related links:
Taken By Storm The Troubled Science, Policy and Politics of Global Warming co-authored by McKitrick
Kyoto climate treaty passes final hurdle
|
|
|
|
|
the.bricoleur
Archon
Posts: 341
Reputation: 8.46
Rate the.bricoleur
making sense of change
|
|
Re:The Broken Hockey Stick of Climate Science
« Reply #2 on: 2004-11-09 11:10:19 » |
|
My pleasure David.
Thank you for the link to 'Taken By Storm' - one I will be adding to the library.
Iolo.
|
|
|
|
|
the.bricoleur
Archon
Posts: 341
Reputation: 8.46
Rate the.bricoleur
making sense of change
|
|
Re:The Broken Hockey Stick of Climate Science
« Reply #3 on: 2005-06-27 06:59:49 » |
|
The US House Energy Committee has launched a federal investigation of the hockey stick fiasco. Chairman Barton may have in mind new legislation to compel sharing of policy sensitive scientific data, along lines suggested by Steve McIntyre. About time.
Energy Commerce
It’s a fascinating situation and reminds me a little of Lomborg and that Danish Scientific Honesty Committee he found himself being judged by. A higher authority eventually exonerated him on the grounds that a polemical book was not a piece of research and so fell outside the committee’s remit.
It is of course splendid news that even one element of the research base lurking behind climate policy is being scrutinised this way, I hope that others soon follow. But in a funny way I sympathise with Mann too; a young researcher from a soft-science background doing a meta-analysis armed with data analysis techniques that he probably did not have much of a grasp of; it was just blue-skies stuff, no special consequences, but suddenly he was thrust into the limelight. Phoney research is happening all the time, but science has a way of dealing with its dirty linen when subsequent data (the horrid word consensus springs to mind) puts past work into context. And it happened in this case too – it was McKitrick et al in this instance that started the ball rolling, but in due course someone else would have come along and found these and other flaws.
IMV that the greater fault lies elsewhere. Co-author Ray Bradley was a wiser and older head; Phil Jones has a much stronger statistical grasp. But even these are dwarfed by the general research environment that climatology now finds itself in. Basically climatologists enjoy being in the limelight – after all are they not in the front line of saving the world? So the entire palaeoclimatology tribe simply rolled over and without a murmur allowed the ups and downs of millennial climate history to be air-brushed out by a smoothing function that picked up 13% of the variance. If informed people/groups had spoken out, then things would not have gone so far. Then there’s the political class and their green lobbyists, equally keen for a slice of the saving-the-planet action. It doesn’t matter very much if blue-skies research is wrong or overstated for reasons given above. But when the stakes are as high as they are here then it should be obvious that the highest standards of quality assurance be followed such as a drugs company would expect. It is a mistake to base policy on cutting edge stuff that is almost bound to be overturned or adjusted in due course but based on a settled view of established techniques. In principle the IPCC approach is a good idea but should never have been allowed to fall into the hands of those who stand to gain most from its conclusions.
This brings me to the area where I think the greatest blame of all lies – the Precautionary Principle. It is the PP which legitimises action ahead of information on the topsy-turvy grounds that the more serious the consequences, the more acceptable it is to base it on worst-case. It is an open invitation to declare ever-greater levels of alarm and this pernicious principle will keep doing more and more damage until it is consigned to the landfill.
-Iolo
|
|
|
|
|
rhinoceros
Archon
Gender: 
Posts: 1318
Reputation: 8.34
Rate rhinoceros
My point is ...
|
|
Re:The Broken Hockey Stick of Climate Science
« Reply #4 on: 2005-06-27 09:50:46 » |
|
[Iolo] The US House Energy Committee has launched a federal investigation of the hockey stick fiasco. Chairman Barton may have in mind new legislation to compel sharing of policy sensitive scientific data, along lines suggested by Steve McIntyre. About time.
[rhinoceros]
a) The hockey stick is a "fiasco" only according to the allegations of a ring of persons.
b) As the technical perplexities of the whole argument have shown, there is no information for a politician (or myself) in climate data series.
c) Politicians have full responsibility for their policy decisions as far as the voters are concerned. They can base them on whatever they consider as good information. That does not entitle them to intervene in scienctific work.
[Iolo] This brings me to the area where I think the greatest blame of all lies – the Precautionary Principle. It is the PP which legitimises action ahead of information on the topsy-turvy grounds that the more serious the consequences, the more acceptable it is to base it on worst-case. It is an open invitation to declare ever-greater levels of alarm and this pernicious principle will keep doing more and more damage until it is consigned to the landfill.
[rhinoceros] There are good arguments against some forms of the Precautionary Principle. However, it is perfectly rational from a risk management perspective to evaluate serious consequences vs low probability. That's what insurance companies do.
|
|
|
|
|
rhinoceros
Archon
Gender:
Posts: 1318
Reputation: 8.34
Rate rhinoceros
My point is ...
|
|
Re:The Broken Hockey Stick of Climate Science
« Reply #5 on: 2005-06-27 10:21:20 » |
|
[David Lucifer]
Related links:
Taken By Storm The Troubled Science, Policy and Politics of Global Warming co-authored by McKitrick
[rhinoceros]
I wish I had the time to read this book. It seems to argue that esentially a climate science does not exist. From other readings of mine, I'd wager that they manage to disappoint themselves by examining climate science with the expectations of a Galilean framework, which does not fit well. I'd say that the framework of climate science is a custom-made one, driven by the need to answer questions such as long-term predictions for agriculture. Unlike the oil industry, the agriculture industry needs the truth.
Here is an interesting read.
Towards a Definition of Climate Science
http://web.mit.edu/globalchange/www/MITJPSPGC_Reprint02-9.pdf
<quote>
The intrinsic difficulties in building realistic climate models and in providing complete, reliable and meaningful observational datasets, and the conceptual impossibility of testing theories against data imply that the usual Galilean scientific validation criteria do not apply to climate science. The different epistemology pertaining to climate science implies that its answers cannot be singular and deterministic; they must be plural and stated in probabilistic terms. Therefore, in order to extract meaningful estimates of future climate change from a model, it is necessary to explore the model's uncertainties. In terms of societal impacts of scientific knowledge, it is necessary to accept that any political choice in a matter involving complex systems is made under unavoidable conditions of uncertainty. Nevertheless, detailed probabilistic results in science can provide a baseline for a sensible process of decision making
<end quote>
[rhinoceros] By the way, I noticed this statement in the first blurb about the "Taken By Storm" book:
<quote>
Is global warming the greatest threat facing humanity?
One hundred Nobel Laureates recently signed a statement saying so; a UN panel of scientists has also testified to this; and governments around the world have put the climate change issue at the forefront of their political agendas. Can all of these bodies be wrong?
<end quote>
[rhinoceros] The first sentence has probably been used out of context and with changed focus. No reference is given, but I guess it is this one:
http://www.cpa.org.au/garchve5/1093nuclear.html
Nuclear treaty sham
On December 7, 2001, one hundred Nobel Laureates issued a statement to
mark the 100th anniversary of the Nobel Prize. They warned of the dangers
to world security posed by global warming, social problems and a
militarised world.
They said, "the most profound danger to world peace in the coming years
will stem not from the irrational acts of states or individuals but from
the legitimate demands of the world's dispossessed. Of these poor and
disenfranchised, the majority live a marginal existence in equatorial
climates. Global warming, not of their making but originating with the
wealthy few, will affect their fragile ecologies most. Their situation will
be desperate and manifestly unjust."
"It cannot be expected", said their statement, "that in all cases they will
be content to await the beneficence of the rich."
The Nobel Laureates regarded global warming and a militarised world as the
two things that may lead to the world's destruction.
They expressed hope that social justice and peace could be achieved. They
said that: "some of the needed legal instruments such as the Anti-Ballistic
Missile Treaty (ABM), the Convention on Climate Change and the Strategic
Arms Reduction Treaties and the Comprehensive Test Ban Treaty, were already in place.
<snip>
|
|
|
|
|
rhinoceros
Archon
Gender:
Posts: 1318
Reputation: 8.34
Rate rhinoceros
My point is ...
|
|
Conspiracy?
« Reply #6 on: 2005-06-27 12:26:00 » |
|
[David Lucifer]
Thank you for posting this. It is well worth reading McIntyre and McKitrick's project update.
[rhinoceros]
I found a related post in the RealClimate site (Mann et al) which mentions something about this rejection of McIntyre & McKitrick submision to Nature magazine last year, in the section Myth #4.
Myth vs. Fact Regarding the "Hockey Stick"
http://www.realclimate.org/index.php?p=11
[rhinoceros] It makes some interesting arguments, but it does not contain verbatim quotes from the rejection slip like Mc&Mc do, so I am no more enlightened as to what Mc&Mc chose to leave out themselves.
[Lucifer]
I knew that the actual practice of science is messy and political, but this story reminds me of the quote attributed to Bismarck about law and sausages. It is difficult to not assume conspiracy here (instead of mere incompetence).
[rhinoceros]
We could try to work more on a good conspiracy theory. Where would we start? Can we find motives? No, not world domination, something more realistic, but what?
Also, we should probably restrict the conspiracy only to a big part of the scientific community, especially those involved in peer review. That is because if we extend it to politicians, Blair, the G8 industrialists with the exception of Bush, we won't get anywhere. Those must be the victims of our conspiracy theory.
If we fail to make it stand, we can still get back to mundane expanations, say, a dozen of named professionals very capable with the Web and with words that resonate well with the general educated public.
[Lucifer]
I can only hope that M&M get this published in a well read journal elsewhere and give Nature a black eye in the process.
[rhinoceros]
Different peer reviewed journals do publish papers which contradict each other, on occasion. It would not be a first for Nature or any other journal.
Moreover, it's been some time, and Mc&Mc's position is hardly a secret anymore. People, including scientists and journal reviewers, have had more than a chance to read about it in the last years. In the meanwhile, thousands of papers on various topics are being published in peer revied journals and nobody notices if their authors say "gotcha!". It's the Web that makes the difference.
|
|
|
|
|
|
Logged
