Monday, July 12, 2010

Tiljander Proxies' Modest Effects on Mann08 Estimates... probably

Few things in life can be as rewarding as shouting "I was wrong!"

[Edit 7/13/10: That is, "probably wrong in thinking that the Tiljander proxies make an oversized contribution to Mann08's paleoclimate reconstructions (and still correct about other Tiljander-related issues)." Elaborated upon, below.]

One of the things that's annoyed Dr. Schmidt (see his Collide-a-scape 'guest post' The Main Hindrance to Dialogue (and Detente)) and other AGW Consensus advocates has been my contention that the Tiljander proxies likely provide a substantial contribution to the "hockey stick" shape of the paleotemperature reconstructions in the Mann group's 2008 PNAS paper, "Proxy-based reconstructions of hemispheric and global surface temperature variations over the past two millennia" (see this post for literature references). That stance has provided grist for the mill for protracted discussions at Lucia's Blackboard and other blogs. But...

Last week, I started re-reading some of skeptical blogger Jeff Id's early posts, when he was first trying to sort out Mann08's methods and results. A figure from the 9/28/08 post Mann 08 Series Weight Per Year just about made me fall out of my chair.  Here it is -


There are some caveats surrounding the interpretation of this graph. Mainly, this is data from an emulation of Mann08's land-plus-ocean reconstruction by EIV that was accomplished by back-calculation, and doesn't produce a nearly-exact copy. See Jeff's post for details.

But one point shines through: The Tiljander proxies are represented in gray, at the bottom of the graph (the legend reads "Sediment Finland").

Throughout the time series, they contribute less than 5% to the reconstruction. Other proxies contribute the other 95%-plus portion of the signal. So if Jeff is right -- he probably is --

I was wrong.

[Edit 7/13/10: That is to say, Jeff Id's estimate of the contributions of various proxies suggests only a modest contribution to this Mann08 paleotemperature reconstruction by the Tiljander proxies. Other proxies are much more important. The other points that have been raised about the TIljander proxies remain. These include their uncalibratability to the instrumental record, and thus their faux calibration by Mann08's authors. They also include the use of XRD and lightsum in an orientation that is inverted, with respect to the orientation proposed in Tiljander03. Do these other points still matter? Obviously, it depends on your definition of that word. I think the answer is clearly "yes," for a number of issues. One of them is outlined below.]

I put versions of the following comment at Keith Kloor's Collide-a-scape, Arthur Smith's Not Spaghetti, and Lucia's Blackboard.

Here's some strong evidence in favor of the second part of the Gavin/toto answer to the Calibratability Question ("I don't know, and it doesn't matter.") To be clear, this argues against my assertion that "it does matter."

Back in September 2008, blogger Jeff Id did some analysis of the proxies used in Mann08, and emulated some of Mann08's reconstructions, with much success. His post is Mann 08 Series Weight Per Year

Jeff estimates the contributions of various types of proxies at various points in time. His figure, "Percent Contribution to M08 Temp Reconstruction by Year" is worth examining in detail. It shows that in the earliest parts of the reconstruction he studied -- say, up through ~1100 or ~1300 -- the results are dominated by two types of proxies: cave precipitation records (yellow), and the Punta Laguna proxies (greenish-blue).

The contribution of the Tiljander proxies (gray) is quite modest (well under 5%), throughout.
Does this mean that the protracted controversy over the Tiljander proxies that started within days of Mann08's online publication has been a tempest in a teapot?

I don't think so. There are a host of issues that remain. For instance, one point is raised in this comment that I submitted to Arthur Smith's post "Michael Mann's errors" -
The "Tiljander" argument matters, even if Jeff Id's graph "Percent Contribution to M08 Temp Reconstruction by Year" and/or twice-revised Fig. S8a correctly represent the modest contribution of these proxies to the reconstruction.

Comments in a recent post by IPCC AR5 WG2 author Ed Carr provide a timely illustration. The 9 July 2010 post at "Open the Echo Chamber" is "Apparently, we have learned nothing..." In the ensuing 50+ comment thread, some skeptics chime in with, well, skepticism about the IPCC process and the AR4 report. Here, knowledgeable and articulate commenter 'caerbannog' reviews some reasons why criticisms of Prof. Mann's work by Steve McIntyre are ill-founded. He says,

...you can generate hockey-stick-shaped leading principal components via [a non-centered PCA] method. But there’s an easy way to distinguish a “noise” hockey-stick scenario from one where the hockey-stick results from a real temperature signal.

You look at the eigenvalue magnitudes...

Any competent analyst trying to extract a temperature signal from proxy data via the PCA method would look at the eigenvalue magnitudes before deciding whether to proceed with the regression steps

Had Mann’s eigenvalues looked like McIntyre’s noise eigenvalues, Mann would most likely said to himself, “There’s not much of a common temperature signal in my tree-ring data here; don’t think that I will be able to do much with it.”

If Mann’s tree-ring eigenvalues had looked anything like McIntyre’s noise eigenvalues, Mann certainly would have realized that his tree-ring data did not contain any temperature information worthy of publication.

This whole “spurious hockey-stick” argument used against Mann is completely without merit.
Later in that conversation with 'Nullis in Verba', 'caerbannog' adds,
Having a solid background in formal math/statistics is important, but it is also very important to be able to “relate” the numbers you’ve crunched with physical, real-world processes. That’s where many stats/mathematics types come up short — they haven’t worked with “real world” data enough to get practical experience with “real world” scenarios.

And there’s no substitute for working with lots of “real world” data to get an appreciation for this.
At the (current) tail of the comments [at Ed Carr's blog], I link to this post of Arthur's and remark,
...The use of proxies in the Mann group’s September 2008 article in PNAS (Mann08) sheds much light on [the use by Prof. Mann of screening, verification, and calibration steps in the evaluation and use of proxy datasets for paleotemperature reconstructions.] In particular, the Lake Korttajarvi varved lakebed sediments characterized by Tiljander et al (2003) are an important test case of this claim:

“Mann08 demonstrates methods of proxy selection and calibration for paleotemperature reconstruction that are robust.”

In my opinion, analysis of Mann08′s use of the Tiljander proxies shows that this recent, high-profile paper clearly fails the “robustness” claim.
The dialog between 'caerbannog' and 'Nullis in Verba' demonstrates that many people who are conversant with much of the technical detail of the controversies about the "Hockey Stick" are unaware of the issues associated with the selection and use of the Tiljander proxies. Being uninformed, they have not considered Tiljander's implications for the larger arguments they advance.
I've got some ideas to follow up on, but they'll require that I become proficient in R. For me, that'll be a project with a long time horizon...

7 comments:

  1. Not sure what you are trying to say here Amac? You start by asserting something that I do understand, that you have spent some time debating on Tiljander proxies and their use upside down or not as input data to published research.
    You then go on to say that because some assumptions used to produce an admittedly imperfect replication quantifying the input and of the output of this work, your debating points are wrong.
    I'm not sure I see or understand how the one follows from the other?

    I'd also note in passing that your recent opposite number on these subjects is not someone I would characterise as ever acting in good faith, and hence not worth wasting your time, but hey, I'm also often wrong.

    ReplyDelete
  2. Thanks for your remarks, Chuckles. I have added two Edits to clarify the meaning of "I was wrong."

    I could add that the key questions about the changes that the Tiljander proxies cause to the spaghetti traces have to do with (1) the magnitude of the differences in temperature anomaly that is caused by their inclusion (or exclusion), and (2) the effect of their inclusion/exclusion on the uncertainty estimates.

    "I was wrong" means I now think that the magnitude of the change to the traces is likely modest -- consistent with what is shown in twice-revised Fig. S8a.

    I have no idea how removal of the faux-calibrated Tiljander proxies would affect the size of the calculated error bars. The uncertainty estimates are in some ways as important as the anomaly traces themselves. Mann08's S8a doesn't allow this comparison, and Jeff Id's graph doesn't shed any light here.

    Much will have to wait until I can learn R...

    ReplyDelete
  3. While it's good to see honor in skeptics, I don't think that just because Tiljander didn't reverse the conclusions means you were so solidly wrong. consider that they had to use the proxies upside down in a purely nonsensical fashion. When confronted, Mann gave the sophists answer that multivariate methods are insensitive rubbish - which avoids the point that the data is supposed to be temperature the other way!!

    Clearly then, the data isn't temperature.

    Why are they using it at all?

    My weights were just estimates, since running this analysis Ryan Nic and SteveM have replicated RegEM for the Steig et al rebuttal and I now understand that different weights were applied to each year which contained a different set of data. Therefore, this method which used a single weight per series, keeping the same import for each proxy through time, cannot match Mann08.

    Perhaps you still aren't wrong! I still suspect a small weight from these series as from memory they have only weak - near zero- values. A correlation analysis of the available proxies in the 100-800 AD timeframe should give some hints.

    ReplyDelete
  4. which contained a different set of data.

    which contained a different subset of available proxy data.

    ReplyDelete
  5. Thanks for confirming my point Jeff. I still think Amacs retractions are premature. This does not mean I think he is 'right' or whatever, merely that any conclusions in any direction from the estimates, are dubious.

    ReplyDelete
  6. Well you know now that he's an honest broker of what he finds.

    ReplyDelete
  7. I liked your sharing this, Amac. Thumbs up, man.

    Watch out for Jeff. He can be slow to understand his own errors...can be bullheaded.

    ReplyDelete