Anyone who follows this blog (you know who you are) will have noticed the concentration of late on non-science topics. So I thought it was worth returning to my former stomping ground. This was, in part, because I came across something specific in the press that caught my attention. But it also relates to a much bigger, and therefore more troubling, theme. Science matters because it is clear that it is the best, perhaps the only way, to effectively answer certain types of important questions. It has an impressive (though not unblemished) track record. Some of the problems we face today pose questions of exactly the type science in the past has helped to answer. So if science is in trouble, we’re all in trouble. It is therefore wise to reflect on the position “it” finds itself in.
Let’s start with some specifics. Patrick Brown is a climate scientist. He obtained his PhD (Title: “Magnitude and Mechanisms of Unforced Variability in Global Surface Temperature”) from Duke University in 2016 and has since been fairly productive. As far as I can see, has had three papers published in Nature as “first author” to date; not bad for someone relatively early in his career. It is the third and most recent of these (“Climate warming increases extreme daily wildfire growth risk in California”; Nature 621:760-766, published 30/06/2023) that has excited most comment. However, the comment has not primarily been around the science in his paper. Judging from his citation statistics (a far from perfect metric), Patrick is competent but he hasn’t exactly set the heather alight. It was what he did after his latest Nature paper was published that led to things getting tasty. On September 5th he published an article in “The Free Press” entitled “I left out the full truth to get my climate change paper published”; this article was later also published in the The Times under the title “Groupthink in science is no good for the planet” (The Times, September 9th, p28) generating much more attention (at least on this side of the Atlantic).
Basically he claimed in his articles that he (and his co-authors) had narrowed the focus of their approach in the Nature paper to that of the effect of climate change on wildfires, all the time knowing that much more complex issues were in play. But they knew that if they “overcomplicated” the picture, so that it did not so clearly support the story that important journals like Nature “want to tell”, their paper would have likely been bounced. If they had broadened the focus (in the process presenting a more accurate and useful picture) they would have been seen not to support “certain preapproved narratives” that some journals, including Nature, are pushing. He fairly makes the point that getting published in prestigious journals has a big influence on someone’s academic career, and that these days it is hard to stick out from the crowd of other PhD’s. So this non-scientific factor, as much as the importance of the science they had done, determined important things like what metrics they had used to assess what was going on with Californian wildfires, and how the data that had resulted from their analysis was interpreted and presented. He was just innocently playing the game of building an academic career. But, having moved out of academia, he now felt moved to act the whistleblower and tell all. Not that he is in favour of retracting the paper as he still thinks “it advances our understanding of climate change’s role in day-to-day wildfire behavior”. It’s just that “the process of customizing the research for an eminent journal caused it to be less useful than it could have been”. The fact that there is a competing narrative in this space (i.e. that man-made climate change is a hoax), and that his “exposé” was jumped on as evidence of scientific skulduggery, didn't seem to bother him (at least initially).
I don’t have the expertise to comment on his Nature article. But of course, before it was published, those with appropriate expertise did. Nature published the peer review reports along with the final paper, and interestingly while the paper itself is behind Nature’s paywall, the reports aren’t (you can access them using the Nature link above). What these make clear is that some of the reviewers made the point that some of the wider issues should have been covered in the paper and hadn’t been. Given the tale that Brown subsequently told, this is a bit surprising. But what is even more surprising is that Brown and his fellow authors then robustly defended their approach. This shows that there was no particular “preapproved narrative”, or at least not one of the kind alleged. The reviewers (and the Editor) dealt with the paper on its merits as we all might expect. So his charge that some agenda that is not supported by the science is being prosecuted, looks a lot weaker than at first it appears.
But what Dr Brown seems to miss entirely is that he has told us that on at least one particular occasion he deliberately shaped his approach so that the resulting paper became potentially misleading or at least less useful (something the Nature reviewers in effect picked up on and challenged). In fact in his Free Press article he claims he left academia because “the pressures put on academic scientists caused too much of the research to be distorted”. Presumably he means his own research as well as that of others – a serious charge. Yet, despite confessing to distortion, we are supposed to take his commentary (unchallenged by reviewers and perhaps serving an agenda) at face value. And it’s not as though he is some kind of innocent when it comes to the media. He knows well how the media works specifically when it comes to his area of expertise (climate change). He published a paper about precisely this back in 2016 (“Reporting on global warming: A study in headlines”). Now he is in the private sector, free from that insidious pressure to “distort” (his word, not mine). But presumably he is also now being paid for his words by individual and corporate donors. We can hardly be sure that it is data and careful analysis that are the centre of his considerations. After all, he has form. It all begins to look a bit murky.
And that’s a big problem. All over, science is beginning to look murky. Much of Brown’s commentary is recognisable. There is pressure to publish, and particularly to publish in “top” journals like Nature. I’ve submitted to Nature myself (more in hope than expectation). And decisions do have to be made about both data selection and analysis, even in much simpler situations that those being investigated by Brown and his co-authors. Can this lead to bias and misrepresentation? Yes it can. But that is where the challenge of reviewers and editors, the peer review system, becomes so important. The system seems to have worked in the case of Brown’s Nature paper. Although the reviewers expressed concerns, these were answered by the authors, and the paper was deemed to make a sufficient contribution (something Brown continues to agree with) to be published. Does it present only part of the picture? Of course it does. It’s now up to others to criticise, challenge, refine or refute what’s in that one paper. If it is actually misleading, that will become clear. That’s science.
But the bigger theme here is a problem about journals; they are a key part of science and collectively comprise the “literature”. Brown’s point was that they may not be as neutral and dispassionate as one would like to think (whether justified or not in the case of his Nature paper). There are other problems too, particularly the issue of “predatory” journals which has been discussed for a while in scientific publishing circles (see this article and others on the the Scholarly Kitchen site). Predatory journals are those whose primary concern is to make money not publish good science. They tend to have lax acceptance and reviewing standards because the more they publish the more money they make. This has been encouraged by a change in who pays for published science. It used to be almost entirely the case that the user (i.e. the reader) paid. But this began to change, partly because of technology and partly because of claims that his was discriminatory. Lots of scientists in low and middle income countries were excluded because neither they nor their institutional libraries could afford the subscriptions that were charged for access to journals. So there was a change to a “producer pays” model. Some journals charge a fee simply to consider a manuscript for publication, and all of them charge a fee to publish papers once the peer review process has determined that a paper is of sufficient merit. Publication fees range from a few hundred £s/$s, to several thousand. Some charge flat fees, others charge by the published page. However, once published the research is open to all, and aided by the interweb, accessible to all. But it is clear that what was meant to assist openness and accessibility is being abused, and that the “literature” is being undermined as a result.
It was always the case that nonsense could be published in scientific journals, including the prestigious ones. I used to have to tell students that just because something was published didn’t make it true. There is never any substitute for careful reading and equally careful thinking. But as the number of predatory journals has increased (one 2021 estimate put the number at 15,059), so has the level of murkiness, and gradually we risk the whole scientific enterprise losing the trust of public and politicians alike. What is the root cause of these problems? Well, unfortunately it is something that cannot be fixed (although it can be improved). Science is a human activity, and is therefore as flawed as humans are. Most scientists are competent and conscientious, some are lazy, a very small number are fraudulent, but all are human. Even although as an institution science is to some extent self-correcting, it remains at its core the activity of flawed women and men. Science’s big problem is scientists. And just when we need them too.
No comments:
Post a Comment