Science’s big problem(s)

Anyone who follows this blog (you know who you are) will have noticed the concentration of late on non-science topics. So I thought it was worth returning to my former stomping ground. This was, in part, because I came across something specific in the press that caught my attention. But it also relates to a much bigger, and therefore more troubling, theme. Science matters because it is clear that it is the best, perhaps the only way, to effectively answer certain types of important questions. It has an impressive (though not unblemished) track record. Some of the problems we face today pose questions of exactly the type science in the past has helped to answer. So if science is in trouble, we’re all in trouble. It is therefore wise to reflect on the position “it” finds itself in.

Let’s start with some specifics. Patrick Brown is a climate scientist. He obtained his PhD (Title: “Magnitude and Mechanisms of Unforced Variability in Global Surface Temperature”) from Duke University in 2016 and has since been fairly productive. As far as I can see, has had three papers published in Nature as “first author” to date; not bad for someone relatively early in his career. It is the third and most recent of these (“Climate warming increases extreme daily wildfire growth risk in California”; Nature 621:760-766, published 30/06/2023) that has excited most comment. However, the comment has not primarily been around the science in his paper. Judging from his citation statistics (a far from perfect metric), Patrick is competent but he hasn’t exactly set the heather alight. It was what he did after his latest Nature paper was published that led to things getting tasty. On September 5^th he published an article in “The Free Press” entitled “I left out the full truth to get my climate change paper published”; this article was later also published in the The Times under the title “Groupthink in science is no good for the planet” (The Times, September 9^th, p28) generating much more attention (at least on this side of the Atlantic).

Basically he claimed in his articles that he (and his co-authors) had narrowed the focus of their approach in the Nature paper to that of the effect of climate change on wildfires, all the time knowing that much more complex issues were in play. But they knew that if they “overcomplicated” the picture, so that it did not so clearly support the story that important journals like Nature “want to tell”, their paper would have likely been bounced. If they had broadened the focus (in the process presenting a more accurate and useful picture) they would have been seen not to support “certain preapproved narratives” that some journals, including Nature, are pushing. He fairly makes the point that getting published in prestigious journals has a big influence on someone’s academic career, and that these days it is hard to stick out from the crowd of other PhD’s. So this non-scientific factor, as much as the importance of the science they had done, determined important things like what metrics they had used to assess what was going on with Californian wildfires, and how the data that had resulted from their analysis was interpreted and presented. He was just innocently playing the game of building an academic career. But, having moved out of academia, he now felt moved to act the whistleblower and tell all. Not that he is in favour of retracting the paper as he still thinks “it advances our understanding of climate change’s role in day-to-day wildfire behavior”. It’s just that “the process of customizing the research for an eminent journal caused it to be less useful than it could have been”. The fact that there is a competing narrative in this space (i.e. that man-made climate change is a hoax), and that his “exposé” was jumped on as evidence of scientific skulduggery, didn't seem to bother him (at least initially).

I don’t have the expertise to comment on his Nature article. But of course, before it was published, those with appropriate expertise did. Nature published the peer review reports along with the final paper, and interestingly while the paper itself is behind Nature’s paywall, the reports aren’t (you can access them using the Nature link above). What these make clear is that some of the reviewers made the point that some of the wider issues should have been covered in the paper and hadn’t been. Given the tale that Brown subsequently told, this is a bit surprising. But what is even more surprising is that Brown and his fellow authors then robustly defended their approach. This shows that there was no particular “preapproved narrative”, or at least not one of the kind alleged. The reviewers (and the Editor) dealt with the paper on its merits as we all might expect. So his charge that some agenda that is not supported by the science is being prosecuted, looks a lot weaker than at first it appears.

But what Dr Brown seems to miss entirely is that he has told us that on at least one particular occasion he deliberately shaped his approach so that the resulting paper became potentially misleading or at least less useful (something the Nature reviewers in effect picked up on and challenged). In fact in his Free Press article he claims he left academia because “the pressures put on academic scientists caused too much of the research to be distorted”. Presumably he means his own research as well as that of others – a serious charge. Yet, despite confessing to distortion, we are supposed to take his commentary (unchallenged by reviewers and perhaps serving an agenda) at face value. And it’s not as though he is some kind of innocent when it comes to the media. He knows well how the media works specifically when it comes to his area of expertise (climate change). He published a paper about precisely this back in 2016 (“Reporting on global warming: A study in headlines”). Now he is in the private sector, free from that insidious pressure to “distort” (his word, not mine). But presumably he is also now being paid for his words by individual and corporate donors. We can hardly be sure that it is data and careful analysis that are the centre of his considerations. After all, he has form. It all begins to look a bit murky.

And that’s a big problem. All over, science is beginning to look murky. Much of Brown’s commentary is recognisable. There is pressure to publish, and particularly to publish in “top” journals like Nature. I’ve submitted to Nature myself (more in hope than expectation). And decisions do have to be made about both data selection and analysis, even in much simpler situations that those being investigated by Brown and his co-authors. Can this lead to bias and misrepresentation? Yes it can. But that is where the challenge of reviewers and editors, the peer review system, becomes so important. The system seems to have worked in the case of Brown’s Nature paper. Although the reviewers expressed concerns, these were answered by the authors, and the paper was deemed to make a sufficient contribution (something Brown continues to agree with) to be published. Does it present only part of the picture? Of course it does. It’s now up to others to criticise, challenge, refine or refute what’s in that one paper. If it is actually misleading, that will become clear. That’s science.

But the bigger theme here is a problem about journals; they are a key part of science and collectively comprise the “literature”. Brown’s point was that they may not be as neutral and dispassionate as one would like to think (whether justified or not in the case of his Nature paper). There are other problems too, particularly the issue of “predatory” journals which has been discussed for a while in scientific publishing circles (see this article and others on the the Scholarly Kitchen site). Predatory journals are those whose primary concern is to make money not publish good science. They tend to have lax acceptance and reviewing standards because the more they publish the more money they make. This has been encouraged by a change in who pays for published science. It used to be almost entirely the case that the user (i.e. the reader) paid. But this began to change, partly because of technology and partly because of claims that his was discriminatory. Lots of scientists in low and middle income countries were excluded because neither they nor their institutional libraries could afford the subscriptions that were charged for access to journals. So there was a change to a “producer pays” model. Some journals charge a fee simply to consider a manuscript for publication, and all of them charge a fee to publish papers once the peer review process has determined that a paper is of sufficient merit. Publication fees range from a few hundred £s/$s, to several thousand. Some charge flat fees, others charge by the published page. However, once published the research is open to all, and aided by the interweb, accessible to all. But it is clear that what was meant to assist openness and accessibility is being abused, and that the “literature” is being undermined as a result.

It was always the case that nonsense could be published in scientific journals, including the prestigious ones. I used to have to tell students that just because something was published didn’t make it true. There is never any substitute for careful reading and equally careful thinking. But as the number of predatory journals has increased (one 2021 estimate put the number at 15,059), so has the level of murkiness, and gradually we risk the whole scientific enterprise losing the trust of public and politicians alike. What is the root cause of these problems? Well, unfortunately it is something that cannot be fixed (although it can be improved). Science is a human activity, and is therefore as flawed as humans are. Most scientists are competent and conscientious, some are lazy, a very small number are fraudulent, but all are human. Even although as an institution science is to some extent self-correcting, it remains at its core the activity of flawed women and men. Science’s big problem is scientists. And just when we need them too.

Why does science matter?

Although it’s really my last post that prompted this one, I am admittedly returning to something I’ve blogged about before. It was a while ago, so I won’t take it personally if you can’t remember what those particular posts were about. I’ll try not to repeat any of the specifics here as you can obviously go back and read them (eg here and here). But having opined about why theology matters (about which I know relatively little), it seemed only fair to reflect on what I spent most of my adult life working in.

However, there are a couple of issues we have to deal with first. Although it’s common to talk about “science” as though it is a single institution, it really isn’t. There is no single body that polices a rule book, and the reality is that there is no single agreed definition or set of rules. There is also no single agreed scientific method. It used to be thought that a single recipe for doing good science might be either discoverable or definable, and that a single, coherent method could be established. And of course the philosophers got busy trying to cook one up. But with due respect to the likes of Francis Bacon, John Locke, William Whewell, Karl Popper and Thomas Kuhn, none of them really produced anything that you could pull off a shelf, apply to a problem and obtain a “scientific answer”. Indeed the most many of them managed was an attempt at describing what scientists actually did. This is an interesting exercise in its own right. Mind you, it has always seemed to me that they were overly infatuated with physics, from which they drew many of their key examples. If of course science is just one thing, and there is a single method, then why not start with an area of science that seems to have delivered. Perhaps this explains why “big physics” is often reported in the media and is supported by such massive sums of public money (over the last decade the UK has invested an average of £152M per year in CERN alone). Biology has usually suffered in comparison. The philosophers didn’t seem to like biology that much, it was maybe too wet and messy.

It’s odd, but all this philosophical effort, individually and cumulatively, has had relatively little impact on the activities of scientists themselves. By and large they just got on doing “it”, and apparently quite successfully. It looked like there might be a common core of things that were a good idea, things like collecting evidence, forming tentative explanations, and then testing these rather than just blithely accepting and asserting them. But single, codified, rigorous method? Not really. Occasionally, individual scientists were influenced by reading about what they were supposed to be doing in the writings of one or more of the aforementioned philosophers or thinkers (many of whom were not themselves scientists). They might try to construe their activities in the sort of terms they had read about. But this all tended to be rather post-hoc. Suspiciously, such accounts tended to crop up in books written at the end of careers, as though they were a relatively recent discovery.

Now this all may be a good or a bad thing. But part of the problem is that relatively few pure science degrees (particularly in the Anglo-Saxon world) provide a rigorous introduction to the intellectual procedures involved in science. There are lots of lectures, lots of learning about great previous experiments, occasional attempt to repeat them and so on. Such degrees are certainly fact-packed (and very often great fun too – mine was!). But as to the principles of how your thinking was supposed to operate, one was rather expected to simply imbibe or intuit this. To be fair, this is a criticism that has so often made, that in many degree programmes today there may be an optional module in the philosophy of science. But it is rarely a key component of the education of a young scientist. And this has the disturbing consequence of a highly skilled but philosophically unsophisticated workforce.

None of this means that science (in its various forms) has been generally unsuccessful; clearly it hasn’t. But one unwelcome effect has been the unfortunate inability of many of us scientists (and I include myself in this) to helpfully articulate why science has been successful, what its product has enabled, and why this all matters. What we often end up with is hubristic, triumphalist babble that can sometimes seem  more like paternalistic propaganda. Scientists do all have skin in the game of course, because many of us earn our money from the scientific enterprise. And the source of that money is very often hard-pushed taxpayers, and in the case of the health and clinical sciences, patients. When we try to explain what we’re up to and why it matters, we can sometimes sound rather as though we’re saying that you should simply trust us (and keep paying us) because we know what’s best, and it would be far too complicated to explain to you.

Now there is a sense in which this is true. These days the technical details are often complicated, and a degree of trust is required. But the problem is that because we have not articulated well enough or often enough how science works (in its various forms), trust is now rather lacking. This is illustrated by the range of responses to the undoubted success of the vaccines developed to combat the COVID19 pandemic. The mRNA vaccines that have been so successful are the product of a completely new approach to vaccine development that emerged from years of patient and largely unheralded basic science, working out the details of what goes on in cells at a molecular level. The speed at which this led to highly effective vaccines coming into use and saving lives was unprecedented. And yet, all over the world there is significant resistance to their use and a marked reluctance to their uptake.  

Part of the problem is that science doesn’t exist within a bubble. The “modern” world that science both grew up in and helped to shape, has now morphed into a very different context. Intellectual authority is now a weakness and trust has been undermined. We now have facts, duly established by tried and tested procedures (technical and intellectual) duelling in the media with alt-facts (opinion, suspicion and assertion dressed up as facts). And the individualism that stemmed from the same revolution that gave rise to modern science, means everyone is an expert who has to understand the evidence, even when everyone really isn’t an expert and really can’t weigh the evidence in an appropriate way.

Science really is the best way we have to generate certain types of reliable information of critical importance. It cannot answer any and all questions, but it has and can answer some really important ones. At the edges of course, there is scope for debate as to what is and what is not an appropriate question that can be answered scientifically. Over-claiming, often by prominent scientists, or putting down other approaches in non-scientific domains (like theology among others) has done science no favours. But make no mistake – science has mattered in the past, is making a big impact now, and will be needed in the future. It will continue to matter - bigtime.

Cold fusion and hot money

It turns out that we’ve just passed the 30^th anniversary of the announcement of the most revolutionary of scientific discoveries. On 23^rd March, 1989 Martin Fleischmann and Stanley Pons held a press conference (you can still see it on YouTube) and told the world that they had found a relatively simple way of producing nuclear fusion, the process that fuels stars and very large explosions. Decades and tens of billions of dollars had been spent on finding a way of doing this on earth in a controlled way in order to generate clean energy at minimal cost (the irony!). Fleischmann and Pons (who for brevity I’ll call F & P), claimed they could do it in a test tube with some fancy electrodes, an electric current and water. The process was called “cold fusion” and the reason you’ve probably never have heard of it is, of course, that it quickly turned out that they hadn’t discovered anything of the sort.

It’s hard to overstate the potential implications of their “discovery”. Abundant, cheap, clean energy – imagine the impact that would have had on climate change and the carbon crisis. And it didn’t take long for the notion that cold fusion might have military and strategic applications to start exercising the minds of governments across the globe. In the US, where the announcement was made (F & P conducted their experiments at the university of Utah, where Pons was chairman of the Chemistry department) the Department of Energy went into overdrive.  It ordered its labs to find out if the claims were true, diverting teams of scientists from their own projects. Weekly meetings were convened and reports sent to Washington to the Secretary for Energy. Eventually the President was briefed. This was serious stuff.

The scientific community at large was desperate for details of F & P’s experiment. At the time of their press conference they hadn’t published any of their results, although they had submitted a paper to the Journal of Electroanalytical Chemistry. The Editor understood well the potential importance of their results and fast-tracked the paper through the peer review process. However, when it was published it was relatively short, lacked detail and contained a number of errors. A paper that was submitted to Nature was withdrawn. Very quickly it began to emerge that the initial claims were wrong, the result not so much of fraud or mendacity, but sloppy science, lack of precision and over interpretation. The following year a paper was published reporting results obtained using exactly the same equipment as in the original experiment – no evidence of cold fusion was found. And there you might have though the story would have ended. Interesting to historians or science, but really just a footnote that the rest of us could forget. But then there is the money.

Money, it turns out, is involved in this story from the beginning. The University of Utah quickly had its patent lawyers on the case, and quickly devoted $5M to support cold fusion research. It also lobbied the US Government for tens of millions more dollars for the research (an effort that was unsuccessful). Industry, private equity and philanthropy got involved. By 1992 F & P were in France working with a Toyota subsidiary, an effort that eventually burned through $12M dollars and ended in 1998. Well after mainstream science had moved on, pockets of researchers in both government and private labs continued to beaver away at cold fusion (and still do). There has been no fusion success, although it’s arguable that there might have been some useful spin-offs.

The money kept flowing. Google, no less, spent another $10M on cold fusion research, between 2015 and 2019, announcing only just last week the end of its efforts  (see this commentary in Nature).  It was reported in the Financial Times that a company in North Carolina, founded by a businessman with a background in brickmaking, had attracted upwards of $100M  to develop, you’ve guessed it, cold fusion (although these days it tends to be called “low-energy nuclear reactions”).  Money came from a range of funds and groups. It’s genuinely difficult to tell the grifters from the marks, who are the dupes and who are the gamblers. Somebody appears to making a living (if not useful quantities of electricity) out of the remains of F & P’s ideas.

It turns out that real scientific revolutions are scarce. And they are often only recognised long after the revolution has occurred. Scientific revolutions that have big practical impacts on society, that lead to radical transformations in what and how we do things are even rarer, and usually come from long years of hard slog rather than eureka moments. It’s said of financial advertising that if something seems too good to be true, it probably is. The same is true of scientific claims, particularly those made in press conferences. But it would appear there are gullible people out there, and some of them are minted.

Scientists are human (yes it’s true!) and like most humans we are not immune to influences from outside the lab, from journalists, university administrators, patent lawyers, governments, investors et al. The priority of journalists (for whose benefit press conferences are run), particularly those who have a poor understanding of science, is to simplify and categorise information in the best way get their efforts into news bulletins or prominent pages in the publications they write for. It’s not that they are uninterested in accuracy and precision just that it’s not at the top of their priority list. So we should withstand the temptation of the quick, easy, simple story, and wait for the boring slog of control experiments, confirmation and replication. With cold fusion that’s what happened, and quite quickly. Just not quickly enough for F & P.

While not entirely victims, I do feel a tad of sympathy for F & P. They took the heat (if you’ll allow me to mix my metaphors) that others, who were probably more deserving, escaped. They were the focus of that now infamous press conference. We all marched to the top of the hill with them, before tumbling down the other side. But they then got steamrollered. Many of us, placed under the pressure they found themselves under, might have made some of the poor choices they made. The warning of Proverbs 14:12 comes to mind.