Wednesday 26 May 2021

Appealing to Authority

I have previously provided some opinion posts (such as this one) which I take some digs at the state of scientific research at university level in the world today. However, only recently, and potentially ironically, I want to draw attention to a paper was published on 21 May in Science Advances by two researchers from the university of California (Serra-Garcia & Gneezy 2021). This paper is one which I think appears to prove through robust statistical methods that the process of science research dissemination is badly broken.

If I were to paraphrase Serra-Garcia & Gneezy (2021) it is that the most cited articles in the most ‘reputable’ journals turn out to be false. This seems to apply more to the most ‘reputable’ journals such as Nature, Science and the like as these journals tend to publish more ‘groundbreaking’ research articles. Alas, the research that ends up disproving these 'groundbreaking' research articles are boring, so they not published in the ‘reputable’ journals.


Even worse, Serra-Garcia & Gneezy (2021) have shown that between 40%-62% of studies published in the journals Nature and Science have never been successfully replicated.

The end result is that falsified work is more accessible and more cited than the paper that debunks the original research. Serra-Garcia & Gneezy (2021) show that even after research published in a prestigious journal are proved wrong, the level of citation that still gets positively referred to by other researches to back up their research incredibly frequently occurs 300 times more often than the debunking papers. Specifically, the article states shows that non-replicable publications are cited even more after the replication study is published, and persistence of the citation is not explained by negative citations.

In my mind this demonstrates that falsified science can become the accepted science and an enormous amount of further research is built upon this falsified science; or even that false papers are cited more often than good ones.

Universities measure success of their academics and their papers by the number of citations they have received and by the impact factor of the journal (Nature and Science have some of the biggest impact factors). Therefore, it is possible that this is creating a result that the worst researchers (often the most ‘progressive’) are cited and rewarded the most by their universities and funding bodies. Whereas, good science and good researches are relegated… I wonder if you still can judge a paper by the number of citations it has, but counter-intuitively, the more citations the more likely the paper is wrong!

In summary, keep on questioning. Don’t trust research just because an ‘expert in the field’ wrote it or because it is published in a ‘reputable’ or ‘prestigious’ journal. 

Note: I use the word reputable to actually mean headline-grabbing.

The paper is available in full here.


2 comments:

  1. Wow! That doesn't sound good!
    But are you sure this paper's findings haven't been refuted by another? :-D

    ReplyDelete
    Replies
    1. I know, the irony is not lost on me!

      Delete