• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Peer review...a bad thing....a failure....a waste of time.........

mhardy6647

Grand Contributor
Joined
Dec 12, 2019
Messages
11,373
Likes
24,587
Perhaps we should publish the results of our research via twitter and our discipline may judge the merits of the work by the comments it attracts. :)

Full disclosure -- I've published a fair number of papers in peer-reviewed journals over the years... but not lately.
 

Thomas_A

Major Contributor
Forum Donor
Joined
Jun 20, 2019
Messages
3,459
Likes
2,446
Location
Sweden
I publish and review now and then in my field, and I can agree that the system is far from perfect. Publish bias adds to it.
 

Taiga

Member
Joined
May 3, 2022
Messages
31
Likes
37
I only skimmed the article, but as a published research scientist and peer reviewer my initial thoughts are:

Peer review is far from perfect. But it is only one of many steps, and a very early one at that, for scientific findings to be accepted by peers. Reproducibility by other researchers is a subsequent, and more impactful, hurdle that must be cleared. A large number of reported findings simply fade away if not confirmed, at least partially, by subsequent work.
 

Rja4000

Major Contributor
Forum Donor
Joined
May 31, 2019
Messages
2,752
Likes
4,642
Location
Liège, Belgium

Article about the pitfalls of peer review in science. What do you guys think?
I think that's worth a scientific study.
Questioning habits is always worth it, a fortiori for such an important subject.
 

Keith_W

Major Contributor
Joined
Jun 26, 2016
Messages
2,639
Likes
6,035
Location
Melbourne, Australia
Have you guys heard of the "Sokal Squared" affair? Basically, three people sent a number of bogus papers to various humanities journals that they call "grievance studies", or academic fields which manufacture grievances out of nothing. One paper purports to report rape culture among male dogs in a dog park. Another proposes that Astronomy is sexist, they should practice "feminist astronomy" and model the motion of the stars through interpretive dance.

At least in the humanities, peer review is a way to perpetuate groupthink and prevent contrary viewpoints from making it to publication.
 

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,193
Likes
3,754

Article about the pitfalls of peer review in science. What do you guys think?
Ugh. I frankly loathed the author's (a psychologist) glib style after just a few pages in. If I can get past that I'll try to absorb his arguments.

NB: the social sciences aren't the same as 'nonsocial' sciences. And yes, I'm aware peer review can fail spectacularly in 'hard' science too.
 

krabapple

Major Contributor
Forum Donor
Joined
Apr 15, 2016
Messages
3,193
Likes
3,754
Have you guys heard of the "Sokal Squared" affair? Basically, three people sent a number of bogus papers to various humanities journals that they call "grievance studies", or academic fields which manufacture grievances out of nothing. One paper purports to report rape culture among male dogs in a dog park. Another proposes that Astronomy is sexist, they should practice "feminist astronomy" and model the motion of the stars through interpretive dance.

At least in the humanities, peer review is a way to perpetuate groupthink and prevent contrary viewpoints from making it to publication.

The article in question is not about the humanities.
 

jhaider

Major Contributor
Forum Donor
Joined
Jun 5, 2016
Messages
2,865
Likes
4,655
It's an interesting perspective, but doesn't hit the real target. The current business model of the journal publishers looks like a scam to me.

I agree with this but have a different target - reviewers aren’t compensated for their time. Scientific journals are not charitable enterprises, yet a key step required to create them is expected to executed, anonymously, by highly skilled individuals out of the good of their hearts. It is the only such step in the process. So logically why would one not expect it to be a weak point in the process?
 

Tokyo_John

Active Member
Forum Donor
Joined
Mar 6, 2021
Messages
214
Likes
289
I’m a tenured scientist working internationally (based in Japan). There is a lot going on in scientific publishing today, too much to write here, but what it boils down to is escaping a historical artifact, and commodification of science (i.e., rendered as an instrument of profit).

What is publishing? Basically, publishing is just the formal way for scientists to communicate their research results to one another. That’s all it is.

Why is it commodified? It has been commodified for centuries…just like all of publishing, historically, owning the rights to print material was an essential part of the business model of publishing, and helped to underwrite the cost of owning a printing press (and paying the huge staff to keep it going). Only the most powerful members of society (oligarchs) typically owned and controlled a printing press in the past.

How is it commodified? Publishers get editors, reviewers, and readers (all of whom are the same people) to donate their effort for little/no compensation. The peer review process is part of the commodification, because it adds value (like an endorsement) to the product. Scientific publishing is known to have some of the greatest profit margins in the business world, typically ~40% margin. Publishers have also pushed publishing statistics, which university administrations, government agencies, etc., have seized upon as a lazy way of measuring scientific output (by quantity, rather than quality).

How does commodification taint the process? Because the publisher uses peer review as an official endorsement of the copyrighted content, it is over-emphasized and misinterpreted by the public. It is wrongly perceived as some kind of official stamp of approval, rather than the truth that it is just a process for scientists to engage one another as a community, gathering collegial feedback, sometimes to help catch errors or just to help one another improve their science (and its explication).

With the internet, why can’t scientists take over their own communications? Because they’re busy being asked to serve on all kinds of BS committees, increasingly taking on administrative roles as their profession is downsized, etc., they don’t have the spare time. And funding/influence is consolidated among a smaller group who benefit from the status quo, inbreeding resistance to change. You won’t find many boatrockers among the top ranks today (sadly).

What is happening now? Scientists have begun to rebel against journals, refusing to review for some of the worst offenders (such as Elsevier). This puts strain on editors, who can’t find reviewers and they are starting to quit editing. At the same time, young scientists are moving things forward at the grassroots level, establishing new communications venues for science, including “diamond open access” (unfortunately, it makes a poor acronym) journals that are wholly controlled by the scientific community itself. I have been part of some of these myself, and we are planning to launch more in the future. For young people, these are heroic efforts, especially considering that young scientists do not earn any credit toward getting a tenured position by working on such community-building projects (they should be publishing papers in Nature, instead).

Times are changing…
 
Top Bottom