Comment by aDyslecticCrow

Comment by aDyslecticCrow 6 months ago

33 replies

View on Hacker News

This is amazing. The trust of peer review as a stamp of quality among academics is dwindling, and distrust of science among the population is growing (within the increasing politicisation of some areas of research). Transparent peer review raises the bar for both academics and enhances the potential trust in the process.

This is desperately needed as AI could further deteriorate the quality of science if the publishing process is not made more strict. This represents a significant step forward in rigorous science. I hope other publications follow suit.

jfengel 6 months ago

Of the people who distrust science, how many of them have ever read a scientific paper?

I suspect the number is low. If that's the case, they're unlikely to be more convinced by the presence of published peer review, either.

Reply View 28 replies
  • [removed] 6 months ago
    [deleted]
    Reply View 0 replies
  • mike_hearn 6 months ago

    Lots of them have. Look at any site where science skepticism is regularly posted, you will find that maybe a good 50% of the content is commentary on papers. Of those who don't trust science and haven't read papers, they will usually have read commentary by those who have.

    Source: I've published such skepticism in the past and met people who read my articles, including politicians.

    Nonetheless, you're right that simply publishing peer reviews won't help improve trust. The situation is bad enough that there's no One Weird Trick that can dig academia out of its hole. Some of the most intellectually fraudulent papers I've written about in the past did have transparent peer review, and all it showed was that peer reviewers were often aware of the critical problems found inside and waved it through anyway. Or that their feedback was ignored. Or that they agreed with obviously bogus practices. Or that the parts of the paper that revealed the problems weren't reviewed (eg. appendices, github repositories). After all, nobody cares about papers that got rejected by the system, the distrust is driven by the papers that weren't, so almost by definition such papers either were badly reviewed or the review wasn't used.

    Reply View 0 replies
  • aDyslecticCrow 6 months ago

    I think there are a few groups and reasons of distrust. Some more or less valid.

    Those that distrust authority as a whole and lean into conspiracies cannot be saved with this kind of thing.

    But i think news and science are having similar perception issues recently.

    Distrust for news growing among the average population (for some good reasons). People are loosing faith in the objectivity of established media organisations. Most people are exposed to science through these traditional news.

    So adding back some sense of confidence and authority to scientific institutions is very valuable to non-academics. Even if they themselves would not read the papers or revews.

    Reply View 0 replies
  • NoMoreNicksLeft 6 months ago

    >Of the people who distrust science,

    Why should anyone trust science? Skepticism should always be the default position. Putting it on a pedestal to be worshipped is what led us all into this mess. If science needs trust to work, then whatever it is doing is something I'd like to see fail.

    Reply View 16 replies
    • JumpCrisscross 6 months ago

      > Why should anyone trust science?

      When people talk about those who distrust science, they aren't referring to the carefully sceptical. They're talking about people who come to a conclusion and then reject any evidence against it.

      Reply View 8 replies
      • mikeyouse 6 months ago

        Right. RFK supporters aren’t carefully weighing evidence to arrive at their conclusions, they are blindly following someone who is a mindless contrarian with a 30yr vendetta against vaccines. No amount or quality of peer review is going to change any of their minds.

        Reply View 1 reply
        • JumpCrisscross 6 months ago

          > RFK supporters aren’t carefully weighing evidence to arrive at their conclusions, they are blindly following someone

          I'd actually be curious what fraction of his supporters share his views on vaccines. It strikes me as more likely that they like one of his random, idiosyncratic views and are willing to excuse the anti-vaccine nonsense to get those policy outcomes.

          Reply View 0 replies
      • fc417fc802 6 months ago

        That, but also people who take the blanket view that none of the work produced by the scientific establishment is trustworthy. An ironic stance considering that it's the inverse of the blind worship that they accuse others of.

        Reply View 5 replies
    • kergonath 6 months ago

      > Why should anyone trust science? Skepticism should always be the default position.

      Skepticism is not anti-scientific. Hell, distrusting results and theories is not anti-scientific. Distrusting the scientific method is. There is a difference.

      > Putting it on a pedestal to be worshipped is what led us all into this mess.

      Scientists did not ask for this. Amongst all high profile politicians, those who whine about science becoming political are those who made it so, by taking contrarian positions to rile up their voter base. Most people who make this point are not being honest. If you are, you should make specific arguments rather than rehash propaganda.

      Reply View 6 replies
      • NoMoreNicksLeft 6 months ago

        >Skepticism is not anti-scientific.

        One might think that to be the case, but science is in a hurry to save our planet, and it has no patience for your skepticism. It may already be too late.

        >Distrusting the scientific method is.

        Maybe I don't trust it either. Maybe because I'm distrustful, the scientific method should prove, over and over again, that it's worth a damn. And maybe when it's finally proven that, maybe I go on distrusting it... and make it prove it some more. If one truly though the scientific method worthy of any respect, such a person might say "that's ok, because it will go on re-proving what has already been proven". But so little do I ever hear of that sentiment. Like in this very thread.

        >Scientists did not ask for this.

        Probably not. I'd be skeptical of the theory that they asked for it. But it doesn't seem altogether implausible that they might have enjoyed it once it happened even though they didn't ask. And liking it, they started behaving in ways that encouraged it just the same. And those scientists who enjoyed it the most elbowed their way to the front to egg it on even more. And so that's where we are now. I mean, I'm a little skeptical that it happened that way, and I'd welcome evidence that disproves it.

        >Most people who make this point are not being honest.

        It's really sort of hilarious how it's only my side that can ever be honest. Not the other guys'. And this is true no matter who "me" is. None of your opponents, for instance, can ever be genuine... they're always trying to cheat. And if your side tries to cheat, well, it's with the best of intentions because the stakes are so high. Not that you would cheat, only the other side ever does that.

        >f you are, you should make specific arguments rather than rehash propaganda.

        If we can't start here, then there can be no real conversation. Basically you're just asking me to surrender and admit that I'm the bad guy trying to cheat and swindle, everything I say is a lie, and you're the innocent victims. Haha.

        Reply View 5 replies
  • oerpli 6 months ago

    I would go further: Anyone who has published a "scientific paper" in the last decade or so either "distrusts science" or is more likely than not a mid-wit at best.

    Your posting doesn't give me the impression that you're very familiar with "science".

    Reply View 5 replies
    • iorrus 6 months ago

      Absolutely, the quickest way to lose faith in "The Science" is actually to do Science in a formal research institute....

      Reply View 2 replies
      • probably_wrong 6 months ago

        Isn't that true for everything, though? If I weren't a software developer I wouldn't know that I have to worry about questions like "has this plane been rebooted in the last 51 days?" [1] or "does this bank offer anything other than SMS as second factor?".

        Maybe structural engineers feel safer after their Master's when they traverse a bridge, but I bet that's more the exception than the rule.

        [1] https://www.theregister.com/2020/04/02/boeing_787_power_cycl...

        Reply View 0 replies
      • petschge 6 months ago

        Seeing how the sausage gets made makes you realize ALL the downsides and things you'd rather not have known. That doesn't mean you can think of a way to fix science. Let alone get the required funding to make an actual attempt.

        Reply View 0 replies
    • kergonath 6 months ago

      > Anyone who has published a "scientific paper" in the last decade or so either "distrusts science" or is more likely than not a mid-wit at best.

      That is very unhelpful, to say the least. The amount of noise has increased, but it does not mean that the scientists who know their subject disappeared. They are still around and not any less bright than their predecessors were 30 years ago.

      Reply View 1 reply
      • rcxdude 6 months ago

        Steel-manning their argument, 'distrusting science' doesn't mean they throw the whole thing out, they're just aware that there is disagreement and bullshit going around within the process. As far as I can tell, it's dangerous to try to assess a topic through reading papers alone. A scientist active in the field will have read more widely, be aware of the reputations and biases of the different groups, and likely will have tried some of the published experiments themselves (replication does happen, if it's an interesting result, it's just rarely worth publishing the result).

        Reply View 0 replies
  • Dig1t 6 months ago

    The fact remains that distrust of science continues to grow. Up to now the establishment’s response has been one of condescension. Your comment echoes that attitude.

    Ignoring the problem is not going to fix it, and in fact continuing to regard these people as beneath you is only going to accelerate the downfall of this system.

    Reply View 1 reply
    • vkou 6 months ago

      > The fact remains that distrust of science continues to grow.

      Of course it does, billion-dollar interests who have a vested interest in attacking it have a well-funded propaganda arm, and as we've been discovering over the past century - angry, loud, and frequently repeated bullshit with a profit motive floods any signal out of the room.

      What's really sad is people who have legitimate concerns or desires for improvement hitching their horse-cart to the former.

      No amount of peer review or replication is going to convince someone whose fortune is built on peddling snake oil.

      > Ignoring the problem is not going to fix it,

      Here's a solution. Hold the people making these attacks to the same level of rigor as what they are attacking. Stop giving proven liars a loudspeaker. The day the press will do that is the day some meaningful progress to fix that damage may be made.

      Until you do that, we're going to continue getting shit outcomes.

      Reply View 0 replies
kergonath 6 months ago

> The trust of peer review as a stamp of quality among academics is dwindling

The thing is, peer review is not a stamp of quality, and never was. It is just the basic level of due diligence. The referees cannot reproduce the results most of the time for a lot of very good reasons. They are here as a sanity check, to ensure that the work avoided common pitfalls and actually makes sense.

What most people do not understand is that articles are not good because they are peer-reviewed; it’s the lack of peer review that is a red flag. Amongst reviewed articles, a lot of them will turn out to be wrong or flawed in ways that are impossible for the reviewers to find out.

Reply View 0 replies
thfuran 6 months ago

>and enhances the potential trust in the process.

I highly doubt it's a meaningful factor in public trust.

Reply View 0 replies
Ar-Curunir 6 months ago

The general public does not distrust science because peer reviews are not public.

Reply View 1 reply
  • kergonath 6 months ago

    Right. The general public cannot understand the points that are being made during peer review.

    Reply View 0 replies