Comment by dylan604

Comment by dylan604 2 days ago

4 replies

There's a difference of having your results on your black plastic cookware being off by several factors in an "innocent" math mistake vs deliberately reusing results to fraudulently mislead people by faking the data.

Most people only remember the initial publication and the noise it makes. The updated/retractions generally are not remembered resulting in the same "generally, no consequences" but the details matter

gus_massa 2 days ago

The people in the area remember (probably because they wasted 3 months trying to extend/reproduce the result [1]). They may stop citing them.

In my area we have a few research groups that are very trustworthy and it's safe to try to combine their result with one of our ideas to get a new result. Other groups have a mixed history of dubious results, they don't lie but they cherry pick too much, so their result may not be generalizable to use as a foundation for our research.

[1] Exact reproduction are difficult to publish, but if you reproduce a result and make a twist, it may be good enough to be published.

  • rcxdude a day ago

    This is a general issue with interpreting scientific papers: the people who specialize in the area will generally have a good idea about the plausibility of the result and the general reputation of the authors, but outsiders often lack that completely, and it's hard to think of a good way to really make that information accessible.

    (And I think part of the general blowback against the credibility of science amongst the public is because there's been a big emphasis in popular communication that "peer reviewed paper == credible", which is an important distortion from the real message "peer reviewed paper is the minimum bar for credible", and high-profile cases of incorrect results or fraud are obvious problems with the first statement)

    • gus_massa a day ago

      I completely agree. When I see a post here I had no idea if i's a good journal or a crackpot journal [1]. The impact factor is sometimes useful, but the level in each area is very different. (In math, a usual values is about 1, but in biology it's about 5.)

      Also, many sites just copy&paste the press release from the university that many times has a lot of exaggerations, and sometimes they ad a few more.

      [1] If the journal has too many single author articles, it's a big red flag.

      • rcxdude a day ago

        Yes, I think science communication is also a big part of the problem. It's a hard one to do right, but easy to do wrong and few journalists especially care or have the resources to do it right (and the end results tends to be less appealing, because there's a lot less certainty involved)