tag:blogger.com,1999:blog-8964719845369935777.post522834230513404138..comments2024-03-27T21:23:40.339-04:00Comments on Chemjobber: The costs of irreproducibilityChemjobberhttp://www.blogger.com/profile/15932113680515602275noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-8964719845369935777.post-26134002135479035652015-09-13T23:35:48.319-04:002015-09-13T23:35:48.319-04:00When I first saw a preclin presentation I was pret...When I first saw a preclin presentation I was pretty enthusiastic about the quality of data. Every point had error bars, statistical info etc. My enthusiasm converted to "this is nuts" attitude when I learned that many of the "statistically processed" experiments consisted of two measurements of unknown quality. The data "point" was the average and the up-down error bars were the distances from the average to the two results. Nod politely, smile, walk away...SJnoreply@blogger.comtag:blogger.com,1999:blog-8964719845369935777.post-80642639330689290542015-09-10T10:28:01.626-04:002015-09-10T10:28:01.626-04:00http://www.bloomberg.com/news/articles/2015-09-10/...http://www.bloomberg.com/news/articles/2015-09-10/the-miracle-fat-pill-that-never-was-and-the-downside-it-exposed<br /><br />on cost of irreproducibilitybiotechtoreadornoreply@blogger.comtag:blogger.com,1999:blog-8964719845369935777.post-35054933290450146102015-09-08T17:24:59.890-04:002015-09-08T17:24:59.890-04:00Wait, they still have journalists at C&EN?Wait, they still have journalists at C&EN?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8964719845369935777.post-30814925472737414362015-09-08T16:59:38.267-04:002015-09-08T16:59:38.267-04:00It would be interesting to find out what percentag...It would be interesting to find out what percentage of this is due to lack of comprehension of basic experimental procedure and design (a surprisingly common issue that invalidates many studies), and how much of this is due to outright fraud.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8964719845369935777.post-54258934742513758342015-09-08T13:16:32.669-04:002015-09-08T13:16:32.669-04:00Meh. Here is an anecdote: I know of an "irrep...Meh. Here is an anecdote: I know of an "irreproducible paper" that got in a good journal that implicated a protein to be required for a certain type of cancer metastasis. In that paper, as far as I can know, it was just one key figure that was not reproducible; but its just that the particular figure (a knock-down experiment) that "proves" that this protein is required. It is likely the rest of the work is perfectly fine. So, at best its about 20%, or less of that work is irreproducible, and so 20% of the, what, 1/4 million dollars to do all that work is wrong. It's just the main conclusion of that paper is wrong.<br /><br />I imagine most of the other irreproducible work could be similar: a key experiment that got the paper into a famous journal is not reproducible, but most of the other work is fine. Its just that crap experiment made the work significant.<br /><br />So I think its more like 20% of all experiments reported are irreproducible, but 80% of the conclusions are wrong.<br /><br />I agree that the big Pharm sending research back to academia is not a good idea. We need people out there that are independent of the ferocious competition of grant-sponsored research to confirm it, and bring the bad labs down.NMHnoreply@blogger.comtag:blogger.com,1999:blog-8964719845369935777.post-77762499924436574862015-09-08T13:04:00.204-04:002015-09-08T13:04:00.204-04:00Dr. Lowe's had some posts on this topic, and 5...Dr. Lowe's had some posts on this topic, and 50% doesn't seem ridiculous (Anywhere between 35% and 65% can probably be justified, at least in some areas - I think the recently cited ones were cancer bio and experimental psychology, which might be the least reproducible areas for various reasons). Unfortunately, since there seems to be an inverse correlation between splashy results and reproducibility, and the splashiest results are the ones most likely to garner VC or pharma funding, the results least likely to be reproducible are the ones that are most likely to cost someone money. This would mean that the amount of money that irreproducible results cost is a larger slice of the funding pie than their portion of the publication pie.Hapnoreply@blogger.com