Thursday, June 6, 2013

Amos Smith speaks on data integrity

So this editorial in Organic Letters is both reassuring and more than a little frightening:
I write to alert the organic chemistry community to a serious problem related to the integrity of data being submitted for review and publication by Organic Letters and to outline steps that the Journal is taking to address this concern. Recently, with the addition of a Data Analyst to our staff, Organic Letters has begun checking the submitted Supporting Information more closely. As a result of this increased scrutiny, we have discovered several instances where reported spectra had been edited to remove evidence of impurities. 
Such acts of data manipulation are unacceptable. Even if the experimental yields and conclusions of a study are not affected, ANY manipulation of research data casts doubts on the overall integrity and validity of the work reported. [snip] 
...The Associate Editors and I give notice to the community that Organic Letters will enforce these guidelines and will assess significant penalties for infractions that entail data manipulation. 
In some of the cases that we have investigated further, the Corresponding Author asserted that a student had edited the spectra without the Corresponding Author’s knowledge. This is not an acceptable excuse! The Corresponding Author (who is typically also the research supervisor of the work performed) is ultimately responsible for warranting the integrity of the content of the submitted manuscript. [snip] 
The responsibility to foster a research environment where all involved can confidently present their results, even if they are not optimal, resides with each research supervisor and Corresponding Author.At times, the inherent power of a research advisor’s position can create an atmosphere that leads some to embellish results. In this vein, I echo the recommendation of the IAP-IAC Committee on Research Integrity (Responsible Conduct in the Global Research Enterprise: A Policy Report, InterAcademy Council / IAP, 2012): “Research institutions need to establish clear, well communicated rules that define irresponsible conduct and ensure that all researchers, research staff, and students are carefully trained in the application of these rules of research. Research institutions also need to create an environment that fosters research integrity through education, training, and mentoring and by embracing incentives that deter irresponsible actions.” (boldface added for emphasis).
He goes on to conclude that this is a small percentage of submissions, etc., etc. I believe that this is a big deal, but I'll be interested to see what sanctions (if/when) will be imposed... 

12 comments:

  1. As I said over at my place, this editorial should probably have been distributed to a wider audience (I never would have seen this without your tweet). It probably goes beyond organic chemistry and synthetic chemistry. So thanks for bringing my attention to this piece.

    I also echo your skepticism on punishment because who ever gets punished for ethics violations? (Answer: grad students and post docs)

    ReplyDelete
  2. "In some of the cases that we have investigated further, the Corresponding Author asserted that a student had edited the spectra without the Corresponding Author’s knowledge. This is not an acceptable excuse!"

    Absolutely true. But I wonder how this would alter behavior in practice since even well-established PIs don't usually personally check every peak in the spectrum.

    ReplyDelete
  3. CoulombicExplosionJune 6, 2013 at 4:33 PM

    Wavefunction - Although PIs may not exhaustively double-check all pieces of data, it is their responsibility that those performing research under their guidance understand that such manipulation is not permissible. Furthermore, it is important that PIs establish this working culture because the high-turnover nature of post-doc/grad student labor means it is unlikely to be sustained by any other means.

    Also, I'm glad to see that Amos appreciates the unenviable position of students stuggling with a project under a demanding PI. Although students may feel compelled to manipulate data in order to make a result more appealing, they need to be able to accurately and honestly report their findings, whatever they are, without incurring the wrath of their PI. This is especially frustrating when PIs think they know what the result "should" be before conducting an experiment.

    ReplyDelete
  4. It would be really nice if (somehow) the PIs responsible would be placed on some sort of list that the funding sources could check. Serial (or serious) perpetrators could suffer loss of Federal funding.

    ReplyDelete
  5. I'll be very surprised if this plan is actually implemented - there's a strong culture of sweeping embarrassing facts under the rug in academia, and calling people out for manipulating spectra is going to embarrass some famous PI's.

    ReplyDelete
  6. It's about time that someone cracked down on some of the junk that gets published in Org. Lett.

    ReplyDelete
  7. It is a bit shocking how few organic papers there are here relative to other fields- I guess Westerns etc. are usually a showpiece while NMRs are highly supplemental...

    http://retractionwatch.wordpress.com/

    ReplyDelete
  8. The Iron ChemistJune 7, 2013 at 9:08 AM

    Should we even get started on inflated yields? I know of at least one high-profile lab where workers would weigh their product (1-4 mg scale), be dissatisfied with the yield, touch the tared flask with their bare fingers, then re-weigh it to get a superior yield.

    With respect to Andre's comment, the big fish won't be touched. Excuses will be made and accepted without question, and business will remain as usual.

    ReplyDelete
    Replies
    1. That is particularly appalling. I have never had any problem with bare handed handling of flasks, but that's because I have always intentionally stayed away from mg scale. I prefer having a little more room for my stupid errors. I understand that not everything can be done on a 10-100g scale, but I also know the procedure when your balance has a tolerance of <0.1 g.

      Delete
  9. Until funding agencies weigh in on this with probationary measures or complete retraction/blacklisting of funding, it will continue. Unfortunately, who are the people that sit on panels? The same people applying for funding. I would really enjoy doing some grant review service as a "disinterested" third party, but my job wouldn't allow me the time off to volunteer for a review panel.

    ReplyDelete
  10. Ironic editorial coming from Amos Smith.

    ReplyDelete
  11. Bad news: Some ‘strange’ assignments from C-NMR can be found on

    http://nmrpredict.orc.univie.ac.at/csearchlite/NMR_misinterpretation.html

    Good news: Automatic verification of C-NMR data is available (BTW: since 4 years !)

    http://nmrpredict.orc.univie.ac.at/c13robot/robot_jsme.php

    (its free of charge, registration before usage is necessary)

    ReplyDelete