Re: Journal editor admits involvement in Ermakova "set-up"
Posted: Sun Jan 24, 2016 9:56 amCorrespondence
Nature Biotechnology 25, 1359 - 1360 (2007)
doi:10.1038/nbt1207-1359
NOTICE: THIS WORK MAY BE PROTECTED BY COPYRIGHT
The September Feature was a new format for Nature Biotechnology. My aim in publishing this Feature was to provide an informative presentation of the science behind Ermakova's work, the problems posed by publicizing original data to the media without first publishing it in the peer-reviewed literature, and to open this particular debate to a wider audience. Indeed, many investigators who were unaware of her results now have an opportunity to build on her work and attempt to reproduce it. As I indicated to Ermakova in my original e-mail invitation to her (see Supplementary Materials 1 online), I felt that the biotech community would best be served if she had the opportunity to present her findings and conclusions in her own words—findings and conclusions that could not be published in Nature Biotechnology because of her decision to publicize them in other forums.
As Nature Biotechnology went to press, 20 letters had been submitted to the journal and several directly to the management of Nature Publishing Group concerning the format of this Feature and the process by which it was commissioned. Three letters applauded the journal for a useful and informative analysis of science that had been previously published without peer review. But the vast majority of letters were critical, repeating the points raised here by Irina Ermakova; we have printed above only those letters that present additional concerns.
There appears to be confusion about the way in which this Feature was conceived, commissioned and produced. There is a perception in some quarters that the Feature ultimately published in Nature Biotechnology is the same as a Commentary originally submitted to the journal by Val Giddings. This is not the case. I elected to decline to publish this original Commentary because the critique of Ermakova's work presented was based on data from publicly available sources, which may or may not have been reliable.
Ermakova's existing data were ineligible for peer-reviewed publication because she and others (including Brian John) had already promoted publicly the 2005 data before they received careful scrutiny in a peer-reviewed journal. She had distributed them widely in reports and discussed them with journalists. This contravenes our prepublication policy (http://www.nature.com/authors/editorial ... ality.html). I strongly support this policy. Peer-reviewed publications are the places to publish scientific advances—not press releases, newspapers or postings on the internet. This prepublication policy is shared by all Nature journals and other top-tier science journals. This was made clear to Ermakova several times in our correspondence (see Supplementary Materials 1 online). As Bruce Chassy, Giddings, Alan McHughen and Vivian Moses (Chassy et al.) point out in the September Feature, and Stewart1 has commented in our pages previously, circumventing peer review can have pernicious consequences for the public perception of science.
To provide readers with the most informative article on Ermakova's controversial work, I elected to go directly to her and asked whether she would be willing to describe her work in her own words and to pursue publication in the form of a Feature. My concept was to pose questions to Ermakova and then have a group of researchers respond to her answers. This was explained to Ermakova in the original commissioning e-mail (see Supplementary Materials 1 online).
Because of the controversy surrounding the work, I felt the readers would be interested in a presentation of Ermakova's results in the context of a scientific analysis. Including comments from established scientists was important because to my knowledge her results had not been presented in the context of a skeptical scientific analysis anywhere before.
A concern expressed in the Correspondence by Ermakova and in many letters received by the journal is that the researchers invited to comment on Ermakova's work did not comprise a representative sample of the broad range of views of scientists. On the contrary, Chassy, Moses and McHughen have established publication records, have thought deeply about Ermakova's results, are qualified to discuss their societal impact and can assess the data on the basis of established scientific norms. In drawing up his response, Chassy also consulted with an expert in the field of animal toxicology. In addition, Giddings is a recognized expert and consultant in biotech with respect to policy and regulations. I would also like to point out that contrary to Joe Cummins' assertion, I have no interest in, and never have been, in the field of “public relations on behalf of the biotech industry.”
As Chassy et al. point out, a 'pro-GM' or an 'anti-GM' position is inherently unscientific. I wholeheartedly concur with this viewpoint. The safety and efficacy of any product should be assessed on a case-by-case basis, not according to the method by which it was produced. I am also struck that none of the correspondence elicited by the article has taken issue with the validity of the scientific criticisms made, only the identity of the authors who made them.
I sent Ermakova an initial set of 17 questions, to which she responded. These questions and answers were then forwarded to Giddings and Chassy, who conferred with Moses and McHughen. Their responses were appended to Ermakova's answers and I wrote an introduction explaining why we were publishing the Feature. In the galley proofs seen by Ermakova (Supplementary Materials 2 online), some questions had already been merged and one of the original questions ('What mechanisms do you think might underlie the health effects you observe in your study?') had been removed for conciseness and space constraints. During editing, I dispensed with the question and answer about mechanisms (question number 13 in Supplementary Materials 1 online) as I felt it was unnecessary and inappropriate to speculate on the mechanism of the defects reported by Ermakova, given the serious concerns raised by Chassy et al. over the rigor of the science and the design of the experimental protocol. It turns out that this question is the part of her original draft that contained the references she mentions were removed and gave the impression of her work as “inferior and unsupported by the literature in comparison to the critiques.” Ermakova has now cited some of these omitted references in her letter above; for the rest of the originally cited papers, readers are referred to the list below2, 3, 4, 5, 6, 7.
Ermakova's other concerns related to the editorial process. She asks why I refused to publish new unpublished data from her laboratory, while at the same time assembling and publishing an article that is “a brutal attack on her results.” This is conflating two separate issues, the journalistic criteria for publishing a Feature with the editorial criteria applied to selecting papers for peer review in the Research section. The Feature tackled Ermakova's original 2005 results because of their societal impact and the public attention they garnered when originally circulated widely over the internet and in the media. In contrast, research papers are selected by the journal's editors for evaluation by outside experts on the basis of whether the findings reported are novel, a significant advance over previous work and of sufficient interest to a broad audience. As stated above, Ermakova had disqualified her 2005 data from the latter process by not conforming with our prepublication policy.
I indicated to Ermakova that Nature Biotechnology would be willing to consider any new data she had obtained, and I suggested she submit a presubmission enquiry to the journal. The presubmission enquiry was evaluated by one of our editors, who felt that the results would be better published elsewhere. Ermakova is still welcome to submit the full paper to us; however, promises of being selected for peer review are not made to authors at the presubmission stage. Publication of a journalistic Feature focusing on Ermakova's previous work cannot in any way influence decisions to send new research out for peer review, unless we deem it appropriate according to our editorial criteria for research papers.
Another point raised by Ermakova and by Brian John is that she was sent a 'publication proof' that showed her name as the author. This was a mistake made by Nature Biotechnology when generating the proofs, which I did not check before they were sent to Ermakova. Her name was mistakenly placed on the proof, which contained my introduction and her responses to my questions, but not the comments of Chassy et al. (Supplementary Materials 2 online) The proof was thus much different from the form we had discussed for the final published article (containing comments from other scientists). Clearly, this was confusing and led Ermakova to believe she would be the sole author of the piece.
I accept full responsibility for not reconfirming with Ermakova what I had explained in my original e-mail to her, that her responses were to be part of a larger Feature, and that I would be the author of this journalistic piece. Again, I believe many of the misunderstandings here have arisen due to a wrong perception—both by Ermakova and other correspondents to this journal—that the September Feature is a peer-reviewed research paper, rather than journalistic content.
Ermakova's charge that she never saw the final remarks of Chassy et al. or my introduction to the article also reflects a misunderstanding of the publication process for content that is not peer-reviewed research. The Feature we were preparing on Ermakova's work was intended to be a journalistic Feature for the magazine section of Nature Biotechnology. Like other purveyors of news content who conduct interviews and then publish articles based on the content, there is no precedent for revealing the names or comments of the other contributors to an article. This is standard practice for Nature Biotechnology, other Nature journals and for journalistic content in general. In these circumstances, it is the editor's responsibility to faithfully reproduce the remarks made by the interviewed parties.
There are several take-home lessons from this first experience, if Nature Biotechnology were to repeat this unusual format in the future. We will do a better job ensuring that all authors grasp the process from the start, including authorship and issues surrounding comments made in any interviews. Although I regret that Ermakova misunderstood our publication process, at no time did I indicate that she would be given full authorship of the Feature or that she would see the critiques of the researchers or learn their identities. The key e-mail correspondence between Ermakova and me is presented in Supplementary Materials 1 online so readers can make up their own minds about the quality of the communication process.
In the future, it would be better practice to ask single scientists with particular expertise to respond to different questions rather than publish their comments as a group. In the format published in the September Feature, the comments from Ermakova were appended with collective comments from Chassy et al. In his letter, John raises the point that no one takes “full responsibility” for collective responses. This is one aspect that many of our correspondents found particularly distasteful.
With hindsight, a more thorough editorial effort should be undertaken to ensure that authors whose work is being commented upon have sufficient opportunity to respond to criticisms that are based on insufficiency of data provided. Although I had asked Ermakova to show more behavioral data in response to questions raised by Chassy et al., several other comments in the published text criticized her for not providing other data, to which I gave her no opportunity to respond. That said, Ermakova has now had a full opportunity in these pages to respond to all the comments in full.
I would certainly welcome feedback from readers as to ways in which this Feature format could be improved in the future. One question is whether it is appropriate for a journal to allocate pages in the form of a full research article (as Leifert, Traavik and Heinemann suggest I should have done for Ermakova's experiments) when the primary criteria for editorial selection is the unusual societal and regulatory impact of the work, rather than its scientific quality or impact. Perhaps one solution for such papers would be for their listing on prepublication servers that allow community comment in an open manner and in a neutral environment (e.g., Nature Precedings, http://precedings.nature.com/). Unlike public release in the media, this would not preclude later publication in a journal. I invite readers to make suggestions for ways to present work that has circumvented the traditional peer review process but is nevertheless of interest to the wider research community and public.
Note: Supplementary information is available on the Nature Biotechnology website.
References
1. Stewart, C.N. Nat. Biotechnol. 21, 353–354 (2003) | Article | PubMed | ISI | ChemPort |
2. Pusztai, A., Grant, G., Bardocz, S., Gelencser, E. & Hajos, G. Br. J. Nutr. 77, 933–945 (1997). | Article | PubMed | ChemPort |
3. Richard, S., Moslemi, S., Sipahutar, H., Benachour, N. & Seralini, G.E. Environ. Health Perspect. 113, 716–720 (2005). | PubMed | ChemPort |
4. Rang, A., Linke, B. & Jansen, B. Eur. Food Res. Technol. 220, 438–443 (2005). | Article | ChemPort |
5. Wilusz, C.J., Wang, W. & Peltz, S.W. Genes Dev. 15, 2781–2785 (2001). | PubMed | ISI | ChemPort |
6. Hirotsune, S. et al. Nature 423, 91–96 (2003). | Article | PubMed | ISI | ChemPort |
7. Windels, P., Taverniers, I., Depicker, A., Van Bockstaele, E. & De Loose, M. Eur. Food Res. Technol. 231, 107–112 (2001). | Article |
Nature Biotechnology 25, 1359 - 1360 (2007)
doi:10.1038/nbt1207-1359
NOTICE: THIS WORK MAY BE PROTECTED BY COPYRIGHT
YOU ARE REQUIRED TO READ THE COPYRIGHT NOTICE AT THIS LINK BEFORE YOU READ THE FOLLOWING WORK, THAT IS AVAILABLE SOLELY FOR PRIVATE STUDY, SCHOLARSHIP OR RESEARCH PURSUANT TO 17 U.S.C. SECTION 107 AND 108. IN THE EVENT THAT THE LIBRARY DETERMINES THAT UNLAWFUL COPYING OF THIS WORK HAS OCCURRED, THE LIBRARY HAS THE RIGHT TO BLOCK THE I.P. ADDRESS AT WHICH THE UNLAWFUL COPYING APPEARED TO HAVE OCCURRED. THANK YOU FOR RESPECTING THE RIGHTS OF COPYRIGHT OWNERS.
Response to GM soybeans—revisiting a controversial format
Andrew Marshall responds:
The September Feature was a new format for Nature Biotechnology. My aim in publishing this Feature was to provide an informative presentation of the science behind Ermakova's work, the problems posed by publicizing original data to the media without first publishing it in the peer-reviewed literature, and to open this particular debate to a wider audience. Indeed, many investigators who were unaware of her results now have an opportunity to build on her work and attempt to reproduce it. As I indicated to Ermakova in my original e-mail invitation to her (see Supplementary Materials 1 online), I felt that the biotech community would best be served if she had the opportunity to present her findings and conclusions in her own words—findings and conclusions that could not be published in Nature Biotechnology because of her decision to publicize them in other forums.
As Nature Biotechnology went to press, 20 letters had been submitted to the journal and several directly to the management of Nature Publishing Group concerning the format of this Feature and the process by which it was commissioned. Three letters applauded the journal for a useful and informative analysis of science that had been previously published without peer review. But the vast majority of letters were critical, repeating the points raised here by Irina Ermakova; we have printed above only those letters that present additional concerns.
There appears to be confusion about the way in which this Feature was conceived, commissioned and produced. There is a perception in some quarters that the Feature ultimately published in Nature Biotechnology is the same as a Commentary originally submitted to the journal by Val Giddings. This is not the case. I elected to decline to publish this original Commentary because the critique of Ermakova's work presented was based on data from publicly available sources, which may or may not have been reliable.
Ermakova's existing data were ineligible for peer-reviewed publication because she and others (including Brian John) had already promoted publicly the 2005 data before they received careful scrutiny in a peer-reviewed journal. She had distributed them widely in reports and discussed them with journalists. This contravenes our prepublication policy (http://www.nature.com/authors/editorial ... ality.html). I strongly support this policy. Peer-reviewed publications are the places to publish scientific advances—not press releases, newspapers or postings on the internet. This prepublication policy is shared by all Nature journals and other top-tier science journals. This was made clear to Ermakova several times in our correspondence (see Supplementary Materials 1 online). As Bruce Chassy, Giddings, Alan McHughen and Vivian Moses (Chassy et al.) point out in the September Feature, and Stewart1 has commented in our pages previously, circumventing peer review can have pernicious consequences for the public perception of science.
To provide readers with the most informative article on Ermakova's controversial work, I elected to go directly to her and asked whether she would be willing to describe her work in her own words and to pursue publication in the form of a Feature. My concept was to pose questions to Ermakova and then have a group of researchers respond to her answers. This was explained to Ermakova in the original commissioning e-mail (see Supplementary Materials 1 online).
Because of the controversy surrounding the work, I felt the readers would be interested in a presentation of Ermakova's results in the context of a scientific analysis. Including comments from established scientists was important because to my knowledge her results had not been presented in the context of a skeptical scientific analysis anywhere before.
A concern expressed in the Correspondence by Ermakova and in many letters received by the journal is that the researchers invited to comment on Ermakova's work did not comprise a representative sample of the broad range of views of scientists. On the contrary, Chassy, Moses and McHughen have established publication records, have thought deeply about Ermakova's results, are qualified to discuss their societal impact and can assess the data on the basis of established scientific norms. In drawing up his response, Chassy also consulted with an expert in the field of animal toxicology. In addition, Giddings is a recognized expert and consultant in biotech with respect to policy and regulations. I would also like to point out that contrary to Joe Cummins' assertion, I have no interest in, and never have been, in the field of “public relations on behalf of the biotech industry.”
As Chassy et al. point out, a 'pro-GM' or an 'anti-GM' position is inherently unscientific. I wholeheartedly concur with this viewpoint. The safety and efficacy of any product should be assessed on a case-by-case basis, not according to the method by which it was produced. I am also struck that none of the correspondence elicited by the article has taken issue with the validity of the scientific criticisms made, only the identity of the authors who made them.
I sent Ermakova an initial set of 17 questions, to which she responded. These questions and answers were then forwarded to Giddings and Chassy, who conferred with Moses and McHughen. Their responses were appended to Ermakova's answers and I wrote an introduction explaining why we were publishing the Feature. In the galley proofs seen by Ermakova (Supplementary Materials 2 online), some questions had already been merged and one of the original questions ('What mechanisms do you think might underlie the health effects you observe in your study?') had been removed for conciseness and space constraints. During editing, I dispensed with the question and answer about mechanisms (question number 13 in Supplementary Materials 1 online) as I felt it was unnecessary and inappropriate to speculate on the mechanism of the defects reported by Ermakova, given the serious concerns raised by Chassy et al. over the rigor of the science and the design of the experimental protocol. It turns out that this question is the part of her original draft that contained the references she mentions were removed and gave the impression of her work as “inferior and unsupported by the literature in comparison to the critiques.” Ermakova has now cited some of these omitted references in her letter above; for the rest of the originally cited papers, readers are referred to the list below2, 3, 4, 5, 6, 7.
Ermakova's other concerns related to the editorial process. She asks why I refused to publish new unpublished data from her laboratory, while at the same time assembling and publishing an article that is “a brutal attack on her results.” This is conflating two separate issues, the journalistic criteria for publishing a Feature with the editorial criteria applied to selecting papers for peer review in the Research section. The Feature tackled Ermakova's original 2005 results because of their societal impact and the public attention they garnered when originally circulated widely over the internet and in the media. In contrast, research papers are selected by the journal's editors for evaluation by outside experts on the basis of whether the findings reported are novel, a significant advance over previous work and of sufficient interest to a broad audience. As stated above, Ermakova had disqualified her 2005 data from the latter process by not conforming with our prepublication policy.
I indicated to Ermakova that Nature Biotechnology would be willing to consider any new data she had obtained, and I suggested she submit a presubmission enquiry to the journal. The presubmission enquiry was evaluated by one of our editors, who felt that the results would be better published elsewhere. Ermakova is still welcome to submit the full paper to us; however, promises of being selected for peer review are not made to authors at the presubmission stage. Publication of a journalistic Feature focusing on Ermakova's previous work cannot in any way influence decisions to send new research out for peer review, unless we deem it appropriate according to our editorial criteria for research papers.
Another point raised by Ermakova and by Brian John is that she was sent a 'publication proof' that showed her name as the author. This was a mistake made by Nature Biotechnology when generating the proofs, which I did not check before they were sent to Ermakova. Her name was mistakenly placed on the proof, which contained my introduction and her responses to my questions, but not the comments of Chassy et al. (Supplementary Materials 2 online) The proof was thus much different from the form we had discussed for the final published article (containing comments from other scientists). Clearly, this was confusing and led Ermakova to believe she would be the sole author of the piece.
I accept full responsibility for not reconfirming with Ermakova what I had explained in my original e-mail to her, that her responses were to be part of a larger Feature, and that I would be the author of this journalistic piece. Again, I believe many of the misunderstandings here have arisen due to a wrong perception—both by Ermakova and other correspondents to this journal—that the September Feature is a peer-reviewed research paper, rather than journalistic content.
Ermakova's charge that she never saw the final remarks of Chassy et al. or my introduction to the article also reflects a misunderstanding of the publication process for content that is not peer-reviewed research. The Feature we were preparing on Ermakova's work was intended to be a journalistic Feature for the magazine section of Nature Biotechnology. Like other purveyors of news content who conduct interviews and then publish articles based on the content, there is no precedent for revealing the names or comments of the other contributors to an article. This is standard practice for Nature Biotechnology, other Nature journals and for journalistic content in general. In these circumstances, it is the editor's responsibility to faithfully reproduce the remarks made by the interviewed parties.
There are several take-home lessons from this first experience, if Nature Biotechnology were to repeat this unusual format in the future. We will do a better job ensuring that all authors grasp the process from the start, including authorship and issues surrounding comments made in any interviews. Although I regret that Ermakova misunderstood our publication process, at no time did I indicate that she would be given full authorship of the Feature or that she would see the critiques of the researchers or learn their identities. The key e-mail correspondence between Ermakova and me is presented in Supplementary Materials 1 online so readers can make up their own minds about the quality of the communication process.
In the future, it would be better practice to ask single scientists with particular expertise to respond to different questions rather than publish their comments as a group. In the format published in the September Feature, the comments from Ermakova were appended with collective comments from Chassy et al. In his letter, John raises the point that no one takes “full responsibility” for collective responses. This is one aspect that many of our correspondents found particularly distasteful.
With hindsight, a more thorough editorial effort should be undertaken to ensure that authors whose work is being commented upon have sufficient opportunity to respond to criticisms that are based on insufficiency of data provided. Although I had asked Ermakova to show more behavioral data in response to questions raised by Chassy et al., several other comments in the published text criticized her for not providing other data, to which I gave her no opportunity to respond. That said, Ermakova has now had a full opportunity in these pages to respond to all the comments in full.
I would certainly welcome feedback from readers as to ways in which this Feature format could be improved in the future. One question is whether it is appropriate for a journal to allocate pages in the form of a full research article (as Leifert, Traavik and Heinemann suggest I should have done for Ermakova's experiments) when the primary criteria for editorial selection is the unusual societal and regulatory impact of the work, rather than its scientific quality or impact. Perhaps one solution for such papers would be for their listing on prepublication servers that allow community comment in an open manner and in a neutral environment (e.g., Nature Precedings, http://precedings.nature.com/). Unlike public release in the media, this would not preclude later publication in a journal. I invite readers to make suggestions for ways to present work that has circumvented the traditional peer review process but is nevertheless of interest to the wider research community and public.
Note: Supplementary information is available on the Nature Biotechnology website.
References
1. Stewart, C.N. Nat. Biotechnol. 21, 353–354 (2003) | Article | PubMed | ISI | ChemPort |
2. Pusztai, A., Grant, G., Bardocz, S., Gelencser, E. & Hajos, G. Br. J. Nutr. 77, 933–945 (1997). | Article | PubMed | ChemPort |
3. Richard, S., Moslemi, S., Sipahutar, H., Benachour, N. & Seralini, G.E. Environ. Health Perspect. 113, 716–720 (2005). | PubMed | ChemPort |
4. Rang, A., Linke, B. & Jansen, B. Eur. Food Res. Technol. 220, 438–443 (2005). | Article | ChemPort |
5. Wilusz, C.J., Wang, W. & Peltz, S.W. Genes Dev. 15, 2781–2785 (2001). | PubMed | ISI | ChemPort |
6. Hirotsune, S. et al. Nature 423, 91–96 (2003). | Article | PubMed | ISI | ChemPort |
7. Windels, P., Taverniers, I., Depicker, A., Van Bockstaele, E. & De Loose, M. Eur. Food Res. Technol. 231, 107–112 (2001). | Article |