Duke scientists create guidelines for future research

Researchers at Duke are attempting to draw lessons from the highly publicized missteps of one of their own.

The scientists are creating pioneering guidelines for the use of genomics discoveries in clinical trials after concerns were raised by the work of discredited cancer researcher Dr. Anil Potti.

The guidelines [PDF], crafted by a 19-person committee of some of Duke’s top scientists, address four areas, all of which proved problematic in the Potti case: reproducibility and data handling, mathematical expertise, institutional oversight and conflict of interest management. Potti resigned from Duke after some of his genomics-based chemotherapy research was called into question, and clinical trials based on the research were halted.

Dr. Robert Califf, who chaired the committee, said the guidelines are intended as recommendations for future research, not as an analysis of what went wrong in Potti’s case.

“Our job was not to deal with the details of the Potti issue but to bring out the concepts that were issues and then address them,” said Califf, who is also vice chancellor for clinical research and director of the Duke Translational Medicine Institute.

In all, the guidelines, contained in an 18-page document titled “A Framework for the Quality of Translational Medicine with a Focus on Human Genomic Studies,” will raise the level of scrutiny of translational research—laboratory research that is used to guide how doctors care for patients—in genomics and similar fields.

“If you’ve got findings that are going to cause you to treat patients differently in any way, then those are the findings that need to have absolutely the highest degree of rigor in their evaluation,” said David Goldstein, a committee member and director of Duke’s Center for Human Genome Variation.

Before research is applied to humans, the findings should be reproduced using both initial and outside data, according to the framework document. In addition, research should be conducted in collaboration with biostatisticians or others who have the quantitative expertise needed to design studies and analyze results.

Baggerly added that if the guidelines had been in place, many of the problems with Potti’s research would have been caught earlier. In particular, mislabeled data in some of Potti’s publications would likely have been detected sooner, he said.

In some cases, data in Potti’s research was mislabeled in ways that made his methods appear to work when they in fact did not. It remains unclear if Potti’s data was intentionally mislabeled to improve the results, and a research misconduct investigation is ongoing. When Potti resigned from Duke in November, he accepted responsibility for the problems with his research but did not admit to misconduct.

Work on the translational medicine guidelines began even before Potti left Duke. Dr. Victor Dzau, chancellor for health affairs and president and CEO of the Duke University Health System, sent invitations to committee members in mid-August, less than a month after revelations that Potti falsely claimed to be a Rhodes Scholar led to renewed scrutiny of his research. The committee had held five of its six meetings by the time Potti resigned in November 2010.

The committee penned a draft report that was reviewed by external experts in January, and a subsequent draft was sent to Duke Medicine faculty March 3. The latest draft was produced earlier this month.

Duke now faces the challenge of drawing up detailed implementation guidelines based on the framework.

“To me, it’s all about what you do, not about what you say,” said Christopher Newgard, a committee member and director of Duke’s Sarah W. Stedman Nutrition and Metabolism Center. “And the ‘what you do’ part is what we’re working on now.”

Duke will present its implementation plans to the Institute of Medicine committee that is reviewing Potti’s work when that body meets in Washington, D.C. at the end of June, Califf said.

The IOM, a national nonprofit that provides guidance on healthcare issues, has been scrutinizing Potti’s research as it works to develop its own guidelines for creating clinical tests based on genomics and other similar fields that rely on large quantities of biological data.

Several other institutions have been examining issues related to translational genomics research as well.

The National Comprehensive Cancer Network held a panel on genetic testing and cancer treatment at its annual conference in March. The organization, a group of leading cancer centers including Duke’s Cancer Institute, said there is a need for better methods to determine the validity and usefulness of the tests.

The National Cancer Institute, a federal body that supports cancer research, is holding a workshop on the use of tests based on genomics and similar fields—known as ’omics-based predictors—in clinical trials in late June. The NCI will consider the discussions at the workshop as it drafts criteria for the use of such predictors in trials it sponsors, according to the workshop website.

Lisa McShane, an NCI biostatistician and co-chair of the workshop, said interest in the conference has vastly exceeded her expectations. Within three days, more than 175 people applied to attend the workshop, which was originally slated to be held in a venue that could hold 100.

“The issues surrounding the use of ’omics-based predictors in trials are on everyone’s minds in part because of things that have happened at Duke and the IOM panel that has been put together to review how we do this kind of research,” she said. “The NCI felt that there was a general need for people to understand better some important aspects of developing ’omics predictors and requirements that they should satisfy before going into trials.”

She added that the workshop is not about Potti’s research and that the organization has no plans to “rehash what happened at Duke.”

“We had been thinking of doing a workshop like this about predictor development even before the Potti affair,” but the controversy made it more urgent, McShane said. “It made an impact on people in the research community. They stopped and realized how easily things can go wrong.”