Japanese ADNI Called to Defend Its Data Handling
Quick Links
Allegations of falsified clinical data rocked the Japanese Alzheimer’s Disease Neuroimaging Initiative (J-ADNI) on January 10. An unidentified project researcher made the claims to the Japanese health ministry, which in turn questioned J-ADNI leaders, according to an article in Japan Today. J-ADNI principal investigators deny any wrongdoing. They told Alzforum that changes to the data represented corrections that followed standard quality-control procedures. Leading scientists of the North American ADNI project expressed confidence in J-ADNI methods and said they expected the researchers to be fully exonerated. Meanwhile, the Japanese media picked up the story, in one case publishing a distorted account that contained erroneous quotations, according to the quoted scientist. News outlets in Australia and other countries in the regions ran the story, as well. It is too early to tell if the allegations will damage the research project.
Modeled closely on the U.S. ADNI protocol, J-ADNI launched in 2007 with the goal of identifying biomarkers to help predict progression from mild cognitive impairment (MCI) to AD. The project follows 600 participants at 38 sites in Japan using neuroimaging, fluid biomarkers, and neuropsychological tests, and is funded for roughly $32 million (3.3 billion yen) by a combination of government grants and pharmaceutical partners.
According to newspaper articles, the anonymous researcher claimed two different problems with ADNI data: one, that in a handful of cases, time intervals on a memory test were falsified; and two, that around 20 percent of participants did not meet study criteria, making data from them unusable. Takashi Asada at the University of Tsukuba, Japan, who runs J-ADNI’s clinical core along with Hiroyuki Arai at Tohoku University, Sendai, held a press conference January 10 to address the allegations. According to a January 11 article in the print edition of the Japanese newspaper Yomiuri Shimbun, Asada denied there had been any falsification of data. He said that normal errors in data processing had occurred and had been corrected according to standard protocols. When contacted by Alzforum, Asada confirmed these statements.
The newspaper Asahi Shimbun gave a different account of the conference, quoting Asada as admitting to data falsifications. On January 11, the newspaper published an editorial chastising the state of Japanese clinical science. Asada told Alzforum that the Asahi Shimbun article contained numerous inaccuracies. “All quotes attributed to me are false … This story is fake, and even worse, it besmears the hard work of professionals and participants in this extremely important area,” he wrote to Alzforum (see full comment below).
J-ADNI principal investigator Takeshi Iwatsubo at the University of Tokyo told Alzforum that the allegations reflected a misunderstanding of how data is checked and corrected in large multicenter studies. According to newspaper accounts, the anonymous informant identified several instances where the centralized J-ADNI data core sent queries to investigators that resulted in researchers changing the time interval on a memory test. The informant interpreted this as a falsification. However, Iwatsubo said this process is part of standard data checks. The recall test is supposed to be administered 30 minutes after participants learn the material, so any interval of less than 20 or more than 40 minutes triggers an automatic query from the data center to investigators, Iwatsubo said. The researchers then check their records to see if they made an error in recording the time, and send any corrections back to the data center.
Laurel Beckett at the University of California, Davis, who directs U.S. ADNI’s biostatistics core, said the U.S. study follows the same protocol, which represents best practices for the field. “There is a standard data-checking and cleaning process in place, so that if an inconsistency is noted, the query can be submitted, the checking carried out, and the resolution fully tracked. In a system like ours, the idea of ‘data tampering’ is, frankly, not credible,” she wrote to Alzforum (see full comment below). J-ADNI largely adopted the protocol and processes of the U.S. ADNI in order to enable data pooling and increase power for analysis across countries.
The second allegation asserts that data from 20 percent of J-ADNI participants should be thrown out, either because they turned out not to have Alzheimer’s disease, or because they were taking medications that were not approved for the study. However, Paul Aisen at the University of California, San Diego, who heads U.S. ADNI’s clinical core, told Alzforum that data collected from this group has value. “One of the goals of ADNI is to understand what is and is not Alzheimer’s disease, and part of that process is enrolling people with MCI and following them to see which individuals move in one direction and which move in another direction,” he said. Thus, biomarker data from people who turn out to have other pathologies is helpful in refining AD diagnosis. In the North American ADNI, too, a fraction of people who are enrolled in the MCI group subsequently progress to vascular or frontotemporal dementia, other diseases, or even revert to normal cognition on follow-up.
Data from patients taking unapproved medications can be appropriate as well, Aisen said. “As an observational study, we want to enroll a representative group of individuals, and that means being relatively broad in the characteristics that are allowed,” he noted.
Across the board, U.S. investigators expressed strong support for their colleagues in Japan. “I have worked with the investigators at J-ADNI from the start. I think they are outstanding. I am absolutely confident that there is no misconduct at the J-ADNI program, and that the allegations are entirely false,” Aisen said.
Mark Schmidt, a senior director at Janssen Pharmaceutical, one of many pharma sponsors of the North American ADNI project, wrote to Alzforum, “J-ADNI represents a very important contribution to our understanding of Alzheimer’s disease and to the design of trials for treatments intended to intercept disease progression. My opinion is the Asahi Shimbun report was driven by misinterpretation of data-collection and recording procedures that were conducted in good faith by J-ADNI investigators, and I expect they will be absolved.”
Michael Weiner at the University of California, San Francisco, who is the worldwide ADNI’s principal investigator, wrote to Alzforum, “I know that I can speak for all the core leaders of ADNI when I say we have the highest admiration and respect for Takeshi Iwatsubo and his coworkers in J-ADNI. We look forward to continuing to work with them in the quest to validate biomarkers for AD clinical trials.”
In Japan, a government probe is ongoing to determine whether the issue requires a larger investigation. Yasuo Ihara, a highly respected Alzheimer researcher at Doshisha University in Kyoto and a senior advisor to J-ADNI, expects the situation will be resolved soon. “What I know about the present J-ADNI is that there is no purposeful mishandling of the data, although there were some small errors. Those small errors are not infrequent with such a large-scale project,” Ihara wrote to Alzforum. An investigation committee was appointed at the University of Tokyo, where the ethics committee that approved the original J-ADNI protocol is located.—Madolyn Bowman Rogers.
References
External Citations
Further Reading
Annotate
To make an annotation you must Login or Register.
Comments
University of Tsukuba
I am pleased to have this opportunity to clear up this situation. First, allegations of falsification of data made by me are themselves false. I conducted a press conference on January 10, where I stated that large-scale data sets are susceptible to errors in recording, processing, and management. Our research—the first of its kind on this scale—has encountered difficulties that are common with populations of this size, such as: mis-answered questionnaires, subjects whose medication affects performance and should not be included in the study, and subjects from whom informed consent has not been received. The number of subjects whose data should come under scrutiny for the above reasons is about 80, out of a total sampling cohort of 545. When working with a population that includes individuals with varying degrees of cognitive impairment, these kinds of errors are not unexpected and should be seen in this context. In addition, there are recording errors when the data is entered into the data base. Problems are also occurring within the database itself, which again can be attributed to the novelty of this study. Overall, any alleged purposeful mishandling of data is untrue, and any accidental errors should be seen in the context of scope and nature of this study. Finally, let me comment on what I allegedly said in the Asahi Shimbun article. To put it succinctly, all quotes attributed to me are false. I never made any of the statements appearing in this news article. I will not speculate nor do I wish to comment on reasons for this story from this otherwise respectable newspaper. This story is fake, and even worse, it besmears the hard work of professionals and participants in this extremely important area. This kind of facetious journalism unfortunately not only damages the reputation of science in this country, but also journalistic integrity. I sincerely hope this reassures your readers and others connected to Alzforum that our research team upholds the highest standard of ethical practice in all aspects of study, from receiving informed consent from subjects to the handling and processing of data.
University of California, Davis, School of Medicine
I know about the J-ADNI allegations mostly through our colleagues in Japan. My first impression, at a distance and through several filters, is that the allegations represent a misinterpretation of a standard data-checking process that, as such things do, uncovered a very small number of inconsistencies that were subsequently resolved by checking records and checking with the clinicians. All serious research projects like ADNI and J-ADNI are expected to have a process in place not only for trying to ensure that the data entered into the computer are accurate in the first place, but also to check for any inconsistencies that might be detected later, and to correct them through a careful process that tracks queries, resolutions, and any changes to the data. ADNI and J-ADNI have well-documented data quality procedures; as far as I know, they are very similar. In ADNI, some checks are put in place to catch mistakes at the time of initial data entry—for example, you could not enter into the ADNI database a year of birth "2014" (this happens sometimes on paper forms, when someone just carelessly writes the current year instead of the actual year of birth). But other checks can only be made when you merge files or check data from different places. For example, you can't have the date of a follow-up visit occur before the baseline visit. But you can only check this inside a merged database, where you can look at the dates side by side. So if someone wrote "2013" for the follow-up instead of "2014," the computer might not catch it at time of data entry, but our staff might see it when we do an analysis of change and have to calculate the time between visits. If a researcher finds such an anomaly in the data, a query is sent to a central data-management unit. They go back (or ask the site to go back) and check the records and resolve the problem. Then a correction is submitted to the master data set. Data checking and cleaning is therefore an ongoing process; little questions come up from time to time as people look more intensively at the data. We "version" the data so we can be sure we are looking at the most recent and correct version. We track what corrections are made, and by whom, with a program that allows undoing changes, or going back to an earlier version, if needed. We track queries to make sure they are resolved in a timely fashion, and how they are resolved. J-ADNI has a very similar process, to the best of my understanding. I was in Japan when they were setting up the data management and met with their group, though I have not seen them in action recently. They seemed very capable. The fact that an investigation can track the queries, and how they were resolved and why, to me is a good sign. That means there is a standard data-checking and cleaning process in place, so that if an inconsistency is noted, the query can be submitted, the checking carried out, and the resolution fully tracked. In a system like ours, the idea of "data tampering" is, frankly, not credible. That suggests someone sneaked in during the night and changed some values. I daresay that’s not happening. We track any inconsistencies we find, we track what people do to try to resolve them, and we track what happened to the database. Continual monitoring for possible problems, and fixing them by a documented and reproducible process is not just standard practice, it's "best practice." I do not know enough details about the situation in Japan to comment with certainty. I can only speak about best practices in general, our efforts to conform to those, and my general understanding of J-ADNI practices.
Make a Comment
To make a comment you must login or register.