Casanova R, Varma S, Simpson B, Kim M, An Y, Saldana S, Riveros C, Moscato P, Griswold M, Sonntag D, Wahrheit J, Klavins K, Jonsson PV, Eiriksdottir G, Aspelund T, Launer LJ, Gudnason V, Legido Quigley C, Thambisetty M. Blood metabolite markers of preclinical Alzheimer's disease in two longitudinally followed cohorts of older individuals. Alzheimers Dement. 2016 Jul;12(7):815-22. Epub 2016 Jan 21 PubMed.

Recommends

Please login to recommend the paper.

Comments

  1. Two studies reported contradicting findings on the attempt to identify blood biomarkers of preclinical AD. While Mapstone et al. reported the identification of a panel of 10 markers specific for diagnosis of AD in blood, Casanova et al. failed to confirm this panel in larger cohorts of patients using similar techniques. The former study used 28 samples from converters, the later utilized almost 200. The discrepancy in the findings is troubling since such controversy may undermine the application of an otherwise promising metabolomics approach for early diagnosis of AD.

    Both studies included well-characterized patient populations. Disease diagnosis in both studies was carefully conducted using established and consistent tests.  However, one obvious discrepancy in the study groups is an inconsistent ratio between men and women. Emerging data suggest the presence of sex-specific differences in the development of AD where metabolic pathways, especially those involved in energy production, are affected differentially. Therefore, it is surprising that neither of the studies discussed the potential effect of group composition on the results. Similarly, the studies failed to conduct a separate analysis based on sex. Another pitfall was the lack of detailed list of medications in study groups. Since metabolomics can also detect small molecules produced during drug metabolism, it is imperative to carefully control for that.

    Another important point, which was raised recently at meetings of the Metabolomics Society, is the transparency and availability of the raw metabolomics data for direct comparison. As was pointed out by Casanova et al., it is unclear how the 10-biomarker panel was determined by Mapstone et al. in the first place. In order for comparative analyses to be possible, publications presenting findings using systems biology approaches should include uploaded raw data along with the data analysis and interpretation. Without this level of transparency, it will be impossible to understand what particular parameters account for the lack of reproducibility.

    Additionally, it is very important to standardize sample collection, processing and storage. It has been estimated that up to 46 percent of laboratory errors come from pre-analytic processing (Becan-McBride, 1999). Factors related to blood collection devices (needle gauge, tube lubricants, tube walls) can impact blood marker levels. While Casanova et al. discussed some of the parameters, details on sample collection/storage are missing in the paper of Mapstone et al.

    It is clear that validation of metabolomics data in larger patient cohorts is essential for blood biomarker discovery. What is encouraging is that different groups agree on some of the pathways/metabolites that seem to play a role in disease development and progression.  With the availability of blood samples from AD patients from such initiatives as ADNI, it is now possible to design studies to validate results using different or similar platforms in the same subset of samples. This could help to develop strategies for better accuracy and reproducibility of metabolomics data.

    References:

    Becan-McBride K. Laboratory sampling. Does the process affect the outcome?. J Intraven Nurs. 1999 May-Jun;22(3):137-42. PubMed.

    View all comments by Eugenia (Jania) Trushina

  2. While we applaud the efforts of Casanova et al., we do not believe valid conclusions can be drawn from this attempt at replication. As scientists, we all understand that independent reproducibility of an experiment requires the use of the same experimental method. The use of pre-existing data and methods that are not commensurate with the original experiment do not represent a rigorous attempt at replication.

    While these results may be informative on the broader issue of lipid metabolism in preclinical AD, given multiple differences in the experimental design and approach and sample types, it does not represent a true replication of our work. We have prepared a response, which we are submitting to the Journal.

    View all comments by Mark Mapstone

  3. The search for blood biomarkers is obviously of high interest for Alzheimer’s disease diagnosis, and it is always disappointing to see promising results not being replicated. However, it is not a big surprise, and I encourage people working in this field to look back over the genetic research several years ago.

    Over a period of sixteen years, we were unable to validate a new genetic risk factor in AD. We had many problems related to the selection of the genes to be studied, and the quality and size of the populations. The proliferation of association studies (with small populations of fewer than 100 cases and controls) has resulted in finding numerous—too many—associations (false positive results). Inversely, low statistical power associated with these small association studies also leads us to reject potentially relevant genes due to false negative results. Consequently, discoveries have stagnated, potentially interesting results are barely noticeable, and the genetic approach, especially association studies, has lost a bit of credibility.

    Unfortunately, the classical epidemiological issues are exacerbated by the technical complexity associated with proteomics/metabolomics methodologies. That is why it is really important to circumvent at best the limitations that can be controlled without too many difficulties by well-designed studies, e.g., well-diagnosed cases, sex- and age- matched controls, and, above all, large numbers of samples. We also need adapted statistical approaches, e.g., to avoid overly complex statistical tools and/or over-stratification according to the number of samples.

    This is the basis to overcome the winner’s curse issue we encountered for many years in the genetic field. I hope that the biomarker field will learn from our errors and will not waste as much time as we did.

    View all comments by Jean-Charles Lambert

  4. I would like to thank Dr. Trushina for her comment, while also pointing out that our methods for sample collection, processing, and storage are outlined in the supplemental material of our manuscript. I direct the Alzforum audience's attention to the section labeled “Blood Collection, Shipment, and Specimen Processing Protocols.” This also details clinical and cognitive methods, tandem MS data, and statistical information supporting our methods.

    In regard to Dr. Trushina's point about medication effects, we agree completely. We attempted to minimize medication effects in our study by requiring medication withholding overnight before the blood draw. This is stated in the methods section of our manuscript. It is not clear if other studies utilize this step before drawing blood.

    We fully endorse transparency in biomarker studies. As stated in the methods section of our paper, we made the raw targeted metabolomic data from our study available upon publication. I am pasting the text from our manuscript below for the convenience of interested colleagues: "Accession codes. Lipodomics data were deposited in the European Bioinformatics Institute MetaboLights database with accession code MTBLS72.” We welcome replication attempts, but emphasize that these should be approached thoughtfully and rigorously.

    View all comments by Mark Mapstone

Make a Comment

To make a comment you must login or register.