Most researchers agree that clinical dementia papers need reporting standards, especially when it comes to diagnostic tests. Say you are interested in this question: How well does a given biomarker predict Alzheimer’s disease (AD), or distinguish one form of dementia from another? Researchers wanting to do meta-analyses of the literature to answer questions such as these find it nearly impossible to determine the field’s knowledge because individual papers vary greatly in what and how they report. Enter STARDdem, which stands for “standards for reporting studies of diagnostic accuracy in dementia.” STARDdem is an initiative by the nonprofit Cochrane Dementia and Cognitive Improvement Group in Oxford, UK, to guide researchers in exactly how to report on diagnostic tests in the field of dementia. Compliance would help ensure that researchers control all the necessary factors and, in essence, make studies more meta-analyzable by the standards of evidence-based medicine. At the Alzheimer’s Association International Conference, held 14-19 July 2012 in Vancouver, Canada, Cochrane researchers introduced a STARDdem working document and invited the field to provide feedback during an open comment period lasting until August 31.
“There is a recognition that the literature at the moment needs a degree of standardization,” said Rupert McShane, University of Oxford, UK, who led the study. McShane heads the Cochrane dementia group, which reviews studies in the prevention, treatment, and management of cognitive impairment. “Our hope is that the STARDdem recommendations will be widely accepted and implemented.”
Modeled after the Consolidated Standards of Reporting Trials (CONSORT) project for randomized clinical trials reports (see Moher et al., 2001), the original STARD document aimed to standardize reporting in general studies of diagnostic accuracy (see Bossuyt et al., 2003). However, dementia researchers have been lax about putting its principles into practice, in part because generic standards do not apply to the unique reporting needs of the dementia field. “We are trying to make it easier for people in this field to use those criteria and understand what they mean,” said Leon Flicker, Western Australian Centre for Health & Ageing, Crawley. Flicker presented the document, now open for public comment, at the conference.
STARD is intended for studies that report the sensitivity or specificity of a given dementia test and hold that test up to the “gold standard” of AD diagnosis—usually autopsy confirmation or conversion from MCI to AD. The gold standards come some time after the diagnostic test, and this constitutes one aspect of the dementia field’s unique needs. STARDdem gives guidance on how to write up those studies, defining which aspects need to be reported. For instance, it recommends that studies report the reference standard and cite the study that validates it. It also details how to report missing data, divulge patient exclusion criteria, and include reasons for dropout, among other things. The guidelines intend to ensure that researchers report their findings thoroughly, but also that they think about and control all the important factors, said McShane. Without proper adherence to the guidelines, readers don’t always know how old a study population is or if results are from patients in a memory clinic versus the general population. “There’s rather a lot of scope in the literature for bias to creep in because the reporting suggestions of STARDdem aren't being applied,” said McShane. There are some rare exceptions; For example, a widely cited study comparing CSF diagnostic results across centers did use STARDdem criteria (Mattsson et al., 2010).
The STARDdem draft will stay open for public comment until 31 August 2012. After that, the Cochrane scientists will compile the comments and prepare a new draft, to be presented at the Clinical Trials Conference on Alzheimer’s Disease (CTAD) this October in Monte Carlo, Monaco. Soon after that, the authors hope to publish the guidelines in a scientific journal. It will then depend on journal editors and reviewers to adopt these measures and require compliance to assure widespread adoption by research groups across the field, said McShane.
Standardization is becoming especially important for biomarker studies. These are exploding in the literature and becoming increasingly important for AD diagnosis, said Henrik Zetterberg of Sahlgrenska University Hospital in Mölndal, Sweden. However, Zetterberg considered the first STARDdem draft to be insufficiently informed in terms of biomarker reporting. Zetterberg has critiqued the draft online and stressed that others should follow his example. “I think it is very important that specialists read and comment on the draft to make the first public version as good as possible,” he told Alzforum (see full comment below).
McShane’s team plans to conduct 15 literature reviews on potential diagnostic tests by September of 2013—about half on biomarker tests and half on cognitive tests for dementia. Craig Ritchie, Imperial College London, also in the Cochrane dementia group, presented a poster at the conference on the first of these reviews. He claimed that, based on the 13 studies in the literature that met his review criteria, cerebrospinal fluid (CSF) Aβ42 is neither sensitive nor specific enough to be confidently used as a diagnostic test for progression from mild cognitive impairment to AD. The poster’s conclusion met with considerable skepticism from Alzheimer’s scientists; however, AD scientists do agree that the literature for CSF tests needs to be better standardized.
“The field is at a place now where the next logical step is to bring biomarkers into the clinical realm,” said Anne Fagan, Washington University School of Medicine, St. Louis, Missouri. Before that happens, several committees will decide when and for whom such testing would be appropriate, and they will use published studies to determine those answers. While the STARDdem will not improve previous studies, it may help standardize future ones. For example, it would be helpful if authors provided simple definitions for "cognitively normal," or other qualifiers that are often neglected, to enhance comparability. “I think there has to be a common ground,” she told Alzforum.
Separately, another poster pointed to a perhaps even greater need for standardization, and that is in the still-emerging field of blood-based biomarkers. Unlike CSF, the plasma field has not begun to converge around a few markers that generate similar results among centers and studies; it is considered wide open for discovery. Andrew Watt, from the lab of Kevin Barnham at the University of Melbourne, Australia, reviewed 87 blood biomarker papers. He found no uniform way to collect and store the Aβ samples. Since the biomarker measures vary depending on the time of day samples were collected, speed of centrifugation, storage temperature, and other factors, the field really needs a standard way to collect and analyze samples and report its procedures so that studies become comparable, he told Alzforum. In toto, scientists called for a combination of quality control, assay, and sample handling standardization on the one hand, and reporting standards as developed by STARDdem on the other, to move dementia diagnosis to the next level.—Gwyneth Dickey Zakaib.
- Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet. 2001 Apr 14;357(9263):1191-4. PubMed.
- Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Lijmer JG, Moher D, Rennie D, de Vet HC, . Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. Clin Chem. 2003 Jan;49(1):1-6. PubMed.
- Mattsson N, Zetterberg H, Hansson O, Andreasen N, Parnetti L, Jonsson M, Herukka SK, van der Flier WM, Blankenstein MA, Ewers M, Rich K, Kaiser E, Verbeek M, Tsolaki M, Mulugeta E, Rosén E, Aarsland D, Visser PJ, Schröder J, Marcusson J, de Leon M, Hampel H, Scheltens P, Pirttilä T, Wallin A, Jönhagen ME, Minthon L, Winblad B, Blennow K. CSF biomarkers and incipient Alzheimer disease in patients with mild cognitive impairment. JAMA. 2009 Jul 22;302(4):385-93. PubMed.
- Westman E, Muehlboeck JS, Simmons A. Combining MRI and CSF measures for classification of Alzheimer's disease and prediction of mild cognitive impairment conversion. Neuroimage. 2012 Aug 1;62(1):229-38. PubMed.
- Lu Y, Riddell D, Hajos-Korcsok E, Bales K, Wood KM, Nolan CE, Robshaw AE, Zhang L, Leung L, Becker SL, Tseng E, Barricklow J, Miller EH, Osgood S, O'Neill BT, Brodney MA, Johnson DS, Pettersson M. Cerebrospinal Fluid Amyloid-β (Aβ) as an Effect Biomarker for Brain Aβ Lowering Verified by Quantitative Preclinical Analyses. J Pharmacol Exp Ther. 2012 Aug;342(2):366-75. PubMed.
- Doecke JD, Laws SM, Faux NG, Wilson W, Burnham SC, Lam CP, Mondal A, Bedo J, Bush AI, Brown B, De Ruyck K, Ellis KA, Fowler C, Gupta VB, Head R, Macaulay SL, Pertile K, Rowe CC, Rembach A, Rodrigues M, Rumble R, Szoeke C, Taddei K, Taddei T, Trounson B, Ames D, Masters CL, Martins RN, . Blood-Based Protein Biomarkers for Diagnosis of Alzheimer Disease. Arch Neurol. 2012 Jul 16;:1-8. PubMed.
- Soares HD, Potter WZ, Pickering E, Kuhn M, Immermann FW, Shera DM, Ferm M, Dean RA, Simon AJ, Swenson F, Siuciak JA, Kaplow J, Thambisetty M, Zagouras P, Koroshetz WJ, Wan HI, Trojanowski JQ, Shaw LM, . Plasma Biomarkers Associated With the Apolipoprotein E Genotype and Alzheimer Disease. Arch Neurol. 2012 Jul 16;:1-8. PubMed.
- CSF Testing for AD: Single-center Bliss, Multicenter Woe?
- Worldwide Quality Control of CSF Biomarkers—How Does it Work?
- When Is a C9ORF72 Repeat Expansion Not a C9ORF72 Repeat Expansion?
- Worldwide Quality Control Set to Tame Biomarker Variation
- Little by Little—Standardizing, Validating Those Biomarkers
- The Toxic Fold? Aβ Dodecamers, Tetramers Show Their Conformations
- Experimental α7 Agonist Meets Cognitive and Clinical Endpoints
- New Assays for Aβ Oligomers in CSF Claim Femtogram Sensitivity
- Q&A With Roche’s CNS Leader Luca Santarelli
- Wave of New BACE Inhibitors Heading to Phase 2
- Researchers Join to Draw Posterior Cortical Atrophy Out of Shadows
- CSF Markers: Goodbye, Research Use Only; Hello, Clinical
- Metrology, Certification Heavies Take CSF Tests Under Their Wings