Computerized decision support to optimally funnel patients through the diagnostic pathway for dementia

Abstract

Background

The increasing prevalence of dementia and the introduction of disease-modifying therapies (DMTs) highlight the need for efficient diagnostic pathways in memory clinics. We present a data-driven approach to efficiently guide stepwise diagnostic testing for three clinical scenarios: 1) syndrome diagnosis, 2) etiological diagnosis, and 3) eligibility for DMT.

Methods

We used data from two memory clinic cohorts (ADC, PredictND), including 504 patients with dementia (302 Alzheimer’s disease, 107 frontotemporal dementia, 35 vascular dementia, 60 dementia with Lewy bodies), 191 patients with mild cognitive impairment, and 188 cognitively normal controls (CN). Tests included digital cognitive screening (cCOG), neuropsychological and functional assessment (NP), MRI with automated quantification, and CSF biomarkers. Sequential testing followed a predetermined order, guided by diagnostic certainty. Diagnostic certainty was determined using a clinical decision support system (CDSS) that generates a disease state index (DSI, 0–1), indicating the probability of the syndrome diagnosis or underlying etiology. Diagnosis was confirmed if the DSI exceeded a predefined threshold based on sensitivity/specificity cutoffs relevant to each clinical scenario. Diagnostic accuracy and the need for additional testing were assessed at each step.

Results

Using cCOG as a prescreener for 1) syndrome diagnosis has the potential to accurately reduce the need for extensive NP (42%), resulting in syndrome diagnosis in all patients, with a diagnostic accuracy of 0.71, which was comparable to using NP alone. For 2) etiological diagnosis, stepwise testing resulted in an etiological diagnosis in 80% of patients with a diagnostic accuracy of 0.77, with MRI needed in 77%, and CSF in 37%. When 3) determining DMT eligibility, stepwise testing (100% cCOG, 83% NP, 75% MRI) selected 60% of the patients for confirmatory CSF testing and eventually identified 90% of the potentially eligible patients with AD dementia.

Conclusions

Different diagnostic pathways are accurate and efficient depending on the setting. As such, a data-driven tool holds promise for assisting clinicians in selecting tests of added value across different clinical contexts. This becomes especially important with DMT availability, where the need for more efficient diagnostic pathways is crucial to maintain the accessibility and affordability of dementia diagnoses.

Introduction

Due to the increasing prevalence of dementia and disease-modifying therapies (DMTs) becoming available for patients, there is a high demand for accurate diagnosis, whilst keeping the diagnostic work-up efficient [1]. Accurate diagnosis is important to organize appropriate care and aid in future planning [2, 3]. With the availability of DMTs, an accurate etiological diagnosis is a prerequisite for deciding whether or not to initiate treatment [4].

Timely diagnosis and early identification of patients eligible for treatment require biomarkers of amyloid pathology [5, 6]. Currently available biomarkers for Alzheimer’s disease (AD) pathology (i.e., CSF, PET) are invasive and expensive, and healthcare systems are not ready for the anticipated changes due to a lack of resources and available expertise [7,8,9]. In regular clinical settings, only a small fraction of potentially eligible patients receive a biomarker-based confirmatory diagnosis [10]. Other frequently used tests for dementia diagnosis include cognitive screening, neuropsychological test batteries and magnetic resonance imaging (MRI) [11].

Despite the ongoing transition to a biological rather than a clinical disease model, clinicians in memory clinics may prioritize adequate syndrome diagnosis for appropriate care organization in certain patient populations. Therefore, clinical contexts, such as a focus on syndrome diagnosis or etiological diagnosis, may vary among clinicians due to the nature of their patient population and are also influenced by patients’ preferences and needs [12]. The added value of a specific diagnostic test depends on the clinical context. For example, while CSF sampling may not be necessary for syndrome diagnosis, it is often necessary for determining etiology and crucial for establishing a patient's eligibility for DMT [5, 13]. The increasing demand for accurate diagnosis, along with an anticipated shortage of trained dementia specialists, requires alternative approaches to keep dementia diagnosis accessible and affordable [8, 14].

To reduce the workload of both primary care physicians and dementia specialists, digital cognitive tests that can be administered to patients at home are promising [15]. Another solution is the use of a computerized decision support system (CDSS), an application that analyzes and visualizes all data to help healthcare providers make decisions and enhance patient care [16]. Our previous research demonstrated that a CDSS in memory clinics increased clinicians' confidence in diagnosis [17]. Furthermore, a CDSS can aid clinicians in making diagnostic decisions and funnel patients through the optimal diagnostic pathway [7, 13]. Therefore, a CDSS could be a cost-effective solution to assist clinicians in choosing tests relevant to the clinical scenario at hand, without compromising diagnostic accuracy [18].

A European task force recently defined a patient-centered biomarker-based workflow for specialized outpatient services, to aid clinicians in selecting the right biomarker at the right time [19]. Although this workflow is a positive development, clinicians still need to decide which diagnostic tool to use and when to use it. We add to such an expert-based workflow by presenting a data-driven approach to stepwise diagnostic testing, with the objective of assessing whether the use of such an approach holds promise for effectively and efficiently guiding diagnostic decision-making for three common clinical scenarios: syndrome diagnosis, etiological diagnosis, and potential DMT eligibility.

Methods

Study design and clinical scenarios

Figure 1 illustrates the operationalization of the three clinical scenarios and their respective diagnostic tests. Note that blood/PET in light gray means that these tests may be considered (in the future) in the diagnostic trajectory but are not included in the current study. A data-driven CDSS [17] was used to determine the probability of the diagnosis by generating a disease state index probability score (DSI, 0–1) through the weighted combination of diagnostic test results [20], including digital cognitive screening, neuropsychological and functional assessment, MRI, and CSF biomarkers. A subsequent test is only performed if the diagnosis is inconclusive based on the probability for diagnosis in the previous step.

Fig. 1

Clinical scenarios and the respective diagnostic tests used for each scenario. Note: blood/PET in light grey mean that these tests could be considered (in the future) in these steps, but they are not included in the current study. 1) Syndrome diagnosis, considering a diagnosis of CN, MCI or dementia. 2) Etiological diagnosis, considering a diagnosis of AD dementia, FTD, VaD, or DLB. 3) DMT eligibility, considering whether a patient would be eligible for DMT, based on appropriate use criteria by Cummings [6]. Abbreviations: cCOG: computerized cognitive test, MMSE: Mini-mental state examination, RAVLT: Rey-auditory verbal learning task, TMT-A/B: trail making test A/B, GDS: geriatric depression scale, DAD: disability assessment for dementia, MRI: magnetic resonance imaging, cMTA: computerized medial temporal lobe atrophy, cGCA: computerized global cortical atrophy, WMH: white matter hyperintensities, APS: anterior–posterior score, CSF: cerebrospinal fluid, Aβ42: amyloid β1-42, t-tau: total tau, p-tau: phosphorylated-tau, DMT: disease-modifying therapy

The first scenario involves syndrome diagnosis, including a diagnosis of CN, MCI, or dementia, which is often used in a primary care or (local) memory clinic setting, where the focus is mostly on arranging care. Diagnostic tests included the (digital) cognitive screening test, cCOG [15], followed by neuropsychological assessment (NP), and MRI. For the second scenario, etiological diagnosis, encompassing differential diagnosis of AD, FTD, VaD, and DLB, diagnostic tests included cognitive assessment (cCOG, NP), MRI, and CSF analysis. In the third scenario, assessing potential eligibility for DMT according to the appropriate use criteria by Cummings [6], sequential diagnostic tests, including cCOG, NP, and MRI, were used to detect patients who should undergo CSF confirmation. Below, each diagnostic test battery is described in detail.

Study participants

A total of 883 study participants from two memory clinic cohorts were included. From the Amsterdam Dementia Cohort (ADC) [21], we included data from 758 participants collected between 2004 and 2022, and from the PredictND cohort [17], we included data from 125 participants collected between 2015 and 2016 in three European memory clinics. All participants received a standardized multidisciplinary diagnostic work-up, including medical history, physical and neurological examination, cognitive and functional assessment, laboratory tests, brain imaging and CSF measurements. Participants were included if both brain MRI and CSF results were available. In the PredictND cohort, patients received a 12-month follow-up to confirm the diagnosis.

We included patients who were diagnosed with dementia (n = 504) due to Alzheimer’s disease (AD, n = 302), frontotemporal dementia (FTD, n = 107), vascular dementia (VaD, n = 35), or dementia with Lewy bodies (DLB, n = 60), representing the most prevalent patient groups in clinical practice. Diagnoses were made according to the criteria for probable AD [22, 23], FTD [24], VaD [25], and DLB [26]. Additionally, we included individuals with mild cognitive impairment (MCI, n = 191) [27]. Individuals who did not meet the criteria for MCI or dementia were diagnosed with SCD [28] and served as the cognitively normal (CN) group (n = 188). All clinical diagnoses were made in tertiary memory clinics through a consensus meeting in the ADC and after a 12-month follow-up in the PredictND cohort. Table 1 presents the clinical characteristics of the patients included in this study. All patients provided written informed consent for their data to be used for research purposes.

Table 1 Demographic and clinical characteristics for the pooled ADC and PredictND cohort (n = 883) according to diagnosis

Digital cognitive screening

We aimed to apply a future-proof stepwise approach and thus incorporated a digital cognitive screening test, simulating a situation in which cognitive screening can start at home. As a part of the PredictND study, a subset of patients (n = 111, 13%) completed a digital cognitive screening test. cCOG is a web-based test tool that can accurately detect MCI and dementia [15]. cCOG has a completion time of 20 min and consists of a memory learning and recall task, and modified trail-making tests A and B. For the patients who had not performed cCOG, we simulated the results for each task from their neuropsychological equivalents.

Neuropsychological assessment

Data from the following neuropsychological tests were included: the Mini-Mental State Examination (MMSE) was performed to assess global cognitive function [29]. The Rey auditory verbal learning task and the Consortium to Establish a Registry for Alzheimer’s Disease word list memory test were used to assess learning and recall [30, 31], the Trail Making Test A and B (TMT-A/B) for mental processing speed and executive function [32], animal fluency for language and executive function [33], and forward and backward digit span for attention and executive functioning [34]. To assess neuropsychiatric symptoms, we used the Geriatric Depression Scale (GDS) [35] and the Neuropsychiatric Inventory (NPI, [36]). Missing data ranged from 177 (20%) for the NPI to 4 (0.5%) for the MMSE. To assess functional decline, the disability assessment for dementia (DAD) [37] was used, for which we missed data on n = 373 (42%).

MRI acquisition and automated biomarkers

MRI data were acquired using 1.5 or 3 T scanners. Three-dimensional T1-weighted gradient echo sequence and fast fluid-attenuated inversion recovery (FLAIR) sequence images were used. In this study, we used automated biomarkers obtained with the cMRI quantification tool as described in [38, 39]. The automated imaging biomarkers included: computed medial temporal lobe atrophy (cMTA): which was calculated from hippocampal volumes and inferior lateral ventricles from both hemispheres, and obtained using a multi-atlas segmentation algorithm [38, 40]. Computed global cortical atrophy (cGCA): grey matter concentration measured by voxel-based morphometry analysis [40]. The white matter hyperintensities (WMH) volume was automatically extracted from FLAIR images. The computed Fazekas (cFazekas) was estimated from these volumes combined with deep WMH [38, 40]. The anterior–posterior score (APS) is the ratio of cortical volumes in the frontal and temporal lobes relative to those in the parietal and occipital lobes, providing a specific measure for characterizing frontotemporal atrophy [39]. The AD similarity scale was derived by representing the region of interest (ROI) in the patient image as a linear combination of the corresponding ROIs from a database of previously diagnosed patients [13]. All imaging markers were corrected for head size, age, and sex.

Fluid biomarkers

The CSF biomarkers amyloid β1-42 (Aβ42), total tau (t-tau), and phosphorylated tau (p-tau) were measured locally with commercially available enzyme-linked immunosorbent assays (ELISA) (Innotest®, Fujirebio and Elecsys, Roche). Elecsys results were mapped to Innotest according to [41]. A drift-corrected cutoff of < 813 pg/ml was applied to determine Aβ₄₂ abnormalities [42], and > 375 pg/ml was applied for total tau abnormalities [43]. AD pathology was defined using the total-tau/Aβ₄₂ ratio with a cutoff of ≥ 0.46 [44].

Simulating stepwise testing for different clinical scenarios

Disease state index probability

To predict the diagnosis at each step, we used the disease state index (DSI) classifier. The DSI is a simple, data-driven, machine-learning method that compares different diagnostic (either syndrome or etiological) groups with each other (e.g., CN vs. dementia, or AD dementia vs. VaD) based on a training set of diagnosed patients. The DSI was previously validated in the European PredictND project [17]. For each diagnostic test, the patient data are compared to the distributions of the diagnostic groups in the training set, yielding a scalar index between zero and one that indicates the probability of a specific diagnosis [45]. Patients with low or high DSI values are typically more likely to be correctly classified than patients with intermediate DSI values. The DSI handles different types of variables, such as demographic information, cognitive test results, CSF biomarkers, and MRI data, and tolerates missing data. To reflect real-world practice, we included patients with missing data on neuropsychological tests [20]. The dataset was normalized according to age and sex. Tenfold cross-validation was performed with ten different test/train set divisions. Each time, 10% of the individuals were used as the test set and the remaining 90% were used as the training set. The test sets were separated, meaning that each subject appeared in exactly one test set and nine training sets in each round of cross-validation. The results over the ten cross-validations were combined and averaged to obtain the final result. The method is described in detail in the supplementary files Appendix A.

Probability cutoffs

In this study, the probability cutoffs varied depending on the clinical scenario. Cutoffs were determined visually by plotting sensitivity and specificity against probability cutoff values for each step in each scenario (see supplementary files: Supplementary Figs. 1–5). In clinical practice, there are established cutoffs for certain tests, such as amyloid positivity. However, decisions based on combined data from multiple sources and clinical impressions rely on the confidence of the clinician. Clinicians make decisions based on their confidence level and they may request additional testing or delay the diagnosis if they lack confidence. Confidence is subjective and depends on factors such as the clarity of the findings, the data available, the clinicians’ expertise, and their personality. Probability cutoffs aim to make the decision process more objective when interpreting all acquired data.

It is important to note that the CDSS does not consider clinical impressions, which are an essential part of the diagnostic process, so it should be considered supportive, as diagnosis is ultimately a clinical judgment. Additionally, there is a trade-off between accuracy and the number of tests. Acquiring more data can increase confidence and accuracy, but it also comes at a cost. Finally, decisions are always a compromise between sensitivity and specificity, i.e., how we value false positives and false negatives. For syndrome diagnosis, high sensitivity was considered important for minimizing the number of false-negative cases, while a balance between sensitivity and specificity was chosen for accurate etiological diagnosis.

Clinical scenarios

Figure 1 shows the diagnostic tests considered for each diagnostic scenario. Diagnostic strategies followed a predetermined order based on clinical guidelines and current practice. At each step, DSI values were calculated for the combination of diagnostic tests used. At each step, we assessed whether the DSI values exceeded the predetermined cutoff. If DSI values exceeded the cutoff, a diagnosis was made, and the diagnostic process was stopped. For patients for whom the diagnosis remained uncertain, i.e., for whom the DSI did not exceed the cutoff, additional tests were added to the diagnostic trajectory.

We used letter value plots [46] to depict the distribution of DSI values among different diagnostic groups. Letter value plots provide statistical insights in a visually intuitive manner. Each group is visually represented by boxes. The length of the box represents the interquartile range (IQR), which contains the middle 50% of the data in the group.

Scenario 1: syndrome diagnosis

For scenario 1, syndrome diagnosis, we included all patients. We used a pairwise classifier (‘CN’ vs. ‘dementia’). A DSI close to one increases the likelihood of a dementia diagnosis, and a DSI close to zero suggests CN. Patients with intermediate DSI values were considered to have MCI in the final step. We divided the scenario into two different pathways:

• Scenario 1A: Cognitive and functional assessment (NP)—The approach used in clinical practice today (base case). In choosing the cutoffs, we aimed to reflect decision making in clinical practice and used a DSI cutoff of 0.3 for CN and 0.7 for dementia patients. Patients with a DSI > 0.3 would require additional MRI testing.

  • Step 1: NP

    • ▪ DSI (CN/dem) < 0.3: CN; no next test

    • ▪ DSI (CN/dem) 0.3–0.7: MCI; next test, MRI

    • ▪ DSI (CN/dem) > 0.7: dementia; next test, MRI

• Scenario 1B: To enhance the current base case, in scenario 1B we used the digital cCOG as a prescreening step. We aimed for detecting clearly cognitively normal participants (DSI < 0.1, low probability) and patients with clear cognitive problems (DSI > 0.95, high probability), while the patients in between require additional NP testing (intermediate probability). For steps 2 and 3, the cutoff values were 0.3 for CN and 0.7 for dementia patients.

  • Step 1: cCOG

    • ▪ DSI (CN/dem) < 0.1: CN; no next test

    • ▪ DSI (CN/dem) 0.1–0.95: indeterminate result, next test, NP

    • ▪ DSI (CN/dem) > 0.95: dementia, next test, MRI

  • Step 2: cCOG + NP

    • ▪ DSI (CN/dem) < 0.3: CN; no next test

    • ▪ DSI (CN/dem) 0.3–0.7: MCI; next test, MRI

    • ▪ DSI (CN/dem) > 0.7: dementia; next test, MRI

Scenario 2: etiological diagnosis

For the purpose of etiological diagnosis, we excluded patients with MCI because the question of differential diagnosis only becomes relevant at the stage of dementia. We included 1) cognitive testing using cCOG and NP, 2) MRI, and 3) CSF. For step 1, we used the two-class DSI classifier (‘CN’ vs. ‘dementia’) with a low cutoff (DSI < 0.25) to prioritize sensitivity. Only after MRI can differential diagnosis be performed. Here, we used a multiclass DSI classifier, averaging the DSI value of each of the etiological groups against all other groups (i.e., for AD it is the average of: FTD vs. AD, DLB vs. AD, and VaD vs. AD). In this way, a DSI value (continuous value between zero and one) is provided for each patient and each diagnostic group (AD-FTD-VaD-DLB), estimating the probability of the specific diagnosis. The highest average DSI value defines the most likely class for the patient. For steps 2 and 3, we set the DSI cutoff at > 0.6 to balance between the number of patients who could be given a diagnosis and the accuracy of that diagnosis. As soon as the DSI ≥ 0.6 for any etiological diagnosis was reached the diagnostic process was concluded. If the DSI remained < 0.6, the data did not support a confident diagnosis. Additional data, such as CSF in step 3 or other tests, such as more detailed neuropsychological testing, electroencephalography (EEG), FDG-PET, or DaT-scan are required (not addressed in this paper).

• Step 1: cCOG & NP

  • ▪ DSI (CN/dem) ≤ 0.25: CN, no next test

  • ▪ DSI (CN/dem) > 0.25: dementia; next test, MRI

• Step 2: cCOG & NP + MRI

  • ▪ DSI (multiclass) > 0.6 for AD, FTD, VaD, DLB: diagnosis; no next test

  • ▪ DSI (multiclass) ≤ 0.6 for AD, FTD, VaD, DLB: next test: CSF

• Step 3: cCOG & NP + MRI + CSF

  • ▪ DSI (multiclass) > 0.6 for AD, FTD, VaD, DLB: diagnosis; no next test

  • ▪ DSI (multiclass) ≤ 0.6 for AD, FTD, VaD, DLB: next test or follow-up required (not addressed)

Scenario 3: DMT eligibility

In scenario 3, we included all patients and defined potential DMT eligibility according to Cummings' appropriate use recommendations [6] and the available data in the dataset. Potential eligibility was defined as a diagnosis of MCI/dementia due to AD, with MMSE ≥ 22, cFazekas < 2.5, and positive amyloid biomarkers. According to these criteria, our dataset consisted of a total of 230 potentially eligible patients. The goal of this scenario is to perform a minimum number of CSF tests while identifying a maximum number of eligible patients. We applied the following steps to select patients for confirmatory CSF testing: 1) cCOG, 2) NP, and 3) MRI. Unlike scenarios 1 and 2, the gold standard for diagnosis in this case is amyloid status based on CSF rather than clinical diagnosis. Step 4 involves CSF testing for confirmation of the biomarkers status. For steps 1 and 2, we used a pairwise classifier (CN vs. dementia). At step 1, the DSI threshold was set at 0.1 All patients with a DSI > 0.1 proceeded to step 2, involving NP. Here, the threshold increased to 0.3. All patients who were predicted to be ‘potentially eligible’ after the first two steps continued to step 3, MRI. For step 3, we used a different pairwise classifier (‘AD’, ‘other’), with a cutoff of 0.1, to select patients for CSF. A cutoff of < 0.1 points for a non-AD diagnosis and amyloid confirmation is not needed.

• Step 1: cCOG

  • ▪ DSI (CN/dem) ≤ 0.1: CN, not eligible, no next test

  • ▪ DSI (CN/dem) > 0.1: probable eligible; next test, NP

• Step 2: cCOG + NP

  • ▪ DSI (CN/dem) ≤ 0.3: CN, not eligible, no next test

  • ▪ DSI (CN/dem) > 0.3: probable eligible; next test, MRI

• Step 3: cCOG + NP + MRI

  • ▪ If DSI (AD/other) < 0.1 and DSI for any of the other etiologies > 0.85: non-AD, not eligible, no next test

  • ▪ DSI (AD/other) > 0.1: probable AD and DSI for any of the other etiologies < 0.8, probably eligible; next test, CSF

• (Step 4: Confirmatory CSF testing)

Statistical analyses

All statistical analyses were performed using R version 4.0.3. DSI analyses were performed using a Python implementation of DSI algorithm in Python 3.10.13.

The predicted diagnoses in scenarios 1 and 2 were compared to the clinical diagnoses as made in the respective memory clinics. In scenario 3, the predicted eligibility was compared to actual CSF results. For each scenario, we assessed the share of correct diagnoses (estimated by summing the number of true positive and true negative cases), sensitivity, specificity, and the need for additional testing at each step. The calculations for scenarios 1 and 3 were repeated using the subset of patients with real cCOG data.

Sub analyses were performed to compare the groups of patients with and without diagnosis using analysis of variance (ANOVA) and chi‐squared tests to evaluate differences between the groups in diagnosis, demographics, or clinical characteristics.

Results

Syndrome diagnosis

Scenario 1 was divided into two sub scenarios that included all patients (N = 883). Scenario 1A represents the base case, where the syndrome diagnosis is based on NP. In scenario 1B, the digital cognitive test cCOG was added as a prescreener. A visual representation of both scenarios is shown in Fig. 2.

Fig. 2

Visual representation of scenario 1, syndrome diagnosis, for scenario 1A) syndrome diagnosis using NP, and scenario 1B) stepwise syndrome diagnosis, making use of letter-value plots. NOTE: The figure includes letter-value plots for DSI scores. The length of the first box represents the interquartile range (IQR), which contains the middle 50% of the data in the group, the second box (E) 12.5% at each end, the third box covers (D) 6.25% on each end, and the fourth box (C) 3.13%. The longer the box, the greater the variability. The percentages indicate the proportion of the total population that requires additional testing. Red dashed lines depict the DSI cutoff-vales. MCI: mild cognitive impairment, MRI: magnetic resonance imaging

Scenario 1A. syndrome diagnosis using NP

When applying the cutoff of < 0.3 for CN and > 0.7 for dementia, 203 participants were classified as CN, 131 as MCI, and n = 549 as having dementia. Using this scenario led to a diagnostic accuracy of 0.74 and would require MRI in 77% of patients. Table 2 shows the corresponding confusion matrix.

Table 2 Confusion matrices for syndrome diagnosis scenarios 1A and 1B

Scenario 1B. cCOG + NP + MRI

In scenario 1B, diagnostic tests were added sequentially to reduce NP testing to only when digital screening was insufficient and expanded with MRI when necessary. Table 2 shows the confusion matrices for the consecutive steps. In the first step, cognitive screening with cCOG, the threshold was low (< 0.1) for CN and high (> 0.95) for dementia. In this step, 152 participants were predicted to be CNs, and 355 were predicted to have dementia. Additional NP testing was added for the 376 participants with indeterminate DSI values (43%). The combination of the two steps resulted in a diagnostic accuracy of 0.71 (Table 3) and required MRI in 76% (n = 667). Supplementary Table 1 shows the results for the subset of patients for whom real cCOG data were available.

Table 3 Confusion matrix for patients receiving diagnosis after stepwise application of cCOG, cognitive testing, MRI and CSF

Etiological diagnosis

For the second scenario, we included CN (n = 188) and dementia patients (302 AD, 107 FTD, 35 VaD, 60 DLB). We started with cognitive testing to distinguish between controls and dementia patients. Subsequently, we sequentially added MRI and CSF to determine etiology (Fig. 3).

Fig. 3

Visual representation of scenario 2, stepwise etiological syndrome diagnosis, making use of letter-value plots. NOTE: The figure includes letter-value plots for DSI scores. The length of the first box represents the interquartile range (IQR), which contains the middle 50% of the data in the group, the second box (E) 12.5% at each end, the third box covers (D) 6.25% on each end, and the fourth box (C) 3.13%. The longer the box, the greater the variability. The percentages indicate the proportion of the total population that requires additional testing. Red dashed lines depict the DSI cutoff-vales. MCI: mild cognitive impairment, MRI: magnetic resonance imaging

Based on initial cognitive and functional testing, 162 individuals were identified as CN and 530 were identified as having dementia, with a sensitivity of 97% and specificity of 75% (DSI cutoff < 0.25; supplementary Table 2). Subsequently, 77% of the participants proceeded to the next step, which involved MRI. Following MRI, DSIs were calculated for each of the diagnostic groups (AD, FTD, VaD, DLB) for each individual patient (Supplementary Fig. 6). A DSI > 0.6 for any of the etiological diagnosis confirmed that diagnosis. Of the 531 patients who underwent MRI, 275 patients (52%) received a diagnosis (Supplementary Table 3), and the remaining 256 patients proceeded to the final step of CSF examination. Of the 256 patients who underwent CSF examination (Supplementary Fig. 6), 120 (47%) were ultimately diagnosed with sufficient confidence (DSI > 0.6) (Supplementary Table 4). The combination of all steps led to diagnosis in a total of 556 participants (80%) with an accuracy of 0.77, as shown in Table 3A.

After CSF testing, the diagnosis remained uncertain for 20% of the patients, requiring additional diagnostic testing or follow-up. Of these patients, 16% were CN, and 84% had dementia. The demographic and clinical characteristics of these patients are shown in Table 4. Compared to patients who received a diagnosis, patients without a diagnosis after the stepwise approach were more likely to have a clinical diagnosis of FTD or DLB, were older and had worse neuropsychological and neuropsychiatric assessment outcomes. Furthermore, patients without a diagnosis had a greater vascular burden on MRI (cFazekas, p = 0.032), suggesting the presence of mixed etiologies in these patients.

Table 4 Demographic and clinical characteristics for patients receiving a diagnosis (n = 556) or no diagnosis (n = 136) after stepwise cognitive testing, MRI, and CSF

DMT eligibility

Finally, we studied how a CDSS can efficiently support the determination of eligibility for DMT. We included patients with complete data on amyloid status, MMSE, and cFazekas score (n = 187 CN, n = 190 MCI, n = 500 dementia (all causes)). To determine potential eligibility for DMT, it is important to first detect the clinical stage, followed by the etiology. In contrast to scenarios 1 and 2, where clinical diagnosis was considered the gold standard, scenario 3 used amyloid status as determined in CSF as the gold standard. Therefore, CSF testing is not a separate step as in other scenarios, but rather is performed as a confirmatory test in all potentially eligible patients. The goal of this scenario is to perform a minimum number of CSF tests, while identifying a maximum number of eligible patients. Figure 4 shows a visual representation of the stepwise eligibility testing.

Fig. 4

Visual representation scenario 3, stepwise eligibility testing, making use of letter-value plots. NOTE: The figure includes letter-value plots for DSI scores. The length of the first box represents the interquartile range (IQR), which contains the middle 50% of the data in the group, the second box (E) 12.5% at each end, the third box covers (D) 6.25% on each end, and the fourth box (C) 3.13%. The longer the box, the greater the variability. Red dashed lines depict the DSI cutoff-vales. Abbreviations: AD: Alzheimer’s disease, MCI: mild cognitive impairment, MRI: magnetic resonance imaging, CSF: cerebrospinal fluid, PET: positron emission tomography

The first steps are used to identify MCI and dementia, since CN are not eligible for DMT. Based on digital cognitive screening, 152 patients were identified as non-eligible (DSI cutoff < 0.1). The remaining 725 patients (83%) continued to NP, after which 63 were identified as non-eligible (DSI cutoff < 0.3). Based on the cognitive assessment, 662 patients (75%) were selected for MRI as the next step to detect potential eligibility. After MRI, all patients with a DSI for AD > 0.1 and < 0.8 for any of the other etiologies (n = 528, 60%) were considered potentially eligible and were thus selected for confirmatory CSF testing. Of those patients, 191 met the eligibility criteria (MCI/dementia due to AD, MMSE ≥ 22, cFazekas < 2.5, and positive amyloid biomarkers) whereas 337 did not. Supplementary Table 5 shows the confusion matrix for each step for all patients (A) and the subset with real cCOG data (B), Table 5 shows the confusion matrix of all three steps combined. Stepwise diagnostic testing correctly selected 191/230 potentially eligible patients (true positive, 83%) for confirmatory CSF. Furthermore, 349 patients were identified as non-eligible, 310 of whom did not meet the eligibility criteria (true negative). The distribution of AD diagnoses of the 39 false negative patients is showed in Table 6, indicating that the sensitivity for detecting eligible patients with AD dementia is 90%. Half of the cases were stable MCI patients, who showed no disease progression in the 12-month follow up of the PredictND study.

Table 5 Confusion matrix for patients classified as potentially eligible after stepwise application of cCOG, cognitive testing and MRI

Table 6 Distribution of AD diagnoses of patients classified as eligible in the stepwise approach

Clinical application

To illustrate the practical application of the stepwise testing process using the CDSS, we present an example of the stepwise approach in clinical practice, using a fictive patient (Fig. 5). Patient A is a 69-year-old female who presented to the memory clinic with memory complaints. She underwent cCOG, and based on her cCOG results (DSI = 0.87), she was classified as MCI with an indeterminate DSI value, and it was recommended to continue to the next step to confirm the syndrome diagnosis. As such, cognitive and functional assessments were performed, after which the DSI increased to 0.94, indicating a greater likelihood of dementia. If the main focus is on syndrome diagnosis, the clinician may choose to conclude the diagnostic process. Alternatively, the clinician can continue to investigate the etiology, starting with the addition of MRI. The MRI results revealed the highest probability for AD as the underlying etiology (0.55). Since this value was below the predefined cutoff of 0.6, the clinician might decide to proceed with CSF testing. When CSF was subsequently is added, the probability of AD increased to 0.71, which concurrently reduced the likelihood of the other diseases. Note that this patient would also be selected for CSF testing to assess eligibility for DMT given that the DSI cutoff for AD after MRI exceeded the defined cutoff (> 0.1), and none of the other diagnoses had a DSI value of > 0.8.

Fig. 5

Example of the stepwise approach in clinical practice. Abbreviations: DSI: disease state index, cCOG: computerized cognitive screening, NP: neuropsychological and functional assessment, CN: cognitively normal, MCI: mild cognitive impairment, MMSE: mini-mental state examination, CSF: cerebrospinal fluid, MRI: magnetic resonance imaging, AD: Alzheimer’s disease, DLB: dementia with Lewy bodies, FTD: frontotemporal dementia, VaD: vascular dementia

Discussion

In an era where disease-modifying therapies are becoming available for the first time, timely and accurate diagnosis of dementia is more important than ever. A data-driven tool can be beneficial for assisting clinicians in determining which tests add value in different clinical contexts. In this study, we applied a data-driven stepwise testing approach using a CDSS in three clinical scenarios to assess its potential to guide diagnostic decision-making. The stepwise testing approach results indicated that different scenarios require different quantities of diagnostic tests and thus reduce the number of tests required while maintaining diagnostic accuracy. Such an approach holds promise for ensuring that the diagnosis of dementia is accessible and affordable in the future. For the purpose of this proof-of-principle study, we divided the scenarios into three separate clinical scenarios. In real-world practice, the three could be combined into one harmonized workflow, as illustrated by the hypothetical patient case.

To address the challenges healthcare systems face with increasing patient volumes due to the growing prevalence of dementia and the introduction of DMTs and the significant national and regional differences in the accessibility and availability of diagnostic services it is crucial to improve diagnostic strategies [8, 47,48,49]. An ideal diagnostic pathway is tailored to the needs of both individual patients and clinicians, while being efficient at the same time. In this study, we assessed efficient pathways for three clinical scenarios based on different goals that clinicians have for which they deploy diagnostic testing. These different situations define an important role for a CDSS that can be flexible in the way it recommends ancillary testing to assist with the interpretation of complex, multimodal diagnostic data to provide accurate diagnostic predictions derived from reference populations. Although the diagnostic accuracies in this study were already decent, the CDSS could be further extended to increase the accuracy even further by incorporating other data, such as the extended cCOG version for DLB [50], and amyloid PET [13]. Currently, it is being extended by the development of FDG-PET and DaT-scan modules.

Timely diagnosis and early identification of patients eligible for DMT require biomarkers of amyloid pathology, but in regular clinical settings, only a small fraction of potentially eligible patients receive a biomarker-based confirmatory diagnosis [10]. Blood-based biomarkers (BBB) are rapidly evolving and have the potential to provide accessible evaluation of AD pathology [51, 52]. It is conceivable that for scenario 2 – differential diagnosis – BBBs may quickly replace CSF biomarkers [51]. In [53], it was suggested that CSF or amyloid PET confirmation is needed only for the intermediate probability group based on BBB results. The same concept of the high/ intermediate/low probability was followed for cognitive testing in this study and illustrated in Fig. 1. However, as BBBs are not yet widely in clinical use, it is still necessary to conduct CSF or amyloid-PET investigations when determining DMT eligibility. Therefore, it is highly relevant to find efficient ways to conduct CSF assessments for the right patient at the right time. We demonstrated that a stepwise approach can reduce the number of diagnostic tests while maintaining diagnostic accuracy. For example, our approach required only 61% of all MCI and dementia patients to undergo amyloid testing to identify potentially eligible patients, while still identifying 84% of the truly eligible patients. The 16% of patients who were missed were probably missed due to mixed pathologies. Nonetheless, one should note that our approach did not consider some of the factors that could render a patient ineligible, such as medication use or imaging features such as microbleeds [5, 6]. Improving the underlying models, also with respect to the other eligibility criteria, is currently ongoing in the European PROMINENT study [54].

The early and accurate identification of individuals who may develop dementia due to AD, i.e., ‘preclinical disease’ [55], and who are eligible for DMT requires the implementation of broader large-scale screening measures and cost-effective detection and diagnostic tools [6, 48]. In this regard, the development of digital cognitive test tools holds considerable promise [56]. In this study, we showed that (digital) cognitive screening is a powerful tool for identifying cognitively normal individuals who are not eligible for treatment. This can importantly support primary care in their role as a ‘gatekeeper’, ensuring that only those who need it are referred to memory clinics. However, primary care physicians often have limited time and resources available [9]. Therefore, using digital cognitive tests, such as cCOG, as a first step, at home can alleviate the burden on general practitioners [15]. Although cCOG has shown high accuracy in detecting cognitively healthy participants and dementia patients its ability to detect earliest subtle cognitive changes remains to be elucidated [15, 50].

Our data-driven approach is limited in that it does not consider the preferences and wishes of patients and care partners, or clinical impressions of clinicians. However, the results of our study indicate that using a stepwise approach has the potential to complement patient-centered biomarker-based workflows for specialist outpatient services as recently defined by a European task force [19], by offering data-driven and visually supported diagnostic probabilities for clinicians. Furthermore, patients and their care partners have their own needs and preferences regarding diagnostic testing, which are often not addressed during the consultation [12]. Stepwise diagnostic testing, along with the graphical counterpart, which makes interpretation of results to clinicians more transparent, and the option to visualize test results in easy-to-understand patient report [57], can guide clinicians in explaining diagnostic tests or test results, enhancing patient comprehension and contributing to shared decision-making.

Another often mentioned and important limitation of data-driven CDSSs is that they require multiple pieces of data that are not readily available to the clinician, or that the models are too complex, which limits their clinical footprint [16]. The CDSS should be intuitive (i.e., not a complex black box), time-efficient, and assist rather than replace the clinician [58]. The latter also means that if the tool’s result conflicts with the clinician’s opinion, the clinician would always have the final say and could choose to disregard the tool's outcome. Although the CDSS that we used in this study was developed with the aim of meeting these criteria and has been validated in clinical practice, it is not yet widely used in daily practice [17]. The implementation of new technologies in existing clinical workflows is often hindered by the need for integration with electronic patient files, a lack of experience with these tools and high workloads hindering adequate learning of how to use a tool. In other words, tools need to be tailored to real-world practices and meet the needs and preferences of clinicians. Follow-up research focusing on the implementation of the CDSS in daily practice should address these main requirements.

Strengths and limitations

A major strength of our study is the use of data from two different memory clinic cohorts. All patients came to the memory clinic seeking medical help. The data thus reflect real-life clinical patients. In each clinic, patients were diagnosed using a comprehensive clinical workup, and we were able to use measures that overlapped. Since ground-truth diagnosis was obtained from a clinical diagnosis, not from a neuropathological confirmation, the true accuracy cannot be determined. However, the clinical workup was comprehensive; the material section describes only the data that were used in this study, but more comprehensive data were available for clinical diagnosis, and all clinical diagnoses were made in tertiary memory clinics through a consensus meeting in the ADC and after 12 months of follow-up in the PredictND cohort. In other words, the ground truth represents the current state of the art in clinical diagnosis. Another limitation is that we categorized patients as having a single pathology, while on average 20–40% of patients have multiple pathologies underlying their dementia diagnosis. The fact that we were able to correctly classify 80% of patients in the etiology scenario may be attributed to mixed pathology.

Finally, a limitation of this study is the choice of cutoffs applied in the stepwise approach. The cutoff values were selected ad hoc with the aim of, for example, emphasizing sensitivity over specificity in a certain scenario. Although other strategies could have been applied to optimize the use of cutoffs, we did not employ these in the current study for the purpose of readability and clinical applicability. Future research should address optimization strategies for optimal cutoff values. The stepwise approach shows promise in reducing the number of diagnostic tests performed. However, we did not consider any costs in the present study. To pursue the clinical application of a data-driven approach in real-world practice, a cost-efficiency study is warranted.

Conclusion

The anticipated challenges in real-world dementia practices call for improved diagnostic strategies to establish a tailored diagnostic pathway that meets the specific needs of individual patients and clinicians while keeping dementia diagnosis accessible and affordable. In this study we demonstrated the promising value of a stepwise, data-driven approach for reducing the number of diagnostic tests required while maintaining diagnostic accuracy.