Abstract 2372: Error Rate Estimates for the modified Rankin Score Shift Analysis Using Information Theory Modeling
Academic Article
Overview
Research
Identity
Additional Document Info
Other
View All
Overview
abstract
Background: There remains debate as to the most appropriate functional outcome measurement in acute stroke trials. One approach is to use the whole range of ordinal/categorical variables of the modified Rankin Score (mRS), often termed Shift Analysis. Here we consider one of its main theoretical bases, Shannons information theory. Noise in Shannons model assumes the signal transmitted is uniform. However, van Swieten (Stroke 1988), and others show that the errors (noise), in the mRS are predominantly in the mid-range values of 2-4. In order to test the applicability of Shannons concept to shift analysis, we used entropy models to quantify errors with the primary noise considered in this range. Methods: We identified 16 randomized stroke trials that reported the full range of mRS. A custom Matlab program was written to simulate Shannons model both when the whole range of mRS was used and with commonly used dichotomous cut-points of mRS0-1 or mRS0-2 and mRS 6. Each stroke trials mRS distribution was multiplied with the confusion matrix from Table 1 of van Swietens mRS inter-rater variability. This matrix is equivalent to a band-limited noise in the signal-processing sense. The output of this step produced a misclassification/error percentage for each mRS category both from the full range of mRS or with specified cut-points. Error rates were compared by ANOVA. Results: If the full range of mRS is considered, error rates ranged from 22 to 32% (Mean SD: 26% 2.8). If mRS 0-1 and mRS 6 were cut-points, then error ranged from 2.6 to 13.4% (7.5% 2.5). Error rates for mRS 0-2 and mRS 6 were 6.9 to 14.3% (8.8% 1.8; overall p<0.001). Conclusion: We show from an information theory perspective that when the uncertainty in the midrange mRS is considered, there are relatively high error rates if the full range is used. Lower error rates were found with specified cut off dichotomization. Federov et al (2009) contend there is loss of information with dichotomization. However, this view does not consider the non-random distribution of noise and it assumes a Poisson distribution of mRS values. As seen in the left figure , none of the 16 trials had a Poisson distribution (compare to right panel modeling different Poisson distributions). We conclude that while use of entire mRS range is conceptually appealing, even if these were Poisson distributions, any potential gain of information with considering the complete scale is accompanied by a decrease in fidelity, hence more uncertainty regarding the validity of the outcome compared to dichotomization.
