Conditional confidence intervals for the true classification error
Additional Document Info
In this paper, we consider the joint distribution of the true error and the estimated error, assuming a random feature-label distribution. From it, we derive the conditional expectation of the true error and the 95% upper confidence bound for the true error given the estimated error. Numerous classification and estimation rules are considered across a number of models. Although specific results depend on the classification rule, error-estimation rule, and model, some general trends are seen: (1) the conditional expected true error is larger (smaller) than the estimated error for small (large) estimated errors; and (2) the confidence bounds tend to be well above the estimated error for low error estimates, becoming much less so for large estimates. 2006 IEEE.
name of conference
2006 IEEE International Workshop on Genomic Signal Processing and Statistics