Glaze, Ryan 1983- (2012-12). The Efficacy of Profile Matching as a Means of Controlling for the Effects of Response Distortion on Personality Measures. Doctoral Dissertation.
Thesis
Researchers and practitioners continue to be concerned about the magnitude, extent, and effects of response distortion when self-report personality measures are used in high-stakes testing. One method for mitigating response distortion that has not received much empirical attention is profile matching. Profile matching assesses the fit between test-takers' predictor profiles and a standard profile which represents an ideal or high performing employee's profile. Since profile matching assesses fit, it can capture nonlinear effects. Furthermore, high predictor scores are not necessarily associated with higher criterion scores. Test-takers who distort their responses by choosing inaccurately extreme response options may improve their chances of being hired if a linear model is used, but this approach is unlikely to be effective if a profile matching strategy is used as long as the standard profile is unknown to the test-takers. As such, the primary objective of the present study was to examine the extent to which profile matching may alleviate concerns about response distortion. A secondary objective was to examine characteristics of the standard profile that are associated with the efficacy of this approach. The present study compared the effects of response distortion on personality test scores, and their criterion-related validity in predicting tenure, based on a linear composite and a profile fit score. The present study used data from 996 applicants who completed a personality test in a high-stakes testing context. Missing data were imputed for a subset of applicants who did not complete two response distortion scales. As such, the results provided an initial proof-of-concept of the effectiveness of profile matching as a personnel decision-making strategy using a blend of real and simulated data. The results suggest that profile fit scores are less related to response distortion and display higher criterion-related validity than linear composite scores. However, the difference in criterion-related validity could not be attributed to response distortion. The results further suggest that the amount of scatter in the standard profile is negatively associated with the profile fit score's susceptibility to response distortion and positively related to criterion-related validity.
