What are the Conditions Associated with Subscore Added Value Noninvariance? Implications for Improving Subscore Interpretation Fairness

What are the Conditions Associated with Subscore Added Value Noninvariance? Implications for Improving Subscore Interpretation Fairness

Joseph A. Rios and Alejandra A. Miranda

Journal paper

Educational Measurement: Issues and Practice

2020

Subscore added value analyses assume invariance across test taking populations; however, this assumption may be untenable in practice as differential subdomain relationships may be present among subgroups. The purpose of this simulation study was to understand the conditions associated with subscore added value noninvariance when manipulating: (a) subdomain test length, (b) differences in subgroup mean ability, and (c) subgroup differences in intersubdomain correlations. Results demonstrated that subscore added value was noninvariant for 24–100% of replications (depending on subdomain test length) when the subgroup difference in intersubdomain correlation was equal to.30. To examine if this condition was met in practice, applied invariance analyses of three operational testing programs were conducted. Across these datasets, noninvariant subscore added value was present for some subdomains across sex and ethnic subgroups. Overall, these results indicate that subscore added value noninvariance is largely driven by differential intersubdomain correlations among subgroups, which may be present in some operational testing programs.