Date of Award

6-1-2023

Degree Name

Doctor of Philosophy

Department

Evaluation

First Advisor

E. Brooks Applegate, Ph.D.

Second Advisor

Ya Zhang, Ph.D.

Third Advisor

Kevin Lee, Ph.D.

Keywords

Estimator, IRT, MLR, MML, ULSMV, WLSMV

Abstract

In conducting confirmatory factor analysis with ordered response items, the literature suggests that when the number of responses is five and item skewness (IS) is approximately normal, researchers can employ maximum likelihood with robust standard errors (MLR). However, MLR can yield biased factor loadings (FL) and FL standard errors (FLSE) when the variables are ordinal. Other estimators are available. Unweighted least squares and weighted least squares with adjusted mean and variance (ULSMV and WLSMV) are known as the estimators for CFA with ordinal variables (CFA-OV). Another estimator, marginal maximum likelihood (MML), is used in the item response theory (IRT), specifically the graded response model.

Despite the availability of these estimation methods, there is limited research comparing their performance in the context of ordered response items. None of the reviewed studies compared the performance of the estimation methods in one study or examined the impact of non-normality of latent variable scores (LVSs) on estimated FL, FLSE, and LVS. This study evaluated the performance of MLR, MML, ULSMV, and WLSMV in terms of relative bias and mean square error of FL (RB-FL and MSE-FL), relative bias and mean square error of FLSE (RB-FLSE and MSE-FLSE), percentage of population parameter coverage, convergence, and relative bias and mean square error of LVS (RB-LVS and MSE-LVS).

The models contained three dimensions and 21 items with five ordered responses. Data were generated using six sample types, two FL magnitudes, and two levels of IS, making 24 conditions. For each condition, 500 replicates were generated. Mixed-model analysis was used to analyze data with a repeated measure design. Pairwise multiple comparisons were conducted to determine which estimator performed significantly better. The Benjamini-Hochberg procedure with a 5% false discovery rate was used to identify significant results.

The study suggests that MML, ULSMV, and WLSMV were good estimators for FL when the data conditions involve a multivariate normal distribution for LVSs and skewed items (MVN-S) or a multivariate negative skew distribution for LVSs and skewed items (MNS-S). Under these conditions with high FL, ULSMV and WLSMV were superior to MLR and MML regarding FLSE. Under the MVN-S and MNS-S conditions, MML, ULSMV, and WLSMV had higher parameter coverage percentages compared to MLR, which had unacceptable parameter coverage percentages under almost all conditions. For LVS estimation, MLR was superior to the other estimators under the MNS-S conditions, while ULSMV and WLSMV were superior to MLR under the MVN-S conditions. All estimators showed no significant convergence issues.

Based on the data conditions employed in the study, it is advisable to avoid using MLR for conducting CFA with items containing five ordered responses. Instead, recommended alternative estimators include MML, ULSMV, and WLSMV.

Access Setting

Dissertation-Open Access

Download

Share

COinS