14.4 Internal consistency

The problems found when trying to generate parallel forms are avoided when using internal consistency methods. The basic assumption is to consider the observed variables of a test (i.e., items) as the sample of observations drawn from a specific domain. A way to estimate the internal consistency of the test is to inspect the consistency of respondents in different parts of the test. To do so, we can divide the test in two halves (Split half tests) or use the information of all items (e.g., Cronbach's alpha).

14.4.1 Split half tests

In this method, all respondents complete the test. Then, the researcher splits the observed variables (e.g., items) in two halves. This procedure requires parallelism of both halves. The goal is to correlate the group of scores from both halves (\(\rho_{12}\)). The famous Spearman-Brown correction formula for two halves is used to estimate the reliability coefficient.

\[\begin{aligned} \rho_{xx'} = \frac{\ 2 \rho_{12}}{1 + \rho_{12}} \end{aligned}\]

A more liberal approach is used when we only keep the assumption of true scores' equal variances. These methods are called Tau Equivalent reliability methods. These estimations (e.g., Rulon formula) provide smaller values than computations using Spearman-Brown's equation.

\[\begin{aligned} \rho_{xx'} = 2\Bigg( 1 \ - \frac{ \sigma^{2}_{x_{1}} \ + \sigma^{2}_{x_{2}}} {\sigma^{2}_{x}}\Bigg) \end{aligned}\]

The problem with the Split Half approach is that there are different ways to generate two halves, leading to different estimates of the reliability coefficient. To overcome this problem, we will assume that all items generate scores in tests of the same length (n = 1). These reliability estimates are based on the covariance of the items.

14.4.2 Item covariance tests

14.4.2.1 Cronbach's Alpha (\(\alpha\))

The most famous reliability estimator based on the covariance of the items of a scale is Cronbach's alpha (\(\alpha\)). Alpha is the mean of all possible split half reliabilities but corrected for the test length (Revelle & Zinbarg, 2009). In Cronbach's alpha equation, \(\ n\) is the number of items, \(\sigma^{2}_{i}\) is the variance of the items, and \(\sigma^{2}_{x}\) is the variance of the total test scores.

\[\begin{aligned} \alpha = \frac{\ n}{\ n \ - \ 1} \Bigg( 1 \ - \frac{\displaystyle\sum_{i=1}^{n} \sigma^{2}_{i}} {\sigma^{2}_{x}} \Bigg) \end{aligned}\]

Cronbach's alpha (\(\alpha\)): Properties and limitations

    • Alpha usually yields smaller values than Spearman-Brown's reliability coefficient. For that reason, alpha is considered Spearman-Brown coefficient's lower bound
    • If the elements are parallel, alpha (\(\alpha\)) yields the same value as Spearman-Brown's equation
    • If we consider two parts of the same test, alpha is usually equivalent to the Rulon formula for Tau Equivalent tests
    • The estimates of Cronbach's alpha and Spearman-Brown's reliability coefficients range from 0 to 1
    • Alpha is problematic if we find negative values
    • Alpha is problematic when the factorial structure of a test is not normal
    • Raw alpha is sensitive to differences in the item variances
    • Do not estimate an overall Cronbach's alpha coefficient for a psychological test that is not unidimensional

14.4.2.2 Other reliability coefficients

In addition to Cronbach's alpha (\(\alpha\)), other coefficients of internal consistency have been proposed. For example, the Kuder-Richardson Formula 20 (KR-20) is suitable for dichotomous items. Another reliability estimation was proposed by Hoyt to compute alpha using a Repeated Measures Analysis of Variance (ANOVA) approximation, testing the main effects and interaction of respondents' scores and items' scores.

Guttman also proposed six different coefficients. For example, Guttman's Lambda 3 (\(\lambda_{3}\)) provides the same estimation as Cronbach's alpha. Likewise, Guttman's Lambda 4 (\(\lambda_{4}\)) provides the same estimation as Spearman-Brown's reliability coefficient. Guttman's Lambda 6 (\(\lambda_{6}\))—although is a better reliability estimator than Guttman's Lambdas 3 (\(\lambda_{3}\)) and 4 (\(\lambda_{4}\))—is sensitive to the number of items and the average intercorrelation of the items of the test. Last, Revelle's Beta coefficient (\(\beta\)) provides the reliability estimation of the worst split half of a given scale.

MacDonald's Omega coefficients (\(\omega_{h}\), \(\omega_{t}\)) are an appropriate approximation to the reliability of a test as they are based on hierarchical factor analysis. Omega estimates the reliability of general (\(\omega_{h}\)) and total (\(\omega_{t}\)) factor saturation. It is recommended to run an EFA before using the function omega() from the package psych .

Omega hierarchical (\(\omega_{h}\)) estimates the general factor saturation of a test. We can estimate this coefficient by running an Exploratory Factor Analysis (EFA) using the Schmid-Leiman transformation and an oblique rotation method. On the other hand, Omega total (\(\omega_{t}\)) estimates the total reliability of the test and it is similar to Guttman's \(\lambda_{6}\) (G6).

It is important to note that Omega hierarchical (\(\omega_{h}\)) is based upon the sum of the squared loadings on the general factor, whereas Omega total (\(\omega_{t}\)) is based upon the sum of the squared loadings on all the factors. Estimations of reliability using Omega total (\(\omega_{t}\)) produce similar results than estimations of reliability using Guttman's \(\lambda_{6}\) (G6).