Measuring the combined risk to young children’s cognitive development: 
An alternative to cumulative indices

Abstract
In studies of child development, the combined effect of multiple risks acting in unison has been represented in a variety of ways.  This investigation builds upon this preceding work and presents a new procedure for capturing the combined effect of multiple risks.  A representative sample of 2,899 British children had their cognitive development measured at 36 and 58 months of age along with ten potential risks during this period of development. Comparing a cumulative index of these risks against the previously undocumented alternative of confirmatory factor analysis using formative measurement, this study found differences favoring the factor analysis.  The factor analysis procedure demonstrated greater predictive power of children’s cognitive development while it systematically tested two of the assumptions implicit in cumulative risk indices.  
Keywords: cognitive development, cumulative risk, confirmatory factor analysis.  

Measuring the combined risk to young children’s cognitive development: 
An alternative to cumulative indices

Introduction
When examining the development of young children, the effects of social, environmental, biological, and genetic factors are understood not to act separately, but in combination with one another (Rutter et al., 1997).  Indeed, these are conceptions that pertain to factors both deleterious, (Luthar, Cicchetti, & Becker, 2000) as well as ‘promoting’ and ‘protecting’ (Rutter, 1987).  Furthermore, both these sets of concepts originate from an ecological perspective that emphasizes the context in which children develop and this has been the basis of many longitudinal studies of human development over the past 30 years.  Such investigations have included: the Isle of Wight Studies (Rutter, 1976), the Rochester Longitudinal Study (RLS; Sameroff, Seifer, Baldwin, & Baldwin, 1993), the Kauai longitudinal study (Werner, 1993, 1996), Project Competence (Garmezy, Masten, & Tellegen, 1984) and research by the National Institute of Child Health and Human Development (NICHD; Fearon & Belsky, 2004).  Common across these investigations is the concept of a combined risk that affects children’s development, and the strategy for the measurement of this, through the construction of a cumulative risk index.  In addition, the authors of these studies have also stressed the developmental importance of risks impacting the cognitive skills of young children (e.g. Rutter, 1979) even for development occurring over a period of years (Sameroff, Seifer, Baldwin, & Baldwin, 1993).  

Cumulative Risk Indices
Based on the original works of Rutter (1979), and Sameroff, Seifer, Barocas, Zax, and Greenspan (1987), cumulative risk indices are created through the summation of dichotomized risk factors that are theorized to affect ontogenesis, a procedure outlined by Evans (2003).  Sameroff, Gutman, and Peck (2003) describe the origin of these models: epidemiological research investigating heart disease revealed that it was combinations of risk factors, rather than those specific and individual, that contributed significantly to the occurrence of this disease (e.g. Lloyd-Jones et al., 2004).  These ideas have since been extended to research investigating child development so that children can be said to encounter a combined risk that originates from the amount of risks in their lives rather than from any individual risk or type of risk (Sameroff & Cole, 2003).  As an example, Sameroff et al. (1987) found that from a set of ten environmental variables including maternal mental health, no individual variable was a significant determinant of low cognitive development when examined independently from the others.  

The emphasis on overall volume rather than individual relationships is based upon the notion that there are transactions between risks (Schoon, 2006), such that they attempt to represent individual life histories (see Schoon, 2006) that are comprised of causal chains of effects over time (see Rutter & Silberg, 2002).  Such implicit assumptions led Kraemer, Stice, Kazdin, Offord, and Kupfer (2001) to claim that insufficient attention had been paid to the development of methodological tools for the assessment of risks and that for cumulative indices in particular,
	“Accumulating risk factors and either counting or scoring them does little to 	increase the understanding of etiologic process...”  

In addition to the concerns made about the ability of accumulation to represent transactions between risks, such an approach makes further assumptions concerning how a conceived combined risk may be operationalised.  Primarily, cumulative indices dichotomize individual measures of risk, a procedure that has been criticized as dichotomization incurs substantial loss of information that can sometimes lead to dubious statistical relationships (MacCallum, Zhang, Preacher, & Rucker, 2002).  However, such uncertainty concerning the impact of dichotomization upon statistical relationships is dependent upon the exact procedure followed and the circumstances in which it is used.  For example, although methods that use arbitrary cut-offs have been criticized (e.g. median splits; MacCallum et al.), those that do not have also been acknowledged as useful (e.g. the ‘extreme groups approach’; Preacher, Rucker, MacCallum, and Nicewander, 2005) or even necessary (e.g. dichotomizing a moderator that is hypothesized to have a step function; Baron & Kenny, 1986).  

As well as the potential problems associated with dichotomization, the summation of dichotomous measures assumes that distal and proximal factors may be summed together with equal weight (Luthar, 1993).  Such an assumption is one that Lee (2007) identifies as being the opposite to that of the traditional multiple regression where each risk would be viewed as having a unique and separate effect.  Finally, if the accumulation strategy is used to identify a sub-sample of children who are deemed especially at-risk then this represents the ‘extreme groups approach’; the same procedure that despite have been earlier noted as useful, has nonetheless been criticized (Preacher et al., 2005). 

Alternative methods for measuring combined risk
To compare and contrast alternative approaches for assessing the impact of risks upon development, Burchinal, Roberts, Hooper and Zeisel (2000) investigated the impact of social risks on children’s cognitive development between the ages of 1 and 4 years of age.  Their sample consisted of 87 African American children in full time community based child care who had started between the ages of 1 and 11 months of age (mean = 5.4 months).  Using 9 individual social risk factors, the authors tested 3 statistical approaches that were noted as having been previously used to investigate how proximal risks (e.g. birth weight; Brooks-Gunn, Klebanov, Liaw, & Spiker, 1993) and distal risks (e.g. poverty; Garmezy, 1991) act together when affecting children’s development: multiple regressions, exploratory factor analyses, and cumulative indices.  

The first of these approaches, the traditional multiple regression, establishes and established the effects of risks upon development by modeling the separate effects of individual risks so that they could be compared (Sameroff & Seifer, 1983).  Unfortunately, the frequent presence of multicollinearity in these analyses obscures relationships between individual risks and developmental outcomes and this was the situation in the analyses reported by Burchinal and her colleagues (2000).  The second approach, exploratory factor analysis, is able to overcome the problems associated with multiple regressions and shared many of the advantages of cumulative risk indices.  In the case of the Burchinal et al. study, this included greater predictive power of cognitive development when assessed at 1, 2, 3, and 4 years of age.  Nevertheless, like cumulative risk indices and multiple regressions, factor analysis also has disadvantages: Burchinal and colleagues noted that exploratory factor analysis uses and used an empirical rather than conceptual basis for data reduction.  Consequently, such an approach limits the external validity of any findings and can result in several extracted factors when only a single factor is desired for the representation of combined risk.  The authors concluded that,
	“…both the risk-factor approach and the risk-index approach appear to provide 	viable methods for relating social risk to developmental patterns.”

When considering factor analysis as an alternative procedure for representing combined risk, the issues raised over whether accumulation is an appropriate strategy persist.  This is partly due to both cumulative indices and factor analyses attempting to derive constructs that represent the transactions between individual risks.  More specifically, neither cumulative indices nor factor analyses attempt to represent a relationship that is synonymous with statistical moderation.  Instead, the “working together” (Kraemer et al. 2001) or ‘transactions’ of risks can, in the case of certain types of factor analysis, be represented by significant inter-correlations amongst individual risks (e.g. Agresti & Finlay, 1997).  Furthermore, the explicit representation of the transactions between risks that is possible in factor analysis is one omitted from the cumulative strategy where any set of variables, correlated or not, is dichotomized and tallied.  As a result, of the two attempts at representing the combined effect of risks acting in unison, the transactions between risks as represented by significant correlations is only rendered explicit in factor analysis.  In turn, this is a consideration that adds weight to the claim that factor analyses might be a valid alternative strategy to construction of cumulative indices.  

Formative measurement
One statistical procedure that can be used for the measurement of combined risk and that was not included in the Burchinal et al. (2000) paper is confirmatory factor analysis using formative (rather than reflective) measurement.  Furthermore, this is an approach to measuring underlying constructs that has recently received much attention in its applicability to psychological research (e.g. Howell, Breivik, & Wilcox, 2007).  In addition to stress and socioeconomic status (Edwards & Bagozzi, 2000), risk itself has been put forward as an example of a concept that can be appropriately measured through this strategy (Kleine, 2006).  As an example, Kleine suggests that school achievement and classroom adjustment could be considered at risk due to the combined effects of socioeconomic status, parental psychiatric history, and verbal IQ.  

Figure 1 illustrates the differences between formative and reflective measurement: with formative measurement, each observed risk y is theorized to be a cause () rather than an effect indicator () of an unobserved underlying latent risk  (see Kleine, 2006; Edwards & Bagozzi, 2000).  By referring back to the epidemiological research that investigated the causes of heart disease, the validity of a formative rather than reflective approach to measuring combined risk is elaborated upon.  An increased risk of heart disease was not theorized to be the cause of obesity and smoking, but rather, individual risks such as smoking and obesity in combination were theorized to be causes of the risk.  Furthermore, Figure 1 shows that the direction of causality between observed (‘manifest’) and unobserved (‘latent’) variables has consequences for how measurement error in each of these methods is represented.  With reflective measurement, the individual measurement errors (the ‘residuals’ ) are unique to each manifest indicator, whereas with formative measurement there is a single combined measurement error   (the ‘disturbance’) at the latent construct level.  
(Insert Figure 1 here)

It is the disturbance term  that is at the heart of these ‘latent composites’ as without it Kleine (2006) notes that the procedure would instead be merely equivalent to Principal Component factor Analysis.  However, by estimating measurement error at the construct rather than indicator level with this disturbance term, it becomes harder to measure the reliability of the formative indicators when the latent composite is theorized to be a dependent variable in a larger statistical model (Kleine, 2006).  Diamantopoulos (2006) notes that this difficulty stems from the disturbance term simultaneously capturing the predictive power of variables that are hypothesized as having direct effects upon the latent composite 

Although the differences between the direction of causality in formative and reflective measurement arguably lend validity to the formative approach as a method for representing a combined risk, some of the recent focus on formative measurement has also been critical (e.g. Howell et al., 2007).  Furthermore, the difference in the direction of causality between the approaches obliges researchers to accept different sets of assumptions when attempting to derive an estimate of combined risk.  In addition to the impact of measurement error at different levels, Kleine (2006) also outlines differences concerning the need for inter-correlations (and thus the conceptual transactions) between individual risks.  The rationale of reflective measurement (case (a) in Figure 1), classical test theory (Diamantopoulos, 2006), requires that the indicators should be internally consistent (as represented by inter-correlations).  Alternatively, with formative measurement (case (b) in Figure 1) there is no need for such correlations between individual risks as the rationale for these models is different (as with cumulative indices).  

The differential need for correlations amongst individual risks between these alternative methods impacts the underlying assumption that risks ‘work together’ (Kraemer et al. 2001): the transactions between risks is explicit in reflective measurement (via the correlations) but implicit in formative where ‘working together’ refers to the hypothesis that all of the risks negatively impact children’s development.  The departure from the explicit representation of the transactions between risks that is possible with reflective confirmatory factor analysis is a contrasting drawback to the increased validity of representing a combined risk as the effect of coinciding individual risks.  

Overview of the study
This study was designed to test whether confirmatory factor analysis using formative measurement could produce a valid alternative to a cumulative index in measuring combined risk.  For validation, alternative latent composite measures were compared to a cumulative index created from the same risks and both approaches were used to predict the cognitive development of a sample of young children at the ages of 36 and 58 months.  Two separate composite measures were created to test the validity of two of the assumptions implicit in the creation of cumulative indices: first, that both ordinal and continuous measures may be dichotomized, and second, that these dichotomies may then be tallied.  If the composite measures demonstrated greater predictive power of the children’s cognitive development than the cumulative index, then the use of such measures would be validated for use in future studies.  In summary, this investigation aimed to answer two research questions:

1.	Do confirmatory factor analyses that use formative measurement yield factor scores that are comparable with the measures derived from cumulative indices?
2.	As the factor analysis strategy becomes increasingly differentiated from the cumulative index procedure, do the alternative measures of combined risk display greater predictive power of children’s cognitive development?

Method
Participants
The participants in the current investigation were 2,899 children from six different geographical areas (urban, suburban, and rural) who had their individual risks and general cognitive abilities measured and statistically examined after informed consent was obtained from their parents/legal guardians.  To enable the different measures of combined risk to be compared, information about individual risks and children’s cognitive development was analyzed that had originally been gathered by the Effective Provision of Preschool Education (EPPE) studies (Sylva, Melhuish, Sammons, Siraj-Blatchford, & Taggart, 2004).  Together, the EPPE studies comprise a longitudinal investigation of the effectiveness of preschool provision in the United Kingdom that began in 1997.  EPPE used stratified random sampling to gather a sample of children that was broadly representative of the English population (Sammons et al., 1999).  Table 1 displays the gender and ethnicity of this sample which slightly over represented ethnic minorities when compared to the national figures for this time period (Large & Ghosh, 2006).  As the social and familial demographics of the families constituted risks in this study, the nature of these measurements and their distributions are presented in the following section (see Table 2).  
(Insert Table 1 here)

Measures
Demographics as Potential Risks.  The data from the EPPE studies lent itself to this investigation because a cumulative risk index (termed an ‘index of multiple disadvantage’ in the EPPE studies) had already been constructed as part of a study of Special Educational Needs (see Sammons et al., 2002b; informed by Sammons, Kysel, and Mortimore, 1983).  Table 2 presents the 11 measures that the EPPE team identified as being strongly related to poor development when constructing their cumulative risk index.  These social, familial, and home demographics were assessed through a parental interview when families entered the EPPE study (at mean child age 36 months).  Each demographic measure that was included in the cumulative index was identified from the results of multi‑level (hierarchical linear) regression analyses that found each to be a significant predictor of lower cognitive attainment (p<0.05; see Sammons et al., 2002b).  

(Insert Table 2 here)

Table 2 reveals the distribution of the 11 demographic variables that were used to construct the cumulative index and the percentage of young children who fell below the cut off points for each of these risk indicating variables.  The large disparity between the percentages of children who were identified as at risk across these variables is partly the product of the sample drawn by EPPE over-representing disadvantaged families (Sylva et al., 2004).  For example, only a relatively small percentage of children were born with a low enough weight for this to be found to be statistically associated with their later cognitive development (8%).  Conversely, due to the over‑representation of disadvantaged children, a much larger percentage of children had fathers who were relatively unskilled unemployed or absent, a situation that was also found to be significantly associated with children displaying lower cognitive development (34%).   While Table 2 reveals that 4 of the demographic variables contained missing data, it also shows that this percentage was extremely small: demographics with missing data had either a single or at most 8 non-respondents (0.03% or 0.28% respectively).  

The last of the identified risks presented in Table 2, the Home Learning Environment is a scale measure developed by the EPPE team to assess the learning opportunities available to children in their home environments (see Sylva et al., 2004).  The construction of the scale was based upon parental responses to questions asked in interviews when families entered the EPPE study concerning involvement in pre‑reading, reading, and singing (e.g. the frequency a child was taken to a library; Sammons et al., 1999).  Additionally, although the EPPE team included no social demographic measures (such as parental income or education) in this scale, they found it to have a stronger statistical relationship with cognitive development over the pre‑school period than any measure of social demographics (Melhuish, Sylva, Sammons, Siraj-Blatchford, & Taggart, 2001).  Since its conception, the Home Learning Environment scale and its component questions have been used in further large scale longitudinal studies that have investigated children’s development.  Amongst others, these have included the Families, Children, and Child Care study (FCCC; see Stein, Malmberg, Sylva, Barnes, & Leach, in press), and the Millennium Cohort Study (MCS), a birth cohort investigation which has tracked nearly 19,000 British children born between the years 2000-2002 (e.g. Mathers, Sylva, & Joshi, 2007).  

The descriptions and percentages on the right hand side of Table 2 determined the values that were used to transform the original measures into risk-assessing dichotomies. Importantly, these were not the arbitrary cut-offs that this paper has noted as having been heavily criticized (e.g. MacCallum et al., 2002) but rather the dichotomization was based upon critical values that were obtained on the basis of exploratory multi‑level regression analysis (see Sammons et al., 2002b). The resultant dichotomies were summed to form the cumulative risk index that was then used throughout the remainder of this and the EPPE studies.   

In this investigation, the first measure of combined risk that was used as an alternative to the cumulative index was one that used the same dichotomies but theorized them to be causes of a continuous (rather than ordinal) and latent composite (rather than accumulative) risk.  The second measure followed this procedure but instead used those variables that were originally measured on an ordinal scale in conjunction with the remaining dichotomies (see Table 2).  The first composite measure was created to test the summation strategy, whilst the second composite was designed to additionally test how valid it was to dichotomize previously ordinal data.  In summary, the three alternative measures of combined risk were: 
Method (1): The original cumulative risk measure as used in the EPPE studies 
Method (2): A latent composite measure created from the same dichotomies as (1)
Method (3): A latent composite measure developed from the ordinal (where possible) and dichotomous items that were used in procedures (1) and (2)

Cognitive Development.  The EPPE investigation measured each child’s cognitive abilities at study entry and school entry with the British Ability Scales (BAS; Elliot, NFER-NELSON, Smith, & McCulloch, 1996) and one-to-one assessments by trained researchers.  After their creation, each of the 3 approaches to measuring combined risk were then compared and contrasted in their statistical prediction of the BAS assessed cognitive development of the 2,899 children when they were on average 36 (scores: M= 91.53; SD= 13.93) and 58 months old (scores: M= 96.13; SD= 14.84).  

Analysis
The two confirmatory factor analyses were carried out using the Mplus statistical software (Muthén & Muthén, 2004) where the procedures for specifying formative measurement have been outlined by the authors (Muthén, 2006).  Each set of factor scores were computed separately and along with the cumulative index specified as predictors in the development of children’s cognitive abilities in three separate path models (see Figure 2).  Finally, correlations were used to investigate the similarity of the alternative measures of combined risk.
(Insert Figure 2 here)

Results
Confirmatory Factor Analyses
Descriptives.  Table 3 presents correlations between each of the 11 variables that were identified as indicating a risk to young children’s cognitive development.  The high correlation between low birth weight and whether a child was born prematurely (r = 0.43, p<0.001) was recognized in the original EPPE studies and a composite variable was created which is included in Table 3.  This table illustrates an important disparity between the individual risks: those variables reflecting parental Socio‑Economic Status (SES; e.g. a single parent family) were correlated with many more of the other risks than those that were variables less indicative of SES (e.g. whether a child was born prematurely).  Each of the risks where then specified as causal indicators in the confirmatory factor analysis that was designed to test the effects of the summation strategy used by cumulative risk indices (mean, M = -0.82; standard deviation, SD= 0.74).  In addition, some of the risks were also used as indicators in the subsequent confirmatory factor analysis that tested the dichotomization strategy (M= -1.09; SD= 0.89) when the original risks could not be used due to them being originally assessed by categorical measures (see Table 2).  

(Insert Table 3 here)

Model Fit.  Each of the four measures of model fit displayed in Table 4 returned values that indicated excellent model fit.  This conclusion was based upon: The insignificant results of the Chi-Square Test (see Byrne, 1989); the close to upper limit values (close to 1) returned from both the Comparative Fit Index (CFI, Bentler, 1990) and the Tucker‑Lewis Index (TLI, Bentler & Bonett, 1980); and the value of the Root Mean Square Error of Approximation (RMSEA) statistic that lay within the range deemed as indicating acceptable model fit (see Browne & Cudeck, 1993).  Additionally, whilst the confirmatory factor analysis using dichotomized risks returned slightly better model fit across each measure, these differences were only small.  Finally, although good model fit does not validate any subsequent findings, the fit indices do indicate that the models could accurately represent the original data.  
(Insert Table 4 here)

Factor Loadings.  Table 5 presents the standardized factor loadings of each risk that was theorized to cause a latent composite that attempted to represent the effects of these risks in combination.  The third of these risks, ‘Pre‑maturity or Low birth weight’ was operationalised in a different manner to the others: when the latent composite was indicated by dichotomies, this risk was represented by a single dichotomy.  Alternatively, when the combined latent risk variable was represented by ordinal variables (were possible), ‘Pre‑maturity or Low birth weight’ was a risk that was then represented by both variables with a correlation co-efficient specified between them (r = -.43, p < .001).  
(Insert Table 5 here)

In general, the majority of the 11 risks that were chosen by the EPPE investigation to form a cumulative risk index were also found to be significant causal factors of the latent composites.  Furthermore, this was found whether the individual risks were included in ordinal or in dichotomous forms.  However, Table 5 also reveals a disparity between the weightings of these risks.  For example,  factor loadings were found to range from a statistically insignificant .04 for pre-maturity to highly significant .39 for the home learning environment, a loading nearly 10 times that found for a pre-mature birth.  This large variation between the weightings of individual risks is in stark contrast to the assumption of the cumulative index procedure that individual risks contribute equally.

The factor loadings in Table 5 also allow for a comparison to be made between the impacts on measurement that were associated with dichotomizing.  When individual risks were dichotomized from a previously ordinal form, there was no consistent effect on the relationship between individual and combined risks.   For example, while the factor loading of ‘social class of father’s occupation’ increased with dichotomization, that of ‘large’ family was unchanged.  Such differences signify that the effect of dichotomizing might depend upon the nature of the original risks, although the possibility of a systematic relationship is beyond the scope of this paper.  

Predicting cognitive development
When the three alternative methods for measuring combined risk were used to predict children’s cognitive development at the average ages of 36 and 58 months, a systematic difference between the measures was identified.  As the method for representing combined risk became increasingly different from the cumulative index procedure, the statistical relationship between risk and cognitive development became stronger.  Furthermore, this difference was found over and above the ‘value added pathways’ (Table 6) of the estimated regression co-efficients between general cognitive abilities of the young children at 36 and 58 months of age.  These systematic increases were found both in the size of the standardized regression coefficients and in the amount of explained variance (Table 7).  
 (Insert Tables 6 and 7 here)

For interpretation of the results it is important to note the disparity between the absolute sizes of both the regression co-efficients and the proportions of variance explained across the two time periods.  This discrepancy stems from the necessity to include general cognitive ability at 36 months as a predictor of the same ability 22 months later for a valid interpretation of the results (producing a ‘progress model’).  Tables 4 and 5 show that, based upon this progression model, there remained an effect of combined risk on children’s general cognitive abilities at 58 months even when controlling for the earlier level of this same ability.  In addition, the systematic differences between the methods of measuring combined risk also remained; there was an increase in the absolute values of the standardized regression coefficients and amount of explained variance.  

Similarity of measures 
The results of the confirmatory factor analyses and regressions revealed the impact of making fewer assumptions in measuring the combined risk to cognitive development.  The factor analysis that produced a measure of risk least similar to the cumulative index (r = 0.84, p<0.01) was the analysis that made the fewest same assumptions.  Perhaps coincidentally, the ‘middle ground’ alternative was found to be as similar to the cumulative index as it was to the method with greatest predictive power of cognitive development (in both cases: r = 0.91, p<0.01). 

Discussion
Based upon the results that were obtained from the confirmatory factor and the longitudinal path analyses, each posited research question can now be answered.  Firstly, confirmatory factor analyses that used formative measurement were shown to yield factor scores that were broadly comparable (significant correlations of large magnitude) with the measures that were returned from a cumulative index.  Secondly, as the factor analysis strategy diverged from the cumulative index procedure, so a systematic increase was observed in the relationship between these measures and young children’s general cognitive development.  Furthermore, this was a finding that was observed whilst statistically controlling for earlier levels of development in a ‘value added’ longitudinal analysis.   These findings permitted this paper to extend those that were found in the work of Burchinal and colleagues (2000), by using an approach that was driven more by developmental theory and less by statistical data.

Putting the systematic differences between each of the methods of measurement used by this study into perspective, the magnitude of these differences varied according to the age at which young children’s General Cognitive Ability was examined.  For example, the difference between the methods in the percentage variance that they explained at 36 months of age was 11% whilst at 58 months it was only 2%.  One explanation for this variance is that the abilities at 58 months statistically controlled for the effects of these same abilities 22 months earlier.  Due to the strong statistical association of these repeated measures (Table 6) the percentage of variance that was left for the combined measures of risks to uniquely explain fell.  While there was 100% of the variance available at 36 months, this could only be less at 58 months when the ability at 36 was a simultaneous predictor.  The fact that there remained any systematic difference at 58 months indicates that the differences between the methods were robust in its consequence for impacting cognitive development.  

From its findings, this study has demonstrated that when estimating the combined effect of risks to children’s cognitive development, constructing a cumulative risk index does not always produce a measure with the greatest predictive power.  This conclusion was established by testing the assumptions implicit in cumulative risk indices: that it is valid to dichotomize individual risks, and that each of these dichotomies should then contribute equally (through summation) in a derived measure of combined risk.  It has instead been shown that individual risks can contribute very differently from one another when each is theorized to causes a latent underlying composite risk (see also Sammons, Kysel, & Mortimore, 1983).   

Importantly, the factor loadings of individual risks that are returned from the confirmatory factor analysis constituted a source of information that the accumulation procedure could not provide an equivalent to.  Namely, this was the individual and relative contribution that each risk made to a latent composite representing these risks in combination (Table 5).  With this information, the contribution of each to the estimate of their effect in combination (the latent composites) could be compared and contrasted.  For example, Table 5 revealed that the Home Learning Environment made a greater contribution to the measure of combined risk than whether a mother was in employment.  This finding indicates that the activities mother and child engage in are of more importance to the child’s cognitive development than her occupational status (see Clarke-Stewart & Dunn, 2006).  

The additional information available from the confirmatory factor analysis rather than a cumulative risk index is not the only reason justifying its use as a procedure to estimate a combined risk.  If it was, then there would be little reason to use this strategy instead of the exploratory one used by Burchinel et al., (2000).  The true merit of confirmatory factor analysis that uses formative measurement lies in its ability to simultaneously overcome drawbacks inherent to: cumulative indices, exploratory factor analysis, and confirmatory factor analysis that uses reflective measurement.  The procedure outlined in this paper retains and returns more information than a cumulative index; uses a conceptual basis for deriving a combined risk measure (unlike exploratory factor analysis); and uses a more appropriate method for specifying the relationships between individual and combined risks (due to formative rather than reflective measurement). 

The findings of Burchinal et al., (2000) aid in interpreting why the confirmatory factor analysis procedure used in this paper was found to demonstrate greater predictive power of young children’s cognitive development.  Factor analyses, whether exploratory as in the work Burchinal et al. or confirmatory as used here, appears better able to statistically predict any outcome when compared to a cumulative index.  In turn, this seems to be due to a factor analysis solution retaining more information about the original variables and their inter‑relationships as it neither dichotomizes nor summates. 

From the results of this study and the work of Burchinal et al. (2000), consistent and smaller statistical relationships have been found between combined risk and young children’s cognitive development when a cumulative risk index has been used instead of a multiple regression or a factor analysis.  This is an important finding because it is these statistical relationships that have been relied upon in investigations that have studied ways of preventing the maladaptive development associated with combined risk (e.g. The Rochester Longitudinal Study; Sameroff et al., 1993).  As a result, this study suggests that any of the effects of combined risk identified in these previous studies may have been underestimated.  

In summary, the method of measuring combined risk presented in this paper has been shown to overcome problems primarily associated with the construction of cumulative indices but additionally, also with exploratory factor analysis.  Confirmatory factor analysis employing formative measurement has been noted to have a number of advantages.  It uses a conceptual rather than empirical basis for data reduction, does not dichotomize individual risks, and produces a measure of combined risk that does not assume that each risk contributes equally.  As a result, this method has been shown to return a measure of combined risk which demonstrates greater internal and external validity when examining the cognitive development of young children between the average ages of 36 and 58 months of age. 

Limitations
This study and the methods it presents are also not without their own limitations.  First, factor analysis, whether exploratory or confirmatory, requires a large sample size and this is not always possible in studies of high risk children (Sameroff, Gutman, & Peck, 2003).  However, as an alternative, a cumulative index is not necessarily validated by a small sample size either, especially if this is a procedure based on assumptions of questionable validity.   Second, a cumulative index returns an ordinal measure that at first glance appears simpler to interpret; an increase of one unit represents the child being exposed to one additional risk.  In comparison, a factor analysis will return an interval level measure where the individual values are more abstract.  However, because the combined risk that is indicated by a cumulative index might lack validity due to its assumptions, this apparent simplicity in interpretation might be inappropriate. Thus, although using a cumulative risk index might more easily enable an odds ratio analysis that reveals the effect of exposure to each additional risk upon the likelihood of some negative outcome (e.g. Johnson and Waldfogel, 2002), it is in fact no easier as the cumulative index is a less precise measure of risk.  

Another limitation concerns how the methods presented here, as well as those investigated by Burchinal et al., (2000) of exploratory factor analysis and multiple regression, represent the transactions amongst risks.  Specifically, none of these methods treat this ‘working together’ as necessitating moderation.  Instead, researchers must look to alternatives such as the MacArthur moderator‑mediator approach (see Essex et al. 2006) for such an explicit representation of risks interacting/moderating one another as they impact an outcome such as children’s abilities.

A limitation of this study’s analysis that was particular to formative measurement was the necessity to include a minimum of two measures as being dependent upon the latent composite risk (see Bagozzi, 2007).  Kleine (2006) addresses this drawback by proposing MIMIC factor analyses (multiple indicators multiple causes) as an alternative method to formative measurement which utilizes both formative and reflective measurement simultaneously.  Furthermore, the measurement of some risks was inherently more exact than others, for example, the number of siblings rather than the Home Learning Environment scale.  This difference in measurement poses a problem for confirmatory factor analysis that uses formative measurement as it does not return separate measurement errors for each risk.  Consequently, by using this procedure it is harder to establish the internal validity of the composite risk measure score.  However, the internal validity of formative measurement is instead primarily established though the direction of the causal arrows between individual risks and the combined risk (see Figure 1).  

Future Directions
A final limitation of this study is that only a single domain of development (cognition) was examined over a very limited period of time.  Further work is needed to establish whether the associations documented here are replicable across other areas of children’s development, other periods of time, and importantly, across different sets of individual risks.  Also, further work is required to investigate whether the composite risk measure outlined in this paper affects development in a linear or threshold fashion (Appleyard, Egeland, Dulmen, & Sroufe 2005) and whether the impacts of risk varies across time.  Finally, all of the alternative methods for measuring combined risk could benefit by representing the idea that distal and proximal risks differ in how they act upon development (Luthar, 1993).  Figure 3 demonstrates how the distal and proximal risks in this study could be represented using confirmatory factor analysis whilst also taking into account their nested structure.  Studies such as these would further enable central theories of child development to be explored within investigations of the effects of cumulative risk upon vulnerable children.  
(Insert Figure 3 here)

References
Agresti, A., & Finlay, B. (1997). Statistical Methods for the Social Sciences (3rd ed.). New Jersey: Prentice-Hall.
Appleyard, K., Egeland, E., Dulmen, M. H. M., & Sroufe, L. A. (2005). When more is not better: the role of cumulative risk in child behavior outcomes. Journal of Child Psychology and Psychiatry, 46(3), pp 235–245.
Bagozzi, R. P. (2007). On the Meaning of Formative Measurement and How It Differs From Reflective Measurement: Comment on Howell, Breivik, and Wilcox (2007). Psychological Methods, 12(2), 229-237.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.
Bentler, P. M. (1990). Comparative Fit Indexes in Structural Models. Quantitative Methods in Psychology, 107(2), 238-246.
Bentler, P. M., & Bonett, D. G. (1980). Significance Tests and Goodness of Fit in the analysis of Covariance Structures. Psychological Bulletin, 88(3), 588-606.
Brooks-Gunn, J., Klebanov, P. K., Liaw, F. R., & Spiker, D. (1993). Enhancing the development of low-birth-weight, premature-infants - changes in cognition and behavior over the 1st 3 years. Child Development, 64(3), 736-753.
Browne, M. W., & Cudeck, R. (1993). Alternative Ways of Assessing Model Fit. In K. A. Bollen & J. S. Long (Eds.), Testing Structural Equation Models (pp. 136-162). CA: Sage.
Burchinal, M. R., Roberts, J. E., Hooper, S., & Zeisel, S. A. (2000). Cumulative Risk and Early Cognitive Development: A Comparison of Statistical Risk Models. Developmental Psychology, 36(6), 793-807.
Byrne, B. M. (1989). A primer of LISREL: Basic applications and programming for confirmatory factor analytic models. New York: Springer-Verlag.
Clarke-Stewart, K. A., & Dunn, J. (2006). Families Count: Effects on Child and Adolescent Development. New York: Cambridge University Press.
Diamantopoulos, A. (2006). The error term in formative measurement models: interpretation and modeling implications. Journal of Modelling in Management, 1(1), 7-17.
Edwards, J. R., & Bagozzi, R. P. (2000). On the Nature and Direction of Relationships Between Constructs and Measures. Psychological Methods, 5(2), 155-174.
Elliot, C. D., NFER-NELSON, Smith, P., & McCulloch, K. (1996). British Ability Scales Second Edition (BAS II). Windsor: NFER-NELSON.
Essex, M. J., Kraemer, H. C., Armstrong, J. M., Boyce, W. T., Goldsmith, H. H., Klein, M. H., Woodward, H., Kupfer, D.J., (2006). Exploring Risk Factors for the Emergence of Children’s Mental Health Problems. Archives of General Psychiatry, 63, 1246-1256.
Evans, G. W. (2003). A Multimethodological Analysis of Cumulative Risk and Allostatic Load Among Rural Children. Developmental Psychology, 39(5), 924-933.
Fearon, R. M. P., & Belsky, J. (2004). Attachment and Attention: Protection in Relation to Gender and Cumulative Social-Contextual Adversity. Child Development, 75(6), 1677-1693.
Garmezy, N. (1991). Resiliency and Vulnerability to Adverse Developmental Outcomes Associated With Poverty. American Behavioral Scientist, 34(4), 416-430.
Garmezy, N., Masten, A. S., & Tellegen, A. (1984). The Study of Stress and Competence in Children: A Building Block for Developmental Psychopathology. Child Development, 55(1), 97-111.
Hinshaw, S. P. (2002). Intervention research, theoretical mechanisms, and causal processes related to externalizing behavior patterns. Development and Psychopathology, 14, 798-818.
Howell, R. D., Breivik, E., & Wilcox, J. B. (2007). Reconsidering Formative Measurement. Psychological Methods, 12(2), 205-218.
Johnson, E. I., & Waldfogel, J. (2002). Children of incarcerated parents: Cumulative risk and children's living arrangements. Joint Center for Poverty Research Working Paper 306: Northwestern University/University of Chicago.
Kleine, R. (2006). Formative Measurement and Feedback Loops. In G. R. Hancock & R. O. Mueller (Eds.), Structural Equation Modelling: A Second Course (pp.43-68). Greenwich, CT: Information Age Publishing. 
Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How Do Risk Factors Work Together?  Mediators, Moderators, and Independent, Overlapping, and Proxy Risk Factors. American Journal of Psychiatry, 158, 848-856.
Large, P., & Ghosh, K. (2006). A methodology for estimating the population by ethnic group for areas within England. Population Trends (123), 21-31.
Lee, H. (2007). Using Cumulative Risk Models to Link Social Disadvantage to Obesity Risk in the Transition to Young Adulthood, Population Association of America (PAA) 2007 Annual Meeting Program Chapel Hill: University of North Carolina.
Lloyd-Jones, D. M., Wilson, P. W. F., Larson, M. G., Beiser, A., Leip, E. P., D'Agostino, R. B., et al. (2004). Framingham Risk Score and Prediction of Lifetime Risk for Coronary Heart Disease. American Journal of Cardiology, 94(1), 20-24.
Luthar, S. S. (1993). Annotation: Methodological and Conceptual Issues in Research on Childhood Resilience. Journal of Child Psychology & Psychiatry, 34(4), 441-453.
Luthar, S. S., Cicchetti, D., & Becker, B. (2000). The Construct of Resilience: A Critical Evaluation and Guidelines for Future Work. Child Development, 71(3), 543-562.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the Practice of Dichotomization of Quantitative Variables. Psychological Methods, 7(1), 19-40.
Mathers, S., Sylva, K., & Joshi, H. (2007). Quality of Childcare Settings in the Millennium Cohort Study (QCSMCS). London: DfES.
Melhuish, E., Sylva, K., Sammons, P., Siraj-Blatchford, I., & Taggart, B. (2001). EPPE Technical Paper 7:  Social/Behavioural and Cognitive Development at 3-4 years in relation to family background. London: Institute of Education.
Muthén, B. O. (2006). Indicator Arrows Pointing to a Factor. Retrieved 51/2/2007, from http://www.statmodel.com/discussion/messages/11/226.html?1159727339
Muthén, L. K., & Muthén, B. O. (2004). Mplus user's guide. Los Angeles: Authors.
Preacher, K. J., Rucker, D. D., MacCallum, R. C., & Nicewander, W. A. (2005). Use of the Extreme Groups Approach: A Critical Reexamination and New Recommendations. Psychological Methods, 10(2), 178-192.
Rutter, M. (1976). Research Report: Isle of Wight studies, 1964-1974. Psychological Medicine, 6, 313-332.
Rutter, M. (1979). Protective factors in children's responses to stress and disadvantage. In M. W. Kent & J. E. Rolf (Eds.), Primary prevention of psychopathology (Vol. 3. Social competence in children, pp. 49-74). NH: Hanover: University Press of New England.
Rutter, M. (1985). Family and school influences on cognitive development. Journal of Child Psychology & Psychiatry, 26(5), pp. 683-704.
Rutter, M. (1987). Psychosocial Resilience and Protective Mechanisms. American Journal of Orthopsychiatry, 57(3), 316-331.
Rutter, M., Dunn, J., Plomin, R., Simonoff, E., Pickles, A., Maughan, B., et al. (1997). Integrating nature and nurture: Implications of person–environment correlations and interactions for developmental psychopathology. Development and Psychopathology, 9(2), pp 335-364.
Rutter, M., & Silberg, J. (2002). Gene-Environment Interplay in Relation to Emotional and Behavioral Disturbance. Annual Review of Psychology, 53, 463-490.
Sameroff, A. J., & Cole, R. (2003). Academic Growth Curve Trajectories From 1st to 12th Grade: Effects of Multiple Social Risk Factors and Preschool Child Factors. Developmental Psychology, 69(4), 777-790.
Sameroff, A. J., Gutman, L. M., & Peck, S. C. (2003). Adaptation among Youth Facing Multiple Risks. In S. S. Luthar (Ed.), Resilience and Vulnerability: Adaptation in the Context of Childhood Adversities (pp. 364-391). Cambridge: Cambridge University Press.
Sameroff, A. J., & Seifer, R. (1983). Familial Risk and Child Competence. Child Development, 54(5).
Sameroff, A. J., Seifer, R., Baldwin, A., & Baldwin, C. (1993). Stability of Intelligence from Preschool to Adolescence: The Influence of Social and Family Risk Factors. Child Development, 64(1), 80-97.
Sameroff, A. J., Seifer, R., Barocas, R., Zax, M., & Greenspan, S. (1987). Intelligence quotient scores of 4-year old children: Social-environmental risk factors. Pediatrics, 79, 343-350.
Sammons, P., Kysel, F., & Mortimore, P. (1983). Educational Priority Indices: a new perspective. British Educational Research Journal, 9(1), 27-40.
Sammons, P., Sylva, K., Melhuish, E., Siraj-Blatchford, I., Taggart, B., Smees, R., Dobson, A., et al. (1999). Technical Paper 2: Characteristics of the EPPE Project Sample at Entry to the Study. London: Institute of Education.		
Sammons, P., Sylva, K., Melhuish, E., Siraj-Blatchford, I., Taggart, B., & Elliot, K. (2002a). EPPE Technical Paper 8a: Measuring the Impact of Pre-School on Children's Cognitive Progress over the Preschool Period. London: Institute of Education.
Sammons, P., Taggart, B., Smees, R., Sylva, K., Melhuish, E., Siraj-Blatchford, I., et al. (2002b). EYTSEN Technical Paper 1: Special Educational Needs across the Pre-School Period. London: Institute of Education.
Schoon, I. (2006). Risk and Resilience: Adaptations in changing times. Cambridge: Cambridge University Press.
Stein, A., Malmberg, L. E., Sylva, K., Barnes, J., & Leach, P. (in press). The influence of maternal depression, caregiving, and socioeconomic status in the postnatal year on children’s language development. Child Care, Health and Development.
Sylva, K., Melhuish, E., Sammons, P., Siraj-Blatchford, I., & Taggart, B. (2004). Effective Provision of Pre-school Education: Final Report. London: Institute of Education.
Werner, E. E. (1993). A Longitudinal Perspective on Risk for Learning Disabilities, Annual Conference of the Learning Disabilities Association of America. San Francisco, CA.
Werner, E. E. (1996). Vulnerable but invincible: High risk children from birth to adulthood. European Child & Adolescent Psychiatry, 51, 5-47.
Wu, A. D., & Zumbo, B. D. (2007). Understanding and Using Mediators and Moderators. Social Indicators Research: An International Interdisciplinary Journal for Quality of Life Measurement.

Table 1.  Characteristics of the children sampled in the Effective Provision of Preschool Education studies
Variable	n	%	
Gender	2899	100	
Girls	1404	51.6	
Boys	1495	48.4	
Ethnicity	2897	99.93	
White UK	2152	74.23	
White European	110	3.79	
Black Caribbean	100	3.45	
Black African	60	2.07	
Indian	58	2	
Pakistani	131	4.52	
Bangladeshi	31	1.07	
Mixed	176	6.07	
Any other ethnicity	79	2.73	

Table 2.  Child, parent, and home characteristics associated with lower cognitive development in the Effective Provision of Preschool Education studies
Variable	n	mean	standard deviation	Cut-off for high risk	%	
Risk Factors	
1 First language a	2899			English not first language	9.55	
English	2622					
not English	277					
2 Number of siblings b	2899	1.45	1.18	3 or more siblings	15.04	
3 Pre-maturity a	2898			Premature at birth	16.77	
no	2412					
yes	486					
4 Birth weight in grams b	2891	3331.78	620.50	Below 2500 grams	7.60	
5 Mothers’ qualification levels a	2899			No qualifications	20.73	
none	601					
age 16 vocational	59					
age 16 academic	1077					
age 18 vocational	372					
age 18 academic	250					
degree or equivalent	369					
higher degree	128					
other professional	20					
other	23					
6 Social class of fathers’ occupations b	2891			Semi-skilled, never worked, absent father	33.66	
professional non manual	238					
other professional non manual	550					
skilled non manual	350					
skilled manual	706					
skilled	74					
semi skilled	334					
never worked	20					
father absent	619					
7 Fathers’ employment status a	2899			Not working/Unemployed	11.56	
employed full time	1518					
employed part time	90					
self employed	329					
combination (p/t and self employed)	8					
not employed	335					
father absent	619					
8 Mothers’ age at child birth b	2899			Age 13-17 at child birth	0.83	
13-17	24					
18-22	325					
23-32	1762					
33-42	759					
43-52	24					
53-62	5					
9 Household structure a	2899			Single Parent	13.66	
never married, single parent	396					
never married, living with partner	422					
married, living with spouse	1758					
separated/divorced	300					
widow/widower	4					
other	19					
10 Mothers’ employment status a	2898			Not working/Unemployed	50.69	
employed full time	443					
employed part time	863					
self employed	109					
part time	13					
not working	1469					
other	1					
11 Home Learning Environment b	2899	23.29	7.62	Bottom quartile	25.00	
Cumulative Risk Index	
Derived Index	2899	1.82	1.51	% encountering: 5+ risks	5.5	
				3-4 risks	21.3	
				1-2 risks	49.6	
 	 	 	 	no risks	23.5	
a Measure originally used in categorical form
b Measure originally used in at least ordinal form

Table 3.  Pearson product-moment inter-correlations between the individual dichotomized risks
Pearson's Correlations (2 d.p.)	Home Learning Environment - bottom quartile?	mother employed?	single parent?	young mother?	father employed?	low father social class?	mother has qualifications?	low birth weight or premature?	birth weight < 2500 grams	premature?	3 or more siblings?	
English as an additional language?	0.24***	-0.21***	-0.10***	0.01	-0.20***	0.08***	-0.21***	0.01	0.07***	-0.03	0.10***	
3 or more siblings?	0.15***	-0.14***	-0.04*	-0.03	-0.14***	0.04	-0.20***	0.00	0.02	-0.01	 	
premature?	0.05**	-0.03	0.04*	-0.01	-0.03	-0.01	-0.03	0.93***	0.43***	 	 	
birth weight < 2500 grams	-0.07***	-0.05**	0.05**	0.01	-0.07**	0.02	-0.07***	0.59***	 	 	 	
low birth weight or premature?	0.06**	-0.05*	0.04*	-0.01	-0.05*	-0.00	-0.05*	 	 	 	 	
mother has qualifications?	-0.26***	0.30***	-0.14***	-0.07***	0.25***	-0.23***	 	 	 	 	 	
low father social class?	0.15***	-0.12***	0.05*	0.04*	-0.20***	 	 	 	 	 	 	
father employed?	-0.17***	0.28***	-0.10***	-0.03	 	 	 	 	 	 	 	
young mother?	-0.09	-0.08***	0.09***	 	 	 	 	 	 	 	 	
single parent?	0.07***	-0.16***	 	 	 	 	 	 	 	 	 	
mother employed?	-0.15***	 	 	 	 	 	 	 	 	 	 	
*p < .05**  p < .01***  p < .001

Table 4.  Model fit of each factor analysis
Tests of model fit	Latent composites:	
	From dichotomous risks	From ordinal and dichotomous risks	
1. Chi-Square Test of Model Fit	12.2	16.7	
degrees of freedom	9	10	
P-value	0.20	0.08	
2. Comparative Fit Index (CFI)	0.999	0.998	
3. Tucker-Lewis Index (TLI)	0.997	0.995	
4. Root Mean Square Error of Approximation (RMSEA)	0.011	0.015	
Probability RMSEA <= .05 	1.000	1.000	

Table 5.  Standardized factor loadings of individual risks
Disadvantage indicators.            	Dichotomized Risks	Ordinal and Dichotomous Risks	
Child Variables	
1b	First language	        .42ª	        .35ª	
2c	Large family	        .17***	        .17***	
3	b	Pre-maturity OR 	        .12***	        .04	
	c	Low birth weight		       -.14***	
Parent Variables	
4c	Mothers’ qualification levels	       -.26***	       -.15***	
5c	Social class of fathers’ occupations	        .23***	        .37***	
6b	Fathers’ employment status	       -.15***	       -.06	
7c	Mothers’ ages	        .05	       -.05*	
8b	Household structure	        .09**	        .06*	
9b	Mothers’ employment status	       -.14***	       -.11***	
Home Environment Variables	
10c	Home Learning Environment 	       -.34***	       -.39***	
ª Unstandardardized factor loadings set to 1 for model convergence so there is no returned significance level
b Measure originally used in dichotomous form
c Measure originally used in ordinal form
*p < .05 **p < .01 ***p < .001

Table 6.  Associations between cognitive development at 36 and 58 months
Predicting general cognitive ability at mean age 58 months from general cognitive ability at mean age 36 months: value added pathway	Standardized Regression Coefficient (β)	Standard Error (SE)	Explained variance (r2)	

Cumulative risk index	0.69***	.02	0.55	
Latent composite from dichotomous risks	0.68***	.02	0.57	
Latent composite from ordinal and dichotomous risks	0.64***	.02	0.57	
***p < .001

Table 7.  Associations between alternative risk measures and cognitive development 
Predicting cognitive development	Mean Age 36 months	Mean Age 58 months	
	Standardized regression coefficient (β)	Standard Error	Explained variance (r2)	Standardized regression coefficient (β)	Standard Error	Explained variance (r2)	
							
Cumulative risk index	-0.46***	.17	0.21	-0.17***	.16	0.55	
Latent composite from dichotomous risks	-0.51***	.84	0.26	-0.19***	.42	0.57	
Latent composite from ordinal and dichotomous risks	-0.57***	.80	0.32	-0.23***	.40	0.57	
***p < .001

Figure 1. Confirmatory factor analysis using reflective and formative measurement 
(Adapted from Kleine, 2006)

(a) Reflective Measurement, y =  +

(b) Formative Measurement,  = y + 

Latent ‘composite’

Risk Factor 1

Risk Factor 2

Risk Factor 3

Disturbance 

    Underlying latent construct

(Measurement errors;  )

(Regression residual;  )

Residual 1

Residual 2

Residual 3

y

y

Risk Factor 1

Risk Factor 2

Risk Factor 3

Cognitive development at average age 36 months

Cognitive development at average age 58 months

Each of the 3 alternative measures of combined risk

Direct path

Indirect path,

(value added
pathway)

Figure 2.  Path models tested in this study

Home Learning Environment

Child Level

Family Level

First language

Large family

Pre-mature birth

Low birth weight

Mothers’ qualifications

Mothers’ employed?

Household Structure

Mothers’ ages

Fathers’ employed?

Fathers’ social class

Home Level

Figure 3.  The EPPE risks in a nested structure