Who benefits?

Citation

Axford, N. and Little, M. (2014), "Who benefits?", Journal of Children's Services, Vol. 9 No. 3. https://doi.org/10.1108/JCS-07-2014-0037

Who benefits?

Article Type: Guest Editorial From: Journal of Children's Services, Volume 9, Issue 3

Do evidence-based programmes reduce social inequality?[1]

There has been growing attention in recent years to the issue of how to get so-called "hard-to-reach" families to use services. There are various reasons for this, one of which is expressed well by Vincent La Placa and Judith Corlyon in their paper in this edition:

[S]uccessful inclusion and engagement is linked to overcoming the social gradient of health inequalities [ . . .] The lower an individual's social and economic status is, the poorer his or her health is likely to be. Access to and use of targeted services by hard-to-reach families is a significant mechanism in reducing health and social disparities on the economic and service levels (p. x).

Two assumptions underpin this assertion. One is that so-called hard-to-reach families will benefit from the interventions. The other is that they will actually benefit more than their better-off counterparts. This raises the question of whether these assumptions are well founded. Do some families benefit from interventions more than others? Specifically, do outcomes for families who tend to be more marginalised, for example on socio-economic or ethnic grounds, improve more than they do for those from less marginalised backgrounds? If they do, then the interventions in question are potentially a vehicle for reducing inequalities.

The answer requires, first, that studies measure and report socio-demographic factors, second that that they involve analyses that test for sub-group differences, and third that they find evidence favouring the traditionally more marginalised groups. So, what is known.

A few years ago we reviewed 100 programmes against standards of evidence with a view to identifying and then promoting interventions proven to improve children's health, emotional well-being, behaviour, education and relationships. Of the 47 programmes that met the standards overall, there were 142 studies that met the evaluation quality standards.

Most of these reported sample demographics well. For example, 92 per cent documented ethnic composition, and 75 per cent provided information about socio-economic status (SES). However, ethnicity was often not broken down in detail, and SES indicators varied considerably across studies (e.g. parent employment status, eligibility for reduced price meals at school, receipt of welfare entitlements).

Sub-group analyses were reported for relatively few studies. Of the programmes that did not have homogenous samples, nearly two-fifths (38 per cent) conducted sub-group analyses (mostly for ethnicity, though also for gender). Of these studies, just over a third (36 per cent) found no differences, with the majority finding differences (indicating both greater and less impact) for some sub-groups.

Scientists call analyses of differential effects of interventions "moderator analyses". Such analyses look at whether outcomes differ by the baseline characteristics of participants. So far in this editorial we have focused on socio-demographic variables but the focus could equally be on the severity of the problem or even genetic markers hypothesised to predict differential responsivity to intervention effects.

In randomised controlled trials, moderator analyses tend to be secondary to analyses that test for the main effects of the intervention. The results are potentially valuable for service designers and policy makers because they help with understanding whom interventions are suitable for and who benefits most or least (or is likely to be harmed). As such, the results can help make interventions more efficient and less risky.

Deeper analyses of moderators have been performed in two studies of preventive parenting interventions for children at risk of conduct problems. The first study, of Family Check-Up, found no moderator effects for most variables: families were equally likely to respond at all levels of disadvantage/distress (Gardner et al., 2009). However, children of low-educated mothers did better and children of single mothers did worse.

The second study, of Incredible Years, found that boys and children with the most depressed mothers showed the greatest improvement in conduct problems post intervention (Gardner et al., 2010). There were no predictive effects for other risk factors, notably teen or lone parent and very low income. The authors concluded that the intervention was successful at helping the most disadvantaged families, as well as the more advantaged.

Overall, these studies found little evidence of moderation by social disadvantage. For most ways of grouping by social disadvantage there were no differential effects. But where there was significant moderation, it was more likely to favour the more distressed or disadvantaged families.

Beyond cherry-picking

In a keynote talk at the European Society for Prevention Research, Frances Gardner (2013) pointed out that these results differ from two widely cited reviews of moderator predictor effects in parenting interventions (Lundahl et al., 2006; Reyno and McGrath, 2006), which found that more distressed and disadvantaged families did worse. This partly reflects which studies were (and were not) included in those reviews, and the failure of the reviews to distinguish between intervention "brands". Both Incredible Years and Family Check-Up pay considerable attention to strategies for engaging and motivating very disadvantaged, hard-to-reach families.

There is clearly a case for undertaking more moderator analyses but inevitably care is required when doing so. One problem is the lack of statistical power for such analyses: trials are powered to detect main effects, not effects for sub-groups. Another problem is the temptation to "cherry pick" findings without pre-specified hypotheses. It is not uncommon for evaluations to conclude that there is no main effect but then seek out effects for increasingly contorted sub-groups.

Looking forward, these and other issues could be addressed by pooling data from lots of trials and systematically testing pre-specified hypotheses. An added benefit of this approach is that the results can be generalised across different contexts and places. In the meantime, we can say that while certain family and parenting interventions clearly work overall, the evidence on whether they benefit some families more than others is mixed. This may reflect the fact that some interventions are better than others at helping disadvantaged families.

Nick Axford and Michael Little

Note

1.Parts of this editorial draw on the following presentation: Gardner (2013).

References

Gardner, F. (2013), "For whom do they work? Understanding moderators of outcomes in family and parenting interventions", keynote presentation at Fourth Annual Conference of the European Society for Prevention Research, Paris, 13 November

Gardner, F., Connell, A., Trentacosta, C.J., Shaw, D.S., Dishion, T.J. and Wilson, M.N. (2009), "Moderators of outcome in a brief family-centred intervention for preventing early problem behaviour", Journal Consulting & Clinical Psychology, Vol. 77 No. 3, pp. 543-53

Gardner, F., Hutchings, J., Bywater, T. and Whitaker, C. (2010), "Who benefits and how does it work? Moderators and mediators of outcomes in a randomised trial of parenting interventions in multiple 'Sure Start' services", Journal Clinical Child & Adolescent Psychology, Vol. 39 No. 4, pp. 568-80

Lundahl, B., Risser, H.J. and Lovejoy, M.C. (2006), "A meta-analysis of parent training moderator and follow-up effects", Clinical Psychology Review, Vol. 26 No. 10, pp. 86-104

Reyno, S.M. and McGrath, P.J. (2006), "Predictors of parent training efficacy for child externalizing behavior problems - a meta-analytic review", Journal of Child Psychology and Psychiatry, Vol. 47 No. 1, pp. 99-111