Descriptive values and social justice

Abstract

One paragraph in House (1995) probably best captures what we suspect are the misunderstandings and the real differences between his position and ours. In responding to our previous writings, he says of us:Believing that evaluators cannot or should not weight and balance interests within a given context … suggests that human choice is based on non-rational or irrational preferences, not rational values, and that these things cannot be adjudicated by rational means. It implies a view of democracy in which citizens have irrational preferences and self-interests that can be resolved only by voting and not by rational means such as discussion and debate. A cognitive version of democracy, one in which values and interests can be discussed rationally and in which evaluation plays an important role by clarifying consequences of social programs and policies, is much better (p. 46).If we were to rephrase this paragraph to better fit our own beliefs, we would say:Evaluators can weight and balance interests, and when value conflicts are minimal, they should do so; otherwise they should not, if this means suppressing some of those interests. Human choice is based on non-rational, irrational, and rational grounds, usually all at once. While some of these choices can be adjudicated by rational means, the irrational preferences are usually far more pervasive and far less tractable than the rational ones, so successful rational adjudication, particularly by scientists, is the exception rather than the rule. In a democracy, citizens really do have irrational preferences and self-interests. Lots of them; and they are entitled to them. Sometimes they can be resolved rationally through discussion and debate, and when that is possible, it should be done. But more often than not, they can only be resolved by voting or some other method that voters legitimate. Evaluation can play an important role by clarifying consequences of social programs and policies, especially concerning outcomes that are of interest to stakeholders. But this is unlikely to lead to a so-called cognitive democracy in which values and interests are adjudicated rationally.When it comes to values, we think the world is a much less rational place than House seems to think. We agree with him that it is good to try to increase that rationality, within limits. When stakeholders differ about important values, it can sometimes make sense for the evaluator to engage them to see if they can find some common ground (although that task frequently extends well beyond the evaluator's training and role). When the needs of the disadvantaged look to be omitted from an evaluation, by all means the evaluator should make note of this and consider a remedy (descriptive valuing usually is such a remedy). But all this is within limits. If values were entirely rational, they would not be values by definition. So there is a limit to what the evaluator can hope to accomplish, and we suspect that limit is far lower than House's admirably high aspirations. Trying to impose a cognitive solution on value preferences that have a substantial irrational component is the problem democracy was originally invented to solve. To paraphrase House's (1995) last sentence: Disaster awaits a democratic society that tries to impose such a solution.

Citation

Shadish, W.R. and Leviton, L.C. (2001), "Descriptive values and social justice", Benson, A.P., Michelle Hinn, D. and Lloyd, C. (Ed.) Vision of Quality: How Evaluators Define, Understand and Represent Program Quality (Advances in Program Evaluation, Vol. 7), Emerald Group Publishing Limited, Leeds, pp. 181-200. https://doi.org/10.1016/S1474-7863(01)80072-7