Preface: Walking backwards into the future

How do we know schools and the educational quality they promote? How do they know themselves and how do they arrive at judgements of quality along with those they serve? How do schools know about each other and learn about other qualities in schooling? Current conventions in school evaluation, too often accompanied by punitive and fearful consequences, are not always designed to answer these questions. Their focus is often on accountability, control and compliance, on competitive individualism among schools, and on often distorting assumptions of performance against universal, fixed criteria. Educational quality is too readily assumed to be apprehended by proxies – achievement, behaviour and employability. These proxies are, in and of themselves, not without meaning. The problem is when they are assumed to be the sum total of educational quality. In this volume, we examine an alternative to these approaches – developmental and negotiated approaches to school self-evaluation, with international examples from Norway, the Netherlands, Australia and others. These examples show both the promise and the challenges of school self-evaluation which respects context, school autonomy, shared learning and local control – a situated analysis of educational quality. Alongside international exemplars, we examine in detail the particular case of New Zealand (NZ) which has a distinctive, systemic alternative of developmental and negotiated approaches to school self-evaluation, and which retains considerable degrees of teacher autonomy with respect to curriculum.

Sometimes, to make an advance we have to look backwards to quality that was lost, dissolved in good but flawed intentions. We have to remind ourselves that the contemporary drift towards command and control systems of schooling is a policy innovation, a departure from many years experience of school autonomy and school-based curriculum development. We also need to remind ourselves that the drift is often based on good intentions, such as improving disparities for culturally and linguistically diverse communities, and the desire to find the most effective lever to achieve these good intentions. As we consider the question of how to identify and nurture educational quality we find ourselves rehearsing ideas hidden behind the veil of history, made inglorious by an intolerant victor in the battle for the control of schooling. But we do not need historical accounts. There remain models and exemplars of past practice that stand today as templates for advances in school evaluation and self-evaluation. At the heart of this volume is the case of NZ which retains much of this, supported by comparators in other countries.

The volume is organised into two sections. The first section provides an insight into the unique context of NZ from multiple perspectives, including that of Māori (indigenous) education. This is a case study of a developmental and negotiated view of school self-evaluation which is embedded in the national education system. It describes how a developmental and negotiated approach is adopted and understood at different levels of the education system and integrated into national policy, national evaluations of schools and national assessments. The NZ case reveals the potential of a systemic approach. We will see that it is possible for a national school quality assurance agency to base its evaluations on a negotiated approach to school evaluation which engages both internal and external participants in a collaborative, reasoned evaluation process. It is possible to experiment with national, standardised tests which meet rigorous psychometric criteria and is also experiential, local, and contextualised: we will read of an assessment design that claims to meet accountability, monitoring and developmental needs simultaneously. It is possible for schools to form partnerships with researchers that have at their heart the development of school-self-evaluation to address longstanding educational issues. These and other possibilities highlight the power of a systematic, but localised, approach to school self-evaluation and suggest ways for the field of programme evaluation to advance.

Challenges of a national approach to school self-evaluation are also highlighted in the NZ case. High trust in a school’s self-evaluation capabilities can mask a school’s capacity to review their own functioning in relation to national goals. Equally, changing governments and government priorities can make the meeting of national goals a taxing enterprise. However, the systemic approach in NZ appears to have produced generations of teachers, researchers and policy-shapers who are committed to school self-evaluation and who collectively subvert challengers, most noticeably in the requirement that national standards be based on teacher judgements across a portfolio of student work rather than on a single standardised test. Concluding the NZ case are comments by two leading international evaluators, who discuss both the unique contribution of the NZ case to the field of educational evaluation, and the application of the NZ case to other contexts.

The second section offers a sample of international perspectives on school self-evaluation which form a contrast to the NZ case. These chapters are designed to highlight important issues in the NZ case which are also of concern to other countries, and provide to some extent an implicit comment on its generalisability. For example, in the Dutch chapter, despite the high autonomy schools have, a key evaluation criterion of school quality used by their national school quality assurance agency is achievement results in national tests. Not surprisingly, a Dutch focus has been on supporting the school’s use of data, including achievement data, for school self-review. The chapters also demonstrate how established theories like democratic evaluation still have currency today, albeit these theories take on different forms in different educational contexts.

As in any volume, these chapters are by no means exhaustive of many exemplary school self-evaluation projects; nor can they be exhaustive of all involved in school self-evaluation. However, what is missing is a strong representation of those from indigenous and ethnic minorities who are most directly affected by any form of school evaluation. While some chapters focus on these ethnic groups, what are missing are ‘insider’ voices from these communities themselves – the students, parents and community leaders. We have one chapter with an ‘insider’ voice but more, and more diverse voices, are missing. Moreover, the chapters do not systematically address, or go far enough to address, the curriculum problem (i.e. what is meaningful to young people) or the teaching problem (the primacy of teacher judgement). Having these insights are necessary for further advances in this field of programme evaluation, not just in NZ, but also internationally.

An unintended but welcome derivative of the volume is that it challenges the school effectiveness/school improvement axis and invites the reader to look beyond models to how those involved in school self-evaluation understand their work. On the surface, some of the chapters might fall within the ‘school improvement’ camp, but a close reading reveals important differences, for example the idea that an intervention overwhelms context does not apply to all the NZ interventions, despite the seemingly ‘school improvement’ focus on designing educational interventions to improve achievement. Language and theoretical thinking can prevent the field of programme evaluation from engaging with the very people who are responsible for school self-evaluation, which is why this volume is written primarily by authors who would not call themselves programme evaluators. The chapters are by those who would call themselves developmental psychologists, government officials and research developers. Yet, in the descriptions of each chapter, it is clear that these authors are involved in programme evaluation more generally and developmental and negotiated ideas of school self-evaluation more specifically.

Nonetheless, it remains the case that the question of how we come to now schools has been overtaken in recent years by what is broadly known as the School Improvement Movement. The movement takes the school as the principal unit of change in order to leverage gains in the performance of the school system and/or the quality of education (quite distinct goals). Earlier movements in relation to schooling have focused on other units of change – for example on the quality of curriculum (Schwab, 1978; Tyler, 1949); or the quality of classroom interaction (Elliott, 1991; Smith & Geoffrey, 1968) on the teacher as a curriculum theorist (Stenhouse, 1975); on the conditions of learning (Bruner, 1977); or on the quality of assessment practices (Torrance & Pryor, 1998).

It is important to note that this is an array of options, and that the politics, economics and the prevailing logic of educational administration dictate the relative priorities of each (Glass, 2008; Lagemann, 2000; Ravitch, 2010). For example, the landmark study of educational equity (Coleman, 1966) found that economic and social effects on educational attainment overwhelmed school effects; and this was reinforced by the influential argument of Bernstein (2003) that owing to structural/socio-linguistic advantage built into the schooling of the middle classes ‘education cannot compensate for society’. These conclusions favoured political arguments focused on social and fiscal policy, acknowledging the primary responsibility of government to create economic equality as a prelude to equalising educational outcomes. They also favoured arguments for schooling to resist its role in reproducing social structure and to embrace a democratic curriculum ethic that made transparent social structure in a way that would ‘consciencentise’ young people (Apple, 1993). However, as governments have pulled back from such radical obligations for economic distribution they have passed the buck back to schools under the broad umbrella of assumption that equality of educational opportunity (i.e. not outcomes) will generate social equity. Here lies the birth of the School Improvement Movement and its influence on educational enquiry and theory.

It is worth, therefore, stating and critically reflecting upon the assumptions of the movement. The following are generalisations and do not apply in each aspect to all advocates of school improvement.

• That there is a singular entity – ‘a school’ – that can be ‘improved’

• That this entity has ‘goals’ that are meaningful throughout its constituencies

• That the sub-components of the entity are dependent rather than independent variables

• That intervention overwhelms context – that is that the logic of improvement applies from one setting to others

• That ‘improvement’ (do what you've always done, but do it better – avoid risk) is the same thing as ‘change’ (depart from the familiar – embrace risk)

• That improvement tends to be solution-driven rather than problem-driven

• That leadership is a primary determining variable in pedagogical performance

• That there is a hierarchy of effects from management to organization to teaching to learning

• That within that hierarchy there is a direct relationship assumed between what is taught and what is learned

Each or all of these may be true and may point to legitimate ways in which we come to know schools. However, each of these reflects one option among others – indeed, some critics of the school improvement movement argue that this list of items reflects a comprehensively flawed epistemology among alternatives (Elliott, 1998). In any event, no single item is a sine qua non. All can be – should be – questioned for their correspondence with what is deemed in practice to be educational quality and its shifting nature. As we have seen, there are alternative ‘units of change’ – that is the school itself (its philosophy and ethos, its curriculum ethic, its relationships) may and can be a variable dependent upon the summation of its diverse practices and values; we might focus on educational dilemmas rather than manufactured solutions; we sometimes acknowledge that there is only an indirect (and dynamic) relationship between teaching and learning. These assumptions are somewhat challenged in this volume, especially where we see examples of school review, improvement and self-evaluation initiatives based on negotiation and developmental effort between internal and external perspectives.

We need, therefore, a compass to guide us through some of the complexity we will encounter in the accounts presented in this volume. For this, we refer to Stake (2004) and his recasting of the putative ‘paradigm war’ between the qualitative and the quantitative. It has, for some time, been argued that these are not adequate in describing the differences in value position we encounter in the way we attempt to know schools and educational quality – or that the debate has been a distraction from more pressing social dilemmas. Stake offers, instead, a dynamic model of ‘criterial’ and ‘experiential’ or ‘episodic’ judgement. Criterial judgement involves generating criteria for making judgements of educational quality beyond the boundary of the case. If we want to make judgements about the quality of an educational interaction in a classroom, for example we may import a standard or some norm-based criterion which has been generated and perhaps verified elsewhere. As a standard or a criterion it functions irrespective of context, generalised, non-situated.

This is contrasted with ‘experiential’ or ‘episodic’ judgement which is made on the basis of criteria generated within the case boundaries – within, say, the classroom. Criteria for identifying quality, that is to say, are generated out of direct experience and in ‘episodes’ of classroom life. Quality, here, is situated. At issue is the question of control and professional integrity. Experiential understanding emphasises the autonomy of the teacher and the authenticity of their judgement. Stake goes on to argue for ‘binocular vision’ – the dynamic relation between the two that allows for ‘depth of field’ in the search for educational quality. This is exemplified in a number of accounts here, especially in the NZ case. We alert the reader to be aware of this as a lens through which to see and judge the quality of the accounts in this volume.

Mei Lai

Saville Kushner

Editors