Can we transform global education without transforming how we monitor progress?

Abstract

Purpose

This paper critically aims to review existing monitoring strategies of Target 4.7 of the Sustainable Development Goals and proposes an alternative approach for reporting country progress on relevant Target 4.7 themes. Since this target constitutes one of the most ambitious and transformative education targets there is considerable value in developing a comprehensive reporting and monitoring strategy.

Design/methodology/approach

The paper draws on key policy documents to clarify processes leading up to the definition and measurement of a global indicator for Target 4.7. It also discusses limitations associated with the current reporting and measurement strategy.

Findings

The paper finds that the current monitoring approach to Target 4.7, based on an existing reporting mechanism for the 1974 Recommendation concerning Education for International Understanding, Co-operation and Peace and Education relating to Human Rights and Fundamental Freedom, is unfit for purpose and needs to be overhauled. The current process for revising the 1974 Recommendation is unlikely to result in a new monitoring strategy that would address existing weaknesses in the current strategy.

Originality/value

To the best of the authors’ knowledge, a critical review of measurement, reporting and monitoring strategies of Target 4.7 has not been undertaken. Also new in this paper is the proposed global observatory of Target 4.7 policies, practices and initiatives, which, if established, would work to: (1) create a more dynamic and informative monitoring infrastructure; (2) foster peer learning among countries; and (3) identify notable strategies of national, regional and international action in relation to Target 4.7.

Citation

Benavot, A. and Williams, J. (2023), "Can we transform global education without transforming how we monitor progress?", Journal of International Cooperation in Education, Vol. 25 No. 1, pp. 42-61. https://doi.org/10.1108/JICE-07-2022-0020

Introduction

Midway through the implementation of the 2030 Agenda for Sustainable Development, the United Nations (UN) Secretary-General António Guterres called for a high-level summit focused on transforming education, which was held in September 2022 at UN Headquarters in New York. The Summit, which brought together leaders, teachers, students, civil society and other partners, secured the commitments of more than 130 countries to prioritize and reboot their education systems and accelerate action to end the learning crisis, especially among children from economically vulnerable communities [1]. In the run-up to the Summit, the UN Secretary General argued:

Education systems today are at a crossroads. The COVID-19 pandemic had a devastating impact on the learning opportunities of hundreds of millions of children and young people, exacerbating a pre-existing learning crisis and growing inequalities. Education must be transformed in order to equip learners with the knowledge, skills and values they need to address new challenges and thrive in our rapidly changing world (UNESCO, 2022).

This paper focuses on the monitoring of Target 4.7, arguably one of the most ambitious and transformative targets of the UN 2030 Agenda for Sustainable Development. Target 4.7 calls on countries to rethink the purposes and content of their educational systems in line with specific global and humanistic themes – including, for example, human rights, gender equality, peace and nonviolence, global citizenship, cultural diversity and sustainability. This paper argues that national commitments to transform education along these lines are unlikely to gain momentum so long as current reporting and monitoring strategies of country progress remain encased in conventional practice [2]. To that end, we critically discuss existing reporting and monitoring approaches to Target 4.7. For example, do they provide policy makers with relevant information to rethink and redesign educational policies and engage in peer learning? Do they capture country progress (or the lack thereof) in mainstreaming Target 4.7 in both formal and nonformal education as well as in teacher preparation and student assessment? Do they provide useful comparable information of how countries are transforming their education systems? To the best of our knowledge, previous research has not addressed these questions.

Admittedly, developing a monitoring strategy to capture the breadth of an ambitious target like Target 4.7 is challenging, especially given the reliance on quantitative indicators. Nevertheless, we find current monitoring efforts to be inadequate and limited. We propose an innovative alternative strategy: namely, building up a crowd-sourced global observatory of Target 4.7 policies, practices and efforts. Such an alternative reporting and monitoring approach would, if implemented, help to: (1) create a more dynamic and comprehensive platform to share relevant information and data; (2) foster peer learning among countries; and (3) identify notable national, regional and international initiatives in relation to Target 4.7. It would also establish stronger links and synergies with other elements of the Sustainable Development Goals (SDGs). We draw on our experiences of having worked in and collaborated with international organizations in education, mainly United Nations Educational, Scientific and Cultural Organization, to articulate and advance these proposals.

Education goal setting and the transformative potential of target 4.7

The global expansion of education systems and the internationalization of education models are salient features of the past century (Meyer, Ramirez, & Soysal, 1992; Benavot & Resnik, 2006). Members of the international community, both governmental and nongovernmental actors, have consistently argued that education serves crucial cultural, political and economic purposes, in addition to socioemotional and developmental ones for individual learners (Bromley, 2010; Chabbott, 2010; Furuta, 2020; Mundy, 1998). Attention to the consequences of education, at both the societal and individual levels, has been integral to the work of the UN, especially the adoption and implementation of international conventions securing the right to education for all children and youth. The abiding promise of expanding educational opportunity to all has been a core principle for UNESCO since its founding in 1947. This is apparent in its flagship programs and capacity building activities in education and lifelong learning – for example, the Fundamental Education Program, Education for All, Eradicating Adult Illiteracy (Jones, 1988, 1999; Meyer, Strietholt, & HaLevi, 2017; Elfert, 2018). UNESCO’s work in fostering peace and international understanding through education is famously inscribed in its Constitution, “Since wars begin in the minds of men, it is in the minds of men that the defences of peace must be constructed.”

A noticeable shift in recent decades has been the expanding nature and specificity of the educational vision legitimated by international actors, along with increases in the number and types of actors involved in legitimating and exacting these visions (UNESCO, 2021a; Ball, 2012). From Jomtien (1990) to Dakar (2000) and then to Incheon (2015), members of the global education community – government representatives, international multilateral and bilateral agencies, civil societal organizations, foundations and private companies – have committed themselves to a set of time-bound goals and targets. Not only did the global agenda in education and lifelong learning expand to encompass more levels and aspects of education, it also included calls for new reporting mechanisms, consisting of indicators and benchmarks that measure the extent to which different actors, particularly national governments, make good on their commitments. The shifting tone and ambition of the global education community reached an apex in 2015, when 193 UN Member States adopted the 2030 Agenda for Sustainable Development, with its unprecedented 17 SDGs and 169 specific targets. The global goal on quality education, SDG 4, represents the “most far-reaching commitment to quality and equity in education” ever adopted (Antonia Wulff, 2020).

While many SDG 4 targets raise the bar of ambition, Target 4.7 stands out for its transformative potential as it calls on member states, by 2030, to

ensure all learners acquire knowledge and skills needed to promote sustainable development, including among others through education for sustainable development and sustainable lifestyles, human rights, gender equality, promotion of a culture of peace and non-violence, global citizenship, and appreciation of cultural diversity and of culture’s contribution to sustainable development. (https://sdgs.un.org/goals/goal4)

Through its emphasis on global, cultural and humanistic themes – some long-standing; others more recent – Target 4.7 appeals to educational leaders to rethink the purposes and contents of their systems so that they may better contribute to more equitable, peaceful and sustainable societies. Implementing Target 4.7-informed policies would help transform schools and the communities in which they are embedded into sites that prioritize education for sustainability, as well as education for peace, human rights and global citizenship. Employing contextually sensitive pedagogies that promote Target 4.7 principles (e.g., place- and nature-based learning, social and emotional skills, cooperation in projects, intercultural communication) would help empower learners to critically assess global challenges, construct alternative responses and innovative strategies of collective action, and motivate learners to advance sustainability and collective wellbeing. The lifelong learning perspective embedded in SDG 4 also supports Target 4.7 by linking policies and principles in both formal and nonformal education and creating intergenerational synergies.

The challenges of measuring country progress along global education targets

The global targets in education (SDG 4) are part of an increasingly globalized context in which education is framed as a global public good (Mundy et al., 2016); where national governments have agreed, on paper at least, to assume responsibility for advancing those targets; and in which multiple actors – state and nonstate, global, regional and national – are committed to their enactment and achievement. Concurrently, new types of educational data, compiled at different levels through advances in computing, are being used for accountability purposes, in line with neoliberal approaches to the management of education systems (Meyer & Benavot, 2013).

The widespread use of outcome measures in accountability models has sparked considerable controversy. Proponents argue that the measurement of student outcomes and related factors can lead to improvements in education systems. The Organization for Economic Co-operation and Development says about its Program for International Student Assessment (PISA), for example:

Every three years PISA results show what’s possible in education, and countries are keen to learn from each other’s successes… In a hyper-connected world, standards of excellence are no longer fixed at national borders. Countries can learn from each other. Policies can be tweaked and tailored to work in different contexts. Every education system and economy can improve student performance and make its school system more inclusive at the same time. PISA shows that those outcomes are not only desirable, but they are attainable. [3]

Critics have highlighted undesirable consequences of accountability regimes based on such assessment frameworks. These include, for example, inappropriate homogenization of policies and policy-making; the narrowing effects of test-based accountability systems on curriculum; the undermining of teacher professionalism and student engagement; the tendency toward convergence of the multiplicity of educational goals around a limited number of quite narrowly focused indicators; increased competition among national education systems and reforms that may not lead to improvement; and so forth (Sahlberg, 2014; Rizvi & Lingard, 2010; Meyer & Benavot, 2013; Sjøberg, 2019).

Although international learning assessments (ILSAs) were not constructed to enforce compliance at the national level, national education systems have responded to the models and outcomes of ILSAs (UNESCO, 2019). Strategic attempts by countries to raise their PISA scores has had pernicious unintended effects, as noted above. By contrast, creating monitoring strategies to measure progress toward international targets, to which countries have committed themselves, especially by international entities with little to no actual enforcement power, is less prone to deleterious effects than, say, national accountability frameworks that reward compliance to policy decisions among schools and school districts.

We leave for others to ponder whether agreeing on and monitoring global education goals is a worthy and valuable endeavor. UN member states initiated a process that led to the adoption of specific, time-bound goals and targets as well as their periodic monitoring. Here our purpose is to critically review existing reporting and monitoring activities and determine whether they address the purposes for which they were established, mainly in reference to Target 4.7.

From the outset, the vision of the 17 SDGs and 169 Targets were matched by a multiplicity of reporting and measurement challenges. The Inter-agency and Expert Group on SDG Indicators (IAEG-SDGs), composed of UN Member States and including regional and international agencies as observers, was tasked with developing – and later implementing – a global indicator framework for all SDG targets. Members of the IAEG-SDGs spent nearly two years considering the feasibility of alternative measurement strategies. After lengthy deliberations they drafted a global indicator framework composed of 230 specific global indicators which was reviewed, including refinements on several indicators, at the 48th session of the UN Statistical Commission in March 2017. After some revisions, the proposed global indicator framework was adopted by the UN General Assembly on 6 July 2017 (A/RES/71/313).

In what many observers at the time considered a surprise development, countries decided that participation in the SDG indicator framework would not be obligatory. The final approved resolution states that the UN “adopts the global indicator framework…as a voluntary and country led instrument” … “complemented by indicators at the regional and national levels, which will be developed by Member States” (United Nations, 2017). It further stresses that “official [national] statistics and data…constitute the basis needed for the global indicator framework” and “recommends that national statistical systems explore ways to integrate new data sources…to satisfy new data needs of the 2030 Agenda of Sustainable Development.” And if anyone had any doubts as to who was in the driver’s seat of this “follow up and review” approach, the document “urges international organizations to base the global review [of the SDGs] on data produced by national statistical systems and, if specific country data are not available for reliable estimation, to consult with concerned countries to produce and validate modelled estimates before publication.” [emphasis added] Notably, the term “monitoring” never appears in the UN Resolution.

It is also noteworthy that while Target 4.7 emphasized the outcomes of curricular interventions and thematic learning experiences on learners of all ages, focusing as it does on the knowledge, skills, attitudes and values [4] that younger and older learners would need to acquire to further sustainable development, sustainable lifestyles and other societal purposes, the global indicator (4.7.1) for Target 4.7 emphasized educational inputs. Countries were expected to report on the:

Extent to which (i) global citizenship education and (ii) education for sustainable development are mainstreamed in (a) national education policies; (b) curricula; (c) teacher education; and (d) student assessment. (UN Statistics Division, n.d.)

The decision to measure inputs rather than learning outcomes in reference to Target 4.7 stemmed from the simple fact that then, as now, there is no internationally recognized assessment platform or comparatively validated measurement scale that captures the multiple content domains referenced in Target 4.7. The selection of an input-focused global indicator assumes that countries that succeed in mainstreaming education for sustainability or global citizenship in their policies, intended curriculum, teacher education programs and learning assessments are more likely to produce learners who have acquired the kinds of knowledge, skills and competences relevant to 4.7 domains. The fact that comparative evidence bearing on this assumption is tenuous at best has done little to assuage advocates of input-oriented policy reforms. The global indicator 4.7.1, which highlights the mainstreaming of Education for sustainable development (ESD) and Global Citizenship Education (GCED) in education systems, provides a feasible pathway for countries to report on Target 4.7 progress, as we shall discuss below.

Once the UN General Assembly ratified the SDG global indicator framework, it fell to UNESCO, the lead UN agency in education, to oversee the data collection and measurement of all SDG 4 targets. UNESCO empowered its Institute for Statistics (UIS) to establish various form and mechanisms to develop clearly articulated measurement strategies for each SDG 4 target – specifically, the Technical Cooperation Group (https://tcg.uis.unesco.org/tcg-composition/), the Global Alliance on Measuring Learning (https://gaml.uis.unesco.org/) and eventually task forces to explore indicators for Target 4.7 (https://gaml.uis.unesco.org/indicator-4-7-4/and https://gaml.uis.unesco.org/indicator-4-7-5/). There were no easy and obvious solutions to operationalize the global indicator 4.7.1. After a series of internal discussions, UNESCO proposed to redesign an already existing quadrennial reporting mechanism of a nonbinding recommendation (see below) to serve as the basis for monitoring 4.7.1 and 12.8.1 (which are identical) as well as 13.3.1, which focuses on countries integrating climate mitigation, adaptation, impact reduction and early warning into primary, secondary and tertiary curricula (https://tcg.uis.unesco.org/wp-content/uploads/sites/4/2019/08/TCG6-REF-14-Proposal-for-monitoring-of-SDG-indicators-4.7.1-12.8.1-and-13.3.1.pdf). Although it was a rather efficient solution to a complex monitoring challenge, it did little to improve the quality of reported information and country coverage.

Repurposing an older reporting mechanism to fit new needs

The 1974 Recommendation concerning Education for International Understanding, Co-operation and Peace and Education relating to Human Rights and Fundamental Freedoms (hereafter referred to as the “1974 Recommendation”) was adopted by the UNESCO General Conference on November 19, 1974. The initial motivation behind this Recommendation can be traced to an invitation to UNESCO by the UN to issue an international declaration of principles related to education for peace and international understanding (1960-62). Beginning in the mid-1960s, UNESCO convened several meetings to consider how best to feature the relations between education, peace and human rights. Years later, and with the strong advocacy of the Mexican ambassador to UNESCO, an agreement was reached in early 1974, which paved the way for a large majority of countries to agree to adopt the (nonbinding) 1974 Recommendation (Boel, 2022).

The 1974 Recommendation establishes seven guiding principles [5] for educational authorities and professionals to infuse “all stages and forms” of education with the aims of the charter of the UN, the Constitution of UNESCO, and the Universal Declaration of Human Rights. The 1974 Recommendation highlights the role of education in furthering the “full development of the human personality and…strengthening of respect for human rights and fundamental freedoms” (Paragraph 2 of Article 26 of the Universal Declaration of Human Rights) and in eradicating “conditions that threaten human survival and well-being” (UNESCO, 2021b). The 1974 Recommendation defines education in broad terms:

The word ‘education’ implies the entire process of social life by means of which individuals and social groups learn to develop consciously within, and for the benefit of, the national and international communities, the whole of their personal capacities, attitudes, aptitudes and knowledge. This process is not limited to any specific activities. (UNESCO, 1974)

As articulated in UNESCO’s constitution, member states are required to report every four years on their progress in implementing the 1974 Recommendation [6]. Since the 1970s, UNESCO has facilitated seven “consultations” about country implementation, the most recent having been carried out between October 2020 and March 2021 (UNESCO, 2021c). For each consultation, UNESCO drafts a survey, which is approved by member states and then sent to National Commissions to UNESCO, who circulate it to appropriate education authorities in each country (McEvoy, 2017). The survey typically includes both open-ended and closed-response questions; in some instances, member states are also asked to provide additional documentation.

One might ask: What is the relationship between the 1974 Recommendation and Target 4.7 and does it make sense to repurpose the reporting mechanism of the former to serve as a measure of progress of the latter? Reading the 1974 Recommendation some five decades after it was adopted, one is struck by its abiding relevance. Although the contours of the international economic, political, social and environmental landscape have changed dramatically in the intervening decades, much of the tone, substance and purpose of the Recommendation –namely, to mobilize education to address threats to global peace and human survival – remains valid today. (In recognition of the changing landscape, UNESCO has undertaken a significant revision to the Recommendation, which is expected to be adopted in late 2023 (UNESCO, 2021b).

Although an in-depth history of the making of Target 4.7 has yet to be written, it was during the Global Education Meeting in Muscat, Oman (May 2014) that the Japanese and South Korean delegations arrived at a behind the scenes accommodation that enabled “education for sustainable development” and “global citizenship education” to be embedded in a single emergent target (personal communication, Utak Chang, May 2022). Until this meeting, the two delegations had worked strenuously behind the scenes to have their lead concept prioritized. Once these two competing – albeit somewhat overlapping – notions were placed under the same banner, other historical priorities in education, especially those recognized in the 1974 Recommendation, such as human rights, gender equality, peace and nonviolence could also be considered for inclusion. For entirely different reasons, UNESCO also sought to have the value of culture and cultural diversity recognized in this emergent target, since it had gained little recognition in any of the other 169 targets being negotiated at UN headquarters. In sum, several key terms in Target 4.7 were drawn directly from the 1974 Recommendation, whereas other aspects are quite new. On its face, a reporting mechanism to survey country implementation of principles in the 1974 Recommendation would have some validity in measuring and monitoring aspects of Target 4.7.

That said response rates to surveys of the Recommendation’s implementation varied over time and by region. In years before 2015, only 20-30% of Member States responded. After 2015, when the survey was expanded to capture the new domains of Target 4.7, around 40% of member states responded (see Table 1). Interestingly, response rates to the 1974 Recommendation surveys have been nearly as high or higher than surveys of country implementation of other UNESCO standard-setting instruments (Table 2).

Key findings from recent country reports regarding the 1974 Recommendation

During the 7th Consultation (2020/2021), countries were also asked to convey their views on revising, updating and expanding the 1974 Recommendation in the light of evolving threats to global peace, planetary sustainability and international pandemics. Upon securing positive responses from member states, UNESCO embarked on a multiphase revision process of the 1974 Recommendation, which unfolded in parallel to the Futures of Education initiative (UNESCO, 2021d) and the planning of the Transforming Education Summit at UN Headquarters in September 2022.

The revision of the 1974 Recommendation is intended to: “ensure the instrument--its framing and technical guidance--is fit for purpose and able to better inspire the design of relevant policies; and strengthen the resolve of Member States to implement the guiding principles contained in the 1974 Recommendation, and which are echoed in the 2030 Agenda” (UNESCO, 2021b).

A Consolidated Report from the 7th Consultation summarized findings from 71 country survey responses and national reports (four additional responses were received after the analysis had been completed). Results are ambiguously positive. Member states reported generally high levels of integration of the seven guiding principles in national and subnational laws and legal frameworks, curricula, teacher education and student assessment, but noted variations in integration across domains (legal, curricular, etc.) and countries (UNESCO, 2021c).

Despite the strong commitment to the guiding principles laid out in the 1974 Recommendation, the concluding section of this consolidated report highlights more nuanced findings.

1. Levels of integration of ESD and GCED in some education systems are considerably higher than in others.

2. Integration of the guiding principles is especially high in official curricula and in more than half of cases the mainstreaming of ESD and GCED is extensive. Although mainstreaming in teacher education is almost as high [as in national and subnational laws and policies], it is more likely to be partial (50%) than extensive (42%).

3. Integration is more extensive at the primary, secondary and tertiary levels than at preprimary or nonformal education across all domains of education systems (i.e., laws and policies, curricula, teacher education and student assessment).

4. Topics related to learning to live together are more often covered than those relating to learning to live sustainably. The topics least often included are climate change and sustainable consumption and production.

5. Half of all reporting countries indicated that some or all of the guiding principles are included in adult education programs. In addition, in some countries national policies related to adult education aim to ensure the development of sustainability and citizenship skills. (UNESCO, 2021c, p. 4ff)

Among the 40% of UNESCO member states who responded to the survey, almost all recognize the importance of the principles of the 1974 Recommendation and appear to be making some – albeit uneven – progress toward including them in the laws, policies, curricula and teacher training. (Student assessment is less clear.) That said, it is nearly impossible to assess from member state responses the extent to which, and how, the guiding principles are concretely manifest in local schools and classrooms. It is also difficult to determine what, if anything, learners take away from their exposure to these educational experiences and curricular interventions.

Interestingly, the 2021/22 Global Education Monitoring (GEM) Report, in its key messages section on monitoring Target 4.7, notes:

Only ten countries reported fully reflecting or including the guiding principles of UNESCO’s 1974 Recommendation related to Target 4.7 in all four domains, from policies to assessment, in relation to global citizenship education and education for sustainable development. (UNESCO, 2021a, p. 313).

The 2021/22 GEM Report provides more conclusive evidence of country mainstreaming in the four domains of 4.7.1 (see Table 6 in the Annex). Aggregated data at the regional and subregional level (e.g., Africa, West Africa) indicate high levels of mainstreaming in all four domains. At the global level, and keeping in mind the limited response rates, ESD and GCED are surprisingly reported as mainstreamed in 100% of educational policies and frameworks, 84% of the world’s curriculum, 86% of the teacher training and 86% of student assessment.

Drawing on other data sources, the 2021/22 GEM shows the paucity of systematic information for thematic indicators of Target 4.7 (4.7.2-4.7.5). In reference to the percentage of students and youth with an understanding of human immunodeficiency virus (HIV)/acquired immunodeficiency syndrome (AIDS) and sexuality, global citizenship and scientific literacy, most regional cells in the aforementioned table are blank. Where responses are reported, numbers are low. For example, 36 is the median percentage of students and youth in Sub-Saharan Africa found to understand HIV/AIDS and sexuality. Other regions report much lower levels of understanding. In terms of student/youth understanding of global citizenship, North Africa and Western Asia show the highest median figures at 33%. Figures for scientific literacy (56%) are shown only for Europe (UNESCO, 2021a, pp. 449–453).

Overall, current reporting strategies for Target 4.7 raise more questions than provide answers about the quality, accuracy and meaning of country responses. There is much room for improvement. For example, there is little indication that UNESCO or UIS is following up with countries that either provide incomplete data on 4.7.1 or the reasons that 60% of member states fail to report any relevant information. The picture that emerges from country self-reports suggests a disconnection between their reported high levels of commitment to Target 4.7 (and to the principles of the 1974 Recommendation) and the paucity of evidence as to how these commitments are translated into classroom realities and student learning. The survey tells us little how the mainstreaming of the 1974 Recommendation principles in country policies, curriculum, teacher training and assessment influences learner understandings of ESD and GCED. The existing reporting mechanism for Target 4.7, which relies entirely on country self-reports, provides a weak, incomplete and anecdotal basis for determining whether schools are enabling learners to acquire knowledge, skills, values and dispositions conducive to sustainable development, peace/nonviolence, cultural diversity and global citizenship, to say nothing of climate mitigation and adaptation.

The need to improve the monitoring of the 1974 Recommendation and Target 4.7

It would be fair to say that over time the survey instruments employed by UNESCO to monitor country implementation of the 1974 Recommendation have improved. Improvements can be seen in the design, question formulation and response scales, especially in post-2015 surveys that included new domains related to Target 4.7. Survey responses generally indicate that, at least on paper, countries are committed to implementing the provisions of the 1974 Recommendation and to key 4.7 themes, although the extent to which these commitments are realized in practice remains unclear.

Given the decision to use country responses to the 1974 Recommendation survey as the basis for monitoring Target 4.7, it is worth considering the limitations or weaknesses of this reporting approach. For example:

1. Almost all information conveyed by countries is self-reported. Verification is weak or nonexistent, while concurrently, there is “peer pressure” to report progress.

2. Key survey results are reported in rather abstract and general terms. They lack specificity, thereby undermining the effectiveness of the monitoring strategy and the broader value of findings and policy takeaways for different target audiences, both governmental and nongovernmental.

3. While many tables and items in the survey are quite detailed, most of the approximately 400 cells in the tables are completed with simple responses: “yes,” “no,” “unknown/not applicable.” It is impossible to determine gradations within “yes” responses. Thirteen open-ended questions ask for elaboration, but the framing of these questions allows for considerable variation in response, thus undermining comparability. For example, the open-ended question regarding student assessment states:

Please describe in which ways GCED and ESD are covered in student assessments and examinations. Indicate whether students are assessed only on their knowledge of and skills related to the topics being tested or also on their values, attitudes and/or behaviours (UNESCO, n.d.).

1. Conversely, when countries report high levels of progress, it becomes difficult to identify specific areas where improvement is needed or would be meaningful.

2. Similarly, clear descriptions of the actual implementation of the seven guiding principles in different educational contexts are lacking. The range of responses – “implemented”, “partially implemented”, “not implemented” – do not lend themselves to understanding what was implemented, how and why, or what was learned.

3. Given low survey response rates and regional variation, it is difficult to assess the extent of implementation globally. While low response rates may be typical among UNESCO member states in related surveys, such response rates to the 1974 Recommendation survey are not just an internal UNESCO matter. Country responses to the two post-2015 surveys (the 6th and 7th consultations) are reported to UN authorities and used to track progress on the global indicator 4.7.1, in which members of the broader international community also have a stake.

4. Even if response rates were to increase, other issues in survey responses would likely remain:

   • There would be missing information on key survey questions of interest, since many countries submit incomplete reports. As the survey instrument itself becomes increasingly complex, missing responses are more likely because they require access to and understanding of substantial types of information and data.

   • There is little way to independently validate the quality of country-provided information since countries are rarely asked to provide supporting documentation.

   • By and large details are lacking about country implementation of specific principles in the 1974 Recommendation or Target 4.7 topics at each level of formal education or in adult nonformal education.

   • It is nearly impossible to discern relevant patterns in countries with multiple governing jurisdictions – for example, in federal countries where states, province or urban municipalities oversee policy implementation.

   • Also lacking are details about relevant programs initiated by civic society or private entities such as Non-governmental Organizations (NGOs), faith-based organizations, community learning centers, private companies or online programs located outside national boundaries.

   • Information on government efforts to target marginalized populations (e.g., language minorities, indigenous groups, rural populations) in these areas is not provided.

Indeed, a recent GEM Report (2022) blogpost [7] entitled “Target 4.7 – What is at stake for monitoring progress on education for global citizenship and sustainable development?” noted:

In the past low response rates and uneven report quality undermined the monitoring value of such information. Country reports need to be complemented by a more systematic and rigorous approach to all aspects of the global indicator and target itself. (GEM Report in World Education blog, February 2, 2022)

The GEM Report (2022) authored blog argued in favor of a new monitoring approach to Target 4.7:

We recommend that there be a new global mechanism to monitor the content of curricula and textbooks to help measure progress towards this target. This would require close collaboration between national education ministries and regional or international organizations to ensure that the quality of the information is good and that the process is country-led. The mechanism could also cover other aspects of national policies, including teacher education…and learning assessments. It is an essential step if we are to know more about the types of things children are hearing when they step into class each day and have an indication of the types of values they might end up having instilled in them by the time they leave at the end of school. (GEM Report in World Education blog, February 2, 2022)

Different education sectors, similar monitoring challenges

The contributions of educated adults and adult education to the achievement of the SDGs, including Target 4.7 can be considered from several angles (UIL, 2016). For example, the political, cultural and environmental worldviews of young learners are formed at home and in local communities. Adults serve as models for emulation; they also nurture constructive experiences, convey knowledge and anchor values in areas such as peace, tolerance and sustainability. Literate adults are more likely to be active citizens in their communities and demand more enlightened or equitable policies of their political leaders. More educated workers are more likely to see the value of acquiring “green” skills, participate in green training programs and consider the long-term sustainability consequences of their employment decisions (Anderson, 2013).

The seventh International Conference on Adult Education (CONFérence INTernationale sur l’Education des Adultes (CONFINTEA) VII), which took place in June 2022 in Marrakech, Morocco, highlighted the ways in which lifelong learning and adult education can contribute to climate action and sustainable development. CONFINTEA VII participants adopted the Marrakech Framework for Action, which argues that adult learning and education “gives…adults an understanding of [climate] issues, raises their awareness, equips them with knowledge and agency needed to adapt to and counter climate change…. Adult learning and education (ALE) can play an important role in empowering adult and older citizens so that they become role models for children and change agents at local, national and global levels…ALE institutions themselves can act as models for green transition in society by greening their curricula, facilities and management.” (UIL, 2022c, p. 4).

And yet, of all the subsectors of lifelong learning, validated and comparable data about adult access to and participation in nonformal education are lacking in most regions of the world (UIL, 2017) [8]. The Adult learning and education (ALE) sector continues to be the least visible among national policy makers, regional authorities and international agencies, in large part due to the paucity of comparable data on adult nonformal education (Benavot, 2018). This invisibility is also apparent among advocates of Target 4.7, who tend to focus on reform efforts and initiatives that influence the hearts and minds of children and youth, and not adults.

During CONFINTEA VI in 2009, the Belém Framework for Action (BFA) identified ALE as a critical component of the right to education. The BFA specified five areas for action – policy; governance; financing; participation, inclusion and equity; and quality – and called on UNESCO to prepare a Global Report on Adult Learning and Education (GRALE), which would regularly monitor ALE developments in the aforementioned areas. The recently launched Fifth GRALE report (UIL, 2022a) reported that funding for lifelong learning remains inadequate and that those who are most likely to benefit from adult education are least likely to gain access to it. While there is some evidence of improvements in ALE policy, governance, finance, participation, equity and quality, these improvements have been slow and uneven (UIL, 2022a). GRALE 5 concludes that ALE’s potential role in enabling adults to better address local and global challenges and to contribute to the ambitious SDG agenda will go unrealized, if existing funding levels and political commitments to ALE are not significantly increased.

The GRALE report series serves as one of the most visible sources of information on the status and progress of ALE worldwide. However, from a monitoring perspective, GRALE reports lack firm grounding in empirical research and crossnational evidence. Each GRALE report describes select initiatives undertaken by member states and provides illustrative information rather than comprehensive evidence and systematic data. Largely conceptual and advocacy in tone, the GRALE reports have not done as much as they potentially could to platform evidence-oriented policy making and to counter the marginality and continued invisibility of the ALE subsector.

Many of the weaknesses in the monitoring of the 1974 Recommendation, mentioned above, are equally valid in relation to the GRALE report series. Data are based on country self-reports and cannot be independently validated. Most countries provide positive responses to survey questions and are quick to note increases in ALE participation rates over time, without verifiable evidence. Few mechanisms are in place to check the quality, validity and reliability of country supplied information. Challenges haunt the measurement of outcomes in each of the five areas of action. While GRALE reports have highlighted the prevalence of funding challenges and inequalities in access and participation, the impact of the reports on shifts in policy and practice remains unclear (Boeren & Rubenson, 2022). Indeed, the lack of financing and prioritization of adult education within the global education ecosystem suggests another commonality between adult learning and Target 4.7. Quite a few policy messages in the GRALE reports could have been written regardless of the survey evidence reported. In our view, rethinking current monitoring and reporting strategies for ALE and of Target 4.7 hold much in common.

National benchmarks to evaluate progress in SDG 4

In the runup to the adoption of the 2030 Agenda for Sustainable Development, the UN Secretary General circulated a major “Synthesis Report” (United Nations, 2014), which calls on countries – indeed, all stakeholders – to embrace a culture of “shared responsibility”, one based on agreed universal norms, global commitments, shared rules and evidence, collective action, and benchmarking for progress. [emphasis added]. Inspired by this notion, the international education community in the Education 2030 Framework for Action called on countries to define “appropriate intermediate benchmarks” (say for 2025), which would “set for each target…quantitative goalposts for review of global progress vis-à-vis the longer-term goals. Such benchmarks should build on existing reporting mechanisms, as appropriate. Intermediate benchmarks are indispensable for addressing the accountability deficit associated with longer-term targets.” (UNESCO, 2016, p. 35).

These national determined values or benchmarks would, in theory, enable the monitoring of SDG 4 progress in a more context-specific manner and would specify countries’ nationally determined contributions to a common education goal. By mid-2022, three in four countries had committed to 2025 and 2030 target values for at least some of the seven benchmark indicators (UIS and GEM Report, 2021, 2022). Notably, none of these seven benchmark indicators refer to Target 4.7. To the extent that this benchmarking process strengthens policy shifts and implementation on the ground, then the absence of Target 4.7 benchmarks means that we probably cannot expect much progress in this area, certainly for the foreseeable future. Whatever progress in SDG 4 may be achieved by securing commitments to country-specific benchmarks, this effort will not incentivize countries to link their policies and plans to clear benchmarks in relation to education for sustainability, peace, gender quality, cultural diversity and global citizenship – the crux of Target 4.7.

Evaluating alternative monitoring approaches to target 4.7

One can point to several alternative, in some cases complementary, approaches to monitoring Target 4.7. One approach begins with the current formulation of Target 4.7 and focuses on constructing outcome-oriented indicators of levels of knowledge, skills, attitudes and values acquired by learners through different modes and levels of education. Researchers who have embarked on such a strategy, combine data from items on different international assessment platforms (Sandoval-Hernández, Isac, & Miranda, 2019; Sandoval-Hernández & Carrasco, 2020). A second approach focuses on the global indicator 4.7.1 and establishes more validated and comparable measures of the contents of major inputs such as subject syllabi, textbooks, teacher training and classroom pedagogy, all of which structure learning experiences in relevant 4.7 domains. An example of this approach is seen in a recent study by the Asia Pacific Center of Education for International Understanding (APCEIU), which explored shared content domains in global citizenship competence in the Asia-Pacific region based on an analysis of subject syllabi in two dozen countries (Benavot et al., 2022). This approach could be expanded to measure and monitor the integration of guiding principles and content domains referenced in the revised 1974 Recommendation in policies, curricula, teacher education and assessments.

We think that there is value in establishing a set of criteria that could be used to assess the strengths and weaknesses of each approach. In our view, reporting and monitoring systems for Target 4.7 need to be robust and fit for multiple purposes. They should provide comprehensive, reliable and valid data of sufficient detail and comparability to support: presentation and understanding of the current status of practice, research into effective and ineffective practices over time in different national contexts, and knowledge-sharing across countries. With this in mind, the following questions could be considered in assessing alternative reporting and monitoring approaches.

Does the proposed monitoring approach…

1. expand country coverage of relevant information related to Target 4.7, the global indicator 4.7.1 or a revised 1974 Recommendation?

2. provide independently validated and detailed information about specific ESD and GCED topics and levels to assess country and regional progress?

3. provide a basis for enhanced comparability over time?

4. encourage countries to deepen their commitment to implement while engaging in peer learning and information sharing?

5. allow for the construction of innovative measures and indicators?

6. provide different kinds of information and data to a wide array of interested parties and stakeholders?

In our view, specific approaches that meet or exceed most of these criteria are more promising than others.

Towards a dynamic international observatory of target 4.7

Since 2015 the international reporting and monitoring landscape has shifted. Revisions to the international development agenda are being led by country representatives, not international agencies. Country participation in the global SDG indicator framework – expert-defined, and admittedly complex – continues to be voluntary. The term “monitoring” is itself contested, and rarely appears in official documents. Given these circumstances, we might reasonably ask whether any serious strategy to compile and review validated qualitative and quantitative information pertinent to Target 4.7 is feasible at this point? In the absence of clear incentives for countries to provide validated information on Target 4.7 implementation, most countries have decided to either go through the motions of reporting by submitting largely unverifiable self-reports or have withdrawn from the reporting process altogether. Undoubtedly the obstacles to establishing an alternative monitoring strategy are both numerous and formidable.

What we outline below sets forth a different strategy, which would create a new platform for the review of country progress around Target 4.7. It does not solve all the reporting issues raised above, but it does meet most of the criteria for selecting an alternative monitoring approach. In a nutshell, we propose the creation of a large-scale, crowd-sourced, multistakeholder and independent online platform that shares information/data, promotes peer learning and supports the reporting, measuring and monitoring of the global indicator 4.7.1 (and 13.3.1) and a revised 1974 Recommendation. We think that such a platform could be put into place over four to five years and would likely involve five phases.

1. Phase 1 (6 months): Review past initiatives that have created international platforms with national quantitative and qualitative information on relevant education topics, curricular policies and national assessments. Examples include International Bureau of Education (IBE) World Data on Education; the European Union's Education Information Network called the Eurydice Network; the World Bank’s Systems Approach for Better Education Results (SABER); the GEM Report country profiles. The first phase would also entail a brief literature review of the main elements of the 1974 Recommendation, the establishment of a nimble governance structure and the identification of funding sources.

2. Phase 2 (3 months): Develop one or several (flexible) templates of needed data/content to be circulated among, and tested by, potential contributors and education experts. There would likely be a need for separate templates for preprimary/primary/secondary education; higher education/Technical-Vocational Education and Training (TVET); and adult education. Templates would serve as a mechanism for compiling information and relevant documents from different contexts and settings. They would also define categories and types of information needed (See Table 3).

3. Phase 3 (6-9 months): Identify and compile existing data sources with relevant information, which could be accessed/acquired and placed on the platform. Establish a community of experts to review existing materials, assess their accuracy/validity/comparability and help decide which types of information and data to include. Examples of existing data sources include the following: UNESCO’s compilation of national curricular frameworks for over 100 countries; the Monitoring and Evaluating Climate COmmunication and Education (MECCE) – GEM Report compilation of national profiles of climate communication and education (mecce.ca/climate-change-country-profiles/); the International Civic and Citizenship Study (ICCS) Country Encyclopedia (International Association for the Evaluation of Educational Achievement (IEA)) and possibly international learning assessments with contextual information (UIS, 2022). Other existing sources include APCEIU’s collection of subject syllabi in 23 Asia Pacific countries; Sustainability and Education Policy Network (SEPN) (Canada) analysis of 368 submissions from 194 countries to the United Nations Framework Convention on Climate Change (UNFCCC). Large collections of textbooks, historical and contemporary, have been analyzed for 4.7 themes using simple coding schemes (e.g., collections at Stanford University, the UNESCO International Bureau of Education, and the Leibniz Institute for Educational Media/Georg Eckert Institute). Initially, the identification of data sources might focus on a limited number of regions to serve as an exemplar for further efforts.

4. Phase 4 (6-9 months): Request governments, institutes, policy analysts, researchers, consultants and NGOs to submit new contributions, large and small, on different aspects of the overall template. Request could also go out to select university departments with programs in comparative and international education to contribute information. To facilitate these processes there would be a call to expand the network of interested parties – a community of practice – to review submitted materials, assess their accuracy/validity/comparability and meet periodically to decide what materials to include in the global observatory.

5. Phase 5 (12 months+): Revise templates as needed, secure additional funding, and scale up the project to ensure its sustainability over time. It would be important in this phase to establish mechanisms to (1) update, revise and validate information placed on the platform; (2) develop monitoring indicators from submitted information for multiple purposes; and (3) establish ways to communicate and disseminate information about the platform to diverse stakeholders and demonstrate its use value.

In the shadow of the recently concluded Transforming Education Summit, many would argue that there is an urgent need to prioritize and implement the vision advanced in Target 4.7. Not only would this necessitate increased funding, but also, as this paper argues, alternative monitoring strategies. The creation of a large-scale, crowd-sourced, multistakeholder online platform would effectively mobilize a broad and diverse community of actors to shed new light on ongoing efforts to achieve Target 4.7.

Conclusion

In this paper we examined the extent to which current monitoring mechanisms are fit-for-purpose in realizing SDG Target 4.7 and related targets. We drew on available public documents and materials to examine the status of existing reporting and monitoring mechanisms. We believe we have identified an important set of gaps in current practice.

The existing reporting, measuring and monitoring approach to Target 4.7 results in fewer than half of all countries fulfilling their commitment to report on their progress in implementing the 1974 Recommendation and indirectly on the global indicator 4.7.1 (as well as 12.8.1 and 13.3.1). The current approach appears woefully unfit at capturing the quantity and quality of efforts advancing Target 4.7 and their alignment with the ambitious vision laid out in the 2030 Agenda for Sustainable Development. Unless and until an alternative monitoring approach of transformative spaces in global education policy is put into place, and without delay, UNESCO’s ability to serve as a global thought and policy leader in these areas for the foreseeable future will be undermined. More broadly, the international education community will squander a unique opportunity to mobilize a whole sector approach to transform education and lifelong learning to address urgent global challenges.

In sum, concerted efforts and innovative approaches are urgently needed to create a dynamic new platform – a large-scale, dynamic observatory of data, information and research – that can significantly contribute to revealing, comparing and supporting country programs and actions in all matters related to SDG Target 4.7 and a revised 1974 Recommendation.

Table 1

Percentage of UNESCO Member States participating in the last four consultations on the implementation of the 1974 Recommendation

SDG region	4th consultation (2005–2008)	5th consultation (2009–2012)	6th consultation (2013–2016)	7th consultation (2017–2020)	Total countries
Central and Southern Asia	29	36	43	29	14
Eastern and South-Eastern Asia	6	19	44	44	16
Northern Africa and Western Asia	39	38	42	61	23-24
Europe and Northern America	32	39	68	74	43-44
Latin America and the Caribbean	15	21	42	30	33
Sub-Saharan Africa	9	29	25	13	47-48
Oceania	0	13	25	13	16
Overall response rate	19	29	43	39	193-195

Table 2

Percentage of UNESCO Member States who respond to surveys requesting information about country implementation of UNESCO standard setting instruments

Name of UNESCO standard setting instrument	2009	2011	2013	2015	2017	2019	2021
	UNESCO General Conference
	35th	36th	37th	38th	39th	40th	41st
1976/2015 Recommendation on Adult Learning and Education		79				81
2001/2015 Recommendation on Technical Vocational Education and Training		8				46
1974 Recommendation concerning Education for International Understanding	19		28		43		39
1960 Convention and Recommendation against Discrimination in Education	28		30		34		42
1980 Recommendation on the Status of the Artist		23		31		27
2015 Recommendation on Museums						29
2011 Recommendation on the Urban Landscape						28
1993 Recommendation on the Recognition of Qualifications in Higher Education		10				22
1974/2017 Recommendation on Scientific Researchers		16			21		18
2015 Recommendation on Documentary Heritage						20
2003 Recommendation on Multilingualism and Cyberspace		12		11		9

Table 3

Examples of documents from primary and secondary education to include on proposed reporting and monitoring platform*

Types of documents	Specific documents
Laws, regulations and official policies	Constitutional provisions, legal statutes, international covenants
	National strategic plans in education
	Specific policy documents on ESD, GCED
Intended curriculum	National curriculum frameworks
	Subject-specific syllabi
Instructional materials	Textbooks
	Teaching guides
	Digital learning materials
Summative and formative assessment policy	Formative Assessment guidelines
	Standardized learning assessments
Teacher preparation/in service development	Teacher education policies and curricula
	Provision of professional development courses

Note(s): * A different set of documents would be compiled for higher education and adult education or nonformal education

Notes

1. https://www.un.org/en/transforming-education-summit/tes-summit-closing-press-release (accessed on January 5, 2023)

2. In this paper we define to monitoring as an “ongoing, systematic collection of information to assess progress towards the achievement of objectives, outcomes and impacts” (Mcloughlin and Walton, 2012, p. 6).

3. Quoted from https://www.oecd.org/pisa/aboutpisa/ (Accessed 14 February 2023)

4. An earlier version of SDG Target 4.7, which was adopted during the Global Education for All (EFA) Meeting in May 2014 in Muscat, Oman, included learning outcomes beyond knowledge and skills. The Muscat Agreement (UNESCO, 2014) adopted “Global Target 5”, which stated: “By 2030, all learners acquire knowledge, skills, values and attitudes to establish sustainable and peaceful societies, including through global citizenship and education for sustainable development” (UNESCO, 2014, p. 3). Although the reference to these sociopsychological and emotional outcomes was subsequently deleted from the official formulation of SDG Target 4.7, the Education 2030 Framework for Action adopted in May 2015 at Incheon made explicit mention of them (UNESCO, 2016, p. 49).

5. The seven guiding principles are: (1) An international dimension and a global perspective in education at all levels and in all its forms; (2) Understanding and respect for all peoples, their cultures, civilizations, values and ways of life, including domestic ethnic cultures and cultures of other nations; (3) Awareness of the increasing global interdependence between peoples and nations; (4) Abilities to communicate with others; (5) Awareness not only of the rights but also of the duties incumbent upon individuals, social groups and nations towards each other; (6) Understanding – of the necessity for international solidarity and cooperation; and (7) Readiness on the part of the individual to participate in solving the problems of his community, his country and the world at large (UNESCO, 1974).

6. Under Article VIII of UNESCO’s Constitution, Member States are required to submit a report on the legislative and administrative provisions and any other measures they have taken to implement the conventions and recommendations adopted by the organization. In accordance with the specific multistage procedure for the monitoring of the implementation of UNESCO conventions and recommendations for which no specific institutional mechanism is provided…the frequency for submitting reports is set for an interval of four years.

7. https://world-education-blog.org/2016/10/13/target-4-7-what-is-at-stake-for-monitoring-progress-on-education-for-global-citizenship-and-sustainable-development/

8. Notable exceptions include: Eurostat’s Adult Education Survey; Organisation for Economic Co-operation and Development (OECD) reports on adult education in its Education at a Glance report and through its Program for the International Assessment of Adult Competencies (PIAAC) assessment. In the past UIS conducted surveys of adult education in Latin America.

Appendix Supplementary data

The supplementary material for this article can be found online https://unesdoc.unesco.org/ark:/48223/pf0000379875/PDF/379875eng.pdf.multi