Prelims

Half Title Page

Global Perspectives on Educational Testing: Examining Fairness, High-Stakes and Policy Reform

Series Page

Advances in Education in Diverse Communities: Research, Policy and Praxis

Series Editor: Carol Camp Yeakey Recent Volumes:

Title Page

Advances in Education in Diverse Communities: Research, Policy and PraxisVolume 13

Global Perspectives on Educational Testing: Examining Fairness, High-Stakes and Policy Reform

BY

Keena Arbuthnot

Louisiana State University, USA

United Kingdom – North America – Japan India – Malaysia – China

Copyright Page

Emerald Publishing Limited

Howard House, Wagon Lane, Bingley BD16 1WA, UK

First edition 2017

Copyright © 2017 Emerald Publishing Limited

Reprints and permissions service

Contact: permissions@emeraldinsight.com

No part of this book may be reproduced, stored in a retrieval system, transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without either the prior written permission of the publisher or a licence permitting restricted copying issued in the UK by The Copyright Licensing Agency and in the USA by The Copyright Clearance Center. Any opinions expressed in the chapters are those of the authors. Whilst Emerald makes every effort to ensure the quality and accuracy of its content, Emerald makes no representation implied or otherwise, as to the chapters’ suitability and application and disclaims any warranties, express or implied, to their use.

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

ISBN: 978-1-78635-434-1 (Print)

ISBN: 978-1-78635-433-4 (Online)

ISBN: 978-1-78714-896-3 (Epub)

ISN: 1479-358X (Series)

This book is dedicated to William, Alfreda & Stephanie

Contents

List of Tables

Acknowledgments

I have many mentors and colleagues whom I must thank for their selflessness in helping me to achieve my academic and career goals, including, but not limited to, Drs. Edmund W. Gordon, M. Christopher Brown II, William Trent, Sara Lawrence-Lightfoot, Eugene Kennedy, Stafford Hood, Carol Camp Yeakey, Katherine E. Ryan, and Bridget Terry Long. Each of you have been instrumental in shaping my career, and I am truly grateful for your leadership and vision to light the way for those of us who come after you. Additionally, I must acknowledge those whom I consider my contemporaries and who work alongside me to leave our mark in our respective fields. We support and inspire one another and provide much needed shoulders on which to lean in times of trial, and are true encouragers in times of triumph. These individuals include, but are not limited to, Dawn Williams Salter, Maurice Samuels, Darrell Ray, and Sassy Wheeler. Last, I must acknowledge those scholars whom I have had the distinct honor of mentoring and advising during their academic pursuits, including but not limited to, Tireka Cobb, Jared Avery, Adam Elder, Erin Scott-Stewart, and Guadalupe Lamadrid.

Personally, I want to thank my parents William and Alfreda Arbuthnot and my sister Stephanie Arbuthnot whose love and support have always been the wind beneath my wings. I cannot begin to tell each of you how much I appreciate the unconditional love and unwavering support that you have shown me throughout the years. You continue to encourage me to meet my highest potential and to always put God first. As a child growing up, each one of you taught me in your own special way how to face adversities with dignity and fortitude. Those lessons taught me how to survive in the face of opposition and how to thrive in the midst of it. In addition to my immediate family, I have been blessed with an incredibly supportive extended family, including but not limited to, Barbara Whitener Gardner, Kelly Gardner, Kimberly Gardner, Michael Bates, the Murry family and other aunts, uncles, and cousins. I also want to thank my friends, many of whom have become family, for always making the choice to love and support me regardless of the circumstances. There are too many to name but they include Jas and Samaah Sullivan, Vashti Person, Franklin Mosley III, Kaisha Mozee, Valerie Byers, Tyrone Perkins, LaShonda Harvey, Eric Cook, Bridget Perkins, Kimberly Cole, and the Seven Sisters. Last, to all my godchildren, mentees and former students, including but not limited to, Tierra Webb, Aja and Mariah Brooks, Dallas Hawkins, Jennifer Cook, Aisha Camphor and Marcus Jones, you are all an inspiration to me.

I want to also thank all of the friends and colleagues I have made across the world. The opportunity to meet wonderful people from various places around the globe has reshaped my research focus not only on issues plaguing testing and measurement in the United States, but also on expanding this line of research to issues pertaining to education worldwide. Improving education and understanding the way in which all students learn, regardless of race, ethnicity, or country of origin, have become central to my research agenda. My experiences abroad have shown me that all parents, educators, and policymakers intuitively want the same thing: to make education accessible, fair and engaging to all children regardless of who they are or from where they come.

Introduction

Much of my career has focused on investigating group differences in standardized test performance and test fairness issues in the United States. My interest in this area was sparked by my experiences as a high school mathematics teacher. As a teacher, I realized the impact that high-stakes tests had on students, particularly Black students, who are considered to be the minority in the United States. Research has shown that on most standardized tests, White students outperformed their Black counterparts. I wanted to determine whether these high-stakes tests were fair to all test taker groups, and, consequently, spent years as a graduate student and professor examining fairness issues and investigating test- and item-level performance patterns. My research investigated Black and White test takers and highlighted the differences in the test-taking experiences between the two groups (Arbuthnot, 2009, 2011a, Arbuthnot & Ryan, 2005). The results undeniably showed that cultural differences between Blacks and Whites had a significant impact on their standardized test experience and performance (Arbuthnot, 2011a,-, 2015a-, 2015b; Arbuthnot & Lyons-Thomas, 2016). Moreover, the research identified different mathematics subtests and types of items that were differentially harder for one group in comparison to the other. Additionally, the research focused on how students from various groups process test items differently while taking a standardized test. The research provided a new way to understand and conceptualize the way that different race/ethnic groups, with contrasting cultural backgrounds, experienced and performed on tests. My book Filling in the Blanks: Understanding the Black White Achievement Gap (Arbuthnot, 2011) provided comprehensive details from over a decade of research regarding the variations between Black and White test-takers’ experiences. One line of the book’s research examined the similarities in the test-taking experience and performance of Black students and female students on standardized mathematics tests. The results indicated that Black and female students performed similarly on these examinations. To explain this finding, I examined the commonalities between Black students and female students in the United States. Consequently, the cultural similarities between the two groups helped to explain why the test-taking patterns of Black and female students were comparable.

Simultaneously, I dedicated my research to addressing issues related to test fairness as well. My writing focused on examining the high-stakes testing systems and providing empirical research to challenge test fairness issues. This research highlighted the roles and responsibilities of test developers in ensuring that tests were fair to all test takers, as well as challenging test users and consumers to critically examine the way in which they interpreted test results (Arbuthnot, 2011a, 2012a, 2015a, 2015b; Arbuthnot & Lyons-Thomas, 2016).

I continued my research and worked on issues related to domestic test fairness; concurrently, I was presented with opportunities to participate in consulting and grant opportunities abroad. Most of my opportunities abroad involved countries in the Middle East, mainly Qatar and the United Arab Emirates. From these experiences, I developed a better understanding of the educational challenges that Middle Eastern countries faced. I realized that some of the issues that students faced in that region were similar to the difficulties that I recognized in my research with Black and female students in the United States. With my background in standardized testing, I turned to international assessments to investigate the similarities between minority and female students in the United States and students from the Middle East, expanding my line of research on testing and fairness to a global scale.

This book highlights the basis and justification for the research and provides a detailed exploration of how to examine fairness issues on international assessments. It is my hope that audiences around the world will utilize this book in the quest for understanding and conceptualizing variations in the way in which test takers from different countries and cultures learn and perform on tests.

The purpose of this book is to investigate fairness issues on international assessments. The text begins with an overview of the current state of international assessments and reveals the ways in which many countries have utilized results from international assessment initiatives to inform educational policy and practice at the national level. The book then describes the various international assessment programs that have been implemented over the last several decades. The text then focuses on the Trends in International Mathematics and Science Study (TIMSS) assessment, one of the most popular and longstanding international assessment programs. The book utilizes TIMSS assessment data that includes the 4th and 8th grade mathematics test in conjunction with information obtained from a variety of stakeholder questionnaires (i.e., students, teachers, and administrators) from each of the participating countries. Additionally, the use of details concerning fairness issues is included as well as how research conducted on fairness issues in the United States provides a framework for examining issues of fairness at the global level.

In order to investigate fairness issues on the global scale, the author introduces The Arbuthnot Assessment Fairness (TAAF) Framework as a means to systematically examine test- and item-level performance patterns and fairness issues. The TAAF Framework has four phases: (a) country selection, (b) test and subtest performance and test-taking patterns, (c) item-level differences and patterns, and (d) cultural impact. Completing these four phases of the TAAF Framework provides a clearer understanding and interpretation of test- and item-level differences and provides an in-depth examination of fairness issues on international assessment. The author provides a detailed example of how to use the TAAF Framework to inform multiple stakeholder groups about differences in performance patterns among countries and test fairness issues. The countries of Chinese Taipei, Finland, the United States, and Qatar were chosen for the research based on three criteria: cultural context, performance patterns, and educational reform policies. The book provides a critical examination of the educational practices, assessments, accountability measures, and cultural norms for each of these countries.

The book utilizes the empirical data provided by the mathematical portion of the TIMSS international assessments to demonstrate and elucidate how to analyze the international assessment data and use multiple data sources to examine issues of fairness on international assessments. This book ends by challenging readers to deliberate more thoughtfully and to exercise caution with respect to the ways in which test- and item-level performance data is interpreted on international assessments.

To conclude, the author encourages readers to be mindful when interpreting test performance and insists that the various stakeholder groups, the test developers, educational theorists, policymakers, and practitioners, better understand the differences in performance patterns between countries and the issues surrounding test fairness on international assessments.