Challenging the illusion of objectivity: an in-depth analysis of the preselected items evaluation (PIE) method in translation evaluation

Abstract

Purpose

The primary objective of this research paper was to examine the objectivity of the preselected items evaluation (PIE) method, a prevalent translation scoring method deployed by international institutions such as UAntwerpen, UGent and the University of Granada.

Design/methodology/approach

This research critically analyzed the scientific and theoretical bottlenecks associated with the PIE method, specifically focusing on its parameters, namely the p-value and d-index, in adherence to established statistical protocols. Proposed remedies to mitigate the identified bottlenecks and augment the efficacy of the method were grounded in practicality.

Findings

The paper provided an extensive overview of the PIE method, which served as the foundation for the subsequent analysis and discussions. This research presented potential avenues for refinement and contributed to the current debate on objective translation assessment by addressing the theoretical and practical challenges associated with the PIE method.

Research limitations/implications

Translation researchers, practitioners and international institutions seeking to enhance the accuracy and reliability of translation evaluation should consider the implications of this research’s findings.

Originality/value

Although several publications focused on the role of the PIE method in translation evaluation, no study(ies) is available to critically analyze the scientific and theoretical bottlenecks of this translation evaluation method.

Citation

Akbari, A. and Shahnazari, M. (2024), "Challenging the illusion of objectivity: an in-depth analysis of the preselected items evaluation (PIE) method in translation evaluation", Journal of Applied Research in Higher Education, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JARHE-11-2023-0523