Evaluating enterprise education: issues in current practice

Citation

Harte, V. and Stewart, J. (2012), "Evaluating enterprise education: issues in current practice", Education + Training, Vol. 54 No. 4. https://doi.org/10.1108/et.2012.00454daa.001

Evaluating enterprise education: issues in current practice

Article Type: Guest editorial From: Education + Training, Volume 54, Issue 4.

A muddled field

This journal has a long-standing interest in enterprise education (see e.g. special issues 53: 5; 53: 8/9; 52: 8/9; 50: 7). Previous special issues have focused mainly on the pedagogy of enterprise education and related concepts such as entrepreneurship and employability. This special issue is no different in that those concepts are repeatedly referred to in the collection of articles here. Our particular interest lies in methodologies applied to evaluate the impact and outcomes of enterprise education and that was the focus of our call for papers. We were delighted and not a little surprised at the response to the call, which generated over 25 proposals from an international community of scholars working on the topic. The proposals were subjected to a rigorous and independent peer review process, which enabled us to reduce the number to those selected for publication and so included in this special issue. Full papers produced by the authors were again subject to review by independent referees and revised on the basis of those reviews. As readers will see, one point which is clear from this special issue is that enterprise education continues to be bound up with, connected to and sometimes conflated with notions of entrepreneurship and employability. Evaluation perhaps provides the clearest evidence of the existence of much confusion around these three concepts since evaluation is concerned with measurement of outcomes and that in turn often focuses on what is most easily measured, e.g. how many graduates are employed and/or have started a business. Some of the original proposals we received were relatively easily rejected because of a focus on initiatives clearly related to employability. So, both the response to the call and the papers included in this special issue illustrate yet again that there is no settled meaning let alone single definition attached to the concept of enterprise education and, as many of our contributors argue, this remains a problem for design and application of both the pedagogy and evaluation of enterprise education. We conclude therefore that enterprise education remains a muddled field of research and practice.

The question of measurement

What to measure and how to measure it is the second of a number of common themes, which emerge from the collection of work reported in this special issue. As some of the papers here point out, there is an established and ongoing debate on the validity and utility of “global” approaches to evaluation of enterprise education, which are deemed to be relevant to any and all instances of enterprise-related education; see for example Edwards and Muir. Such global approaches tend to rely exclusively on quantitative data and focus on either measures of business start up or on changes in attitudinal and behavioural characteristics of students and graduates, or a mixture of both. The contribution from Rigg and O’Dwyer makes a strong theoretical case against global approaches which focus on collection and analysis of quantitative data, and illustrate their argument with a specific example of enterprise education from Ireland. The paper by Shi and her colleagues also shows the value of mixed methods. While they advocate the well-established concept of self-efficacy, which is a popular example and feature of a global approach, they also use additional forms and sources of data in their evaluation. Our own contribution (Harte and Stewart) offers an argument for and examples of alternative approaches to global evaluation and methods which use exclusively quantitative data in evaluation of enterprise education.

The importance of context

A key argument against global approaches is that they ignore or negate the influence and importance of context. Contextual factors can take a variety of forms. Three which are highlighted and illustrated as being significant here are national/cultural, subject/discipline and pedagogy. Rigg and O’Dwyer's contribution as well as that of Watts and Wray provide supporting evidence and examples that the pedagogy adopted needs to be a factor which influences and shapes the methods applied in evaluating the impact and outcomes of enterprise education. In simple terms, the content and learning and teaching strategies of enterprise education varies and seeks to achieve varying objectives. That being the case, a single and global approach to evaluation has little merit since it will have little validity and utility in the myriad of varying pedagogic contexts. A similar case is made by the work of Shi et al. and by the paper contributed by Kasturiratne and colleagues, only in those cases the contextual factor is cultural differences in different nations. The point made in both of these papers applies equally to pedagogy of enterprise education but it equally applies to their experience of and attempts to evaluate such interventions. The work of Watts and Wray and also that again of Shi and her colleagues demonstrates that subject disciplines vary in their understanding of enterprise education. This too has implications for both pedagogy and evaluation. All of the papers mentioned make a strong case in support of our own argument in our paper that contextual factors vary, are significant and cannot be ignored in design of evaluation studies of enterprise education.

A wider meaning

The papers by Rigg and O’Dwyer, and by Edwards and Muir, make a related argument to that of context. It is simply that established and traditional notions of what it is to be “an entrepreneur” are too narrow and focused on only limited and specified aspects of individuals. Edwards and Muir argue that the whole sense of self and personal identity are central and so “being an entrepreneur”, or being entrepreneurial or enterprising, cannot be separated off from other aspects of self or identity. Evaluation therefore needs to be whole person orientated and focused. We might say that Edwards and Muir take a psychological and constructivist perspective in their work. Rigg and O’Dwyer take a slightly different and wider view in their argument; we might say more of a sociological and constructionist perspective. For them, identity is socially constructed and a social rather than strictly individual phenomenon. That being the case, to be entrepreneurial or enterprising is not a matter exclusively for individual to choose or decide. It follows therefore that both enterprise pedagogy and evaluation needs to be focused on the social context of the recipients. Both sets of authors are clear that evaluation of enterprise education cannot be reduced to quantitative measures of either educational or economic outcomes.

The problem of embedding enterprise education

A number of papers mention and address to a greater or lesser extent the notion of embedding enterprise in the curriculum of higher education (HE). Edwards and Muir, for example describe a range of models adopted by HE institutions to provide enterprise education; e.g. common modules provided by business schools or by a dedicated enterprise unit. Kasturiratne and colleagues identify the problems of varying approaches to learning and teaching in different contexts as one significant aspect of this problem. Our contribution deals directly with the problem and argues the potential of utilising contextual approaches to evaluation for embedding enterprise education across the curriculum. This is a novel idea and is perhaps counter intuitive since the standard approach is to end rather than begin with evaluation. The argument is therefore speculative at the moment but we suggest it merits further investigation. What is clear from the work reported in this special issue is that the ideal of enterprise education being embedded in HE is a long way from being achieved.

Summary and conclusion

In summary, the collection of papers we have assembled in this special issue point to five key issues facing those involved in and tasked to evaluate enterprise education. These are:

1. 1.

   specifying a meaning and definition;

2. 2.

   deciding on what and how to measure;

3. 3.

   taking account of various and varying context factors;

4. 4.

   incorporating identity and the whole person, in context, in evaluating the impact and outcomes of enterprise education; and

5. 5.

   addressing the aim of embedding enterprise education through evaluation.

This list represents just one interpretation of the key messages of the papers we have assembled, and that of course is our interpretation. Not all of the contributions address each of the issues and some, perhaps all, of the contributors may take exception to having their work associated with one or more of the issues. In addition, it is certainly the case that in total the papers address many more and different issues than we identify above. But, we are confident that the list represents the key commonalities across the papers. As such, and to the extent that the contributions can be said to be representative of current practice, we suggest that the five key issues constitute the main challenges to be addressed in the evaluation of enterprise education. We hope readers of this special issue will be motivated by the papers here to address these challenges in future issues of this journal and elsewhere.

There is one final point we would like to make. There has been and continues to be significant investment in enterprise education in HE and so evaluating the impact of that investment is important. But, HE is subject to increasing quantitative measures of its activities, often without much if any justification of the validity or utility of those measures. The papers in this special edition make a consistent case against crude quantitative measures and for more contextualised, sensitive and nuanced judgements of impact. We hope the evidence contained here in support of that message will itself have impact on the practice of evaluating enterprise education.

Victoria Harte and Jim StewartGuest Editors