The role of Southern perspectives in the evaluation of development education

Introduction

This article explores the role of Southern perspectives in the evaluation of Northern-based development education (DE) programmes. The article outlines the rationale for the inclusion of Southern perspectives in the evaluation of DE; explores key challenges in the area; reviews two examples of Northern attempts to engage in evaluation/reflection with Southern partners; and concludes with suggestions for further work in this area.

Rationale for including Southern perspectives in DE evaluation

In recent years, ‘Southern perspectives’ have been the focus of much discussion in the DE community. 80 per cent of the world’s population lives in the global South, and it has become increasingly clear that Southerners need to play a key role in any form of education that claims to be about ‘increasing understanding of our…interdependent and unequal world’ (Irish Aid, 2003: 9).  Although there are contested definitions of the term ‘Southern perspectives’ (IDEA, 2010a), nearly all of today’s development educators would agree with Andreotti (2006: 9) that ‘listening seriously and respectfully to Southern voices’ is a vital first step towards addressing the power imbalances that exist between North and South.  The DE sector has begun to address this challenge, and Southern perspectives are being incorporated in a variety of ways into a growing number of Northern-based DE courses, events and resources.  

However, evaluation is one key area of DE in which Southern perspectives have not yet received much attention. Evaluation is the ‘bottom line’ of any endeavour, and is particularly important in a field such as DE, in which critical reflection occupies a central role in the learning cycle (Dolan, 1998:8). Scriven defines evaluation as the ‘determination of the worth or value of something judged according to appropriate criteria, with those criteria explicated and justified’ (1991).  Scriven’s emphasis on criteria is significant, as it highlights the power dimension of evaluation. Whoever develops the ‘appropriate criteria’ for an evaluation gains the power to assign value to certain ways of thinking and doing.  Therefore, truly bringing Southern perspectives into DE evaluation involves much more than asking for a Southern ‘stamp of approval’ for DE work; it holds the potential to bring real changes to the North-South power dynamic that Andreotti (2006) places at the heart of DE.

The incorporation of Southern perspectives into DE evaluation is also desirable from the point of view of current good practice in educational evaluation. There has been a general trend away from traditional evaluation models which measure success in pre-determined quantitative terms, and towards qualitative models which celebrate a multiplicity of values and perspectives (Christie & Alkin, 2008). An example of this is Guba and Lincoln’s ‘fourth generation’ evaluation:

“Fourth-generation evaluation recognises the constructed nature of findings, takes different values and different contexts (physical, psychological, social and cultural) into account, empowers and enfranchises…it extends both political and conceptual parity to all stakeholders” (Guba & Lincoln, 1989: 11).

This type of model would appear to be ideally suited for DE, as it advocates the creation of an open space for evaluation, in which people from South and North could negotiate values and work collaboratively towards new meanings.

Challenges

There is a clear rationale for including Southern perspectives in the evaluation of DE. However, many challenges emerge when one considers how Southern perspectives might actually be implemented into DE evaluations.  The most significant difficulty relates to identifying who from the South should be involved in the evaluation process.  In ‘development’ projects, it is usual to have specific stakeholders in the global South, such as a Southern community in which a particular education or health care initiative is delivered. However, in DE, Southern stakeholders are usually not obvious (with the exception of projects such as school links, which have a specific, closed set of Northern and Southern participants). The ‘South’ undoubtedly occupies a role, but, depending on the theoretical and practical framework upon which any particular DE project is constructed, the South could be perceived as a ‘stakeholder’, a ‘partner’, a ‘target group’ or a ‘long-term beneficiary’.    

This lack of clarity about the South’s role in DE gives rise to a number of complex questions, such as: how can a sole Southern individual contributing to the evaluation process represent the entire global South in all of its diversity?  If a Southerner occupies the privileged position of expert adviser to a Northern organisation, then can he/she still speak on behalf of the ‘oppressed’?  These questions relate to fundamental difficulties with the North-South binary as a framework for DE. Andreotti and de Souza (2008: 31) point out that the North-South binary is a helpful framework in that it highlights ‘the inherently social constructions of identities and relationships and the power dynamics that permeate relationships between the groups’; however, they also note that the binary tends to ‘essentialise’ the individual experiences of Northerners and Southerners. Applying these observations to the evaluation of DE, it would seem that a Southern evaluator can legitimately act as a spokesperson for ‘the South’ in terms of the broad South-North identities and relationships that underlie a DE programme, but at the same time, it must also be recognised that any individual Southerner brings his/her unique background, perceptions and values to the evaluation process.  There is no easy way of resolving this tension; therefore, the process of choosing an evaluator, or consenting to become one, would require a great deal of dialogue between Southern and Northern partners to clarify roles and representation.  

Even if these challenging issues are worked through, and if practical obstacles to do with distance, language and finance are overcome, there are further difficulties to face. Southern and Northern participants must successfully negotiate decisions around evaluation criteria and methodologies.  As noted above, issues of power underlie these decisions; a Southern evaluation being done ‘to’ a Northern group has a very different power dynamic than one being done ‘for’ or ‘with’ them. Finally, there is the thorny issue of what happens post-evaluation. If Southerners are involved in the evaluation process, but possess no influence in regard to future projects, then do they really have equal power in evaluation? For Southern participation in evaluation to have any meaning, there need to be structures through which Southerners can meaningfully contribute to all aspects of the project cycle.       

Given these challenges, and given the fact that the major Northern funders of DE (such as Irish Aid) do not require, or even suggest, that Southerners should be involved in the evaluation of DE, it is not surprising that very few attempts have been made to engage the South in DE evaluation.  Even when clearly-defined groups of Southern stakeholders are present, such as in a North-South school link, Northern evaluators have rarely sought Southern participation. Indeed, Burr (2008:4) provides a disturbing anecdote about an award-winning UK school that completed an evaluation of its school link but did not involve the Southern school at all ‘because they wouldn’t understand’. 

The RORG-South Evaluation

A notable exception to the norm is the RORG DE network in Norway, which in 2001 made a decision to ‘subject itself to an evaluation from the South’ (van der Merwe, 2003: 6). The rationale for this initiative is clearly articulated:

“DE is basically motivated by a desire to change the world to be a better place, fighting poverty and injustice. Thus, the part of the world most hit by poverty and injustice, the South, should have a say in how DE is done in the North” (van der Merwe, 2003: 20).

RORG commissioned a team of Southern evaluators, led by a South African academic and including members from the Philippines and Nicaragua (unfortunately, there is no available documentation regarding how these individuals were selected). The team’s remit was ‘to assess the efficacy of DE in Norway as viewed from the South’ (van der Merwe, 2003: 6).  The RORG network made it clear that the Southern partners would be ‘in the driving seat’ of the evaluation; this was perceived by both parties to be a reversal of the traditional North-South power dynamic (van der Merwe, 2003: 11).   

The process was inherently ‘a long shot at goal’, with both Northerners and Southerners acting as willing partners in an ‘uncertain and risky process’ (van der Merwe, 2003: 11). Indeed, fundamental challenges emerged at the earliest stages of the project. The Southern team were unclear as to what they were being asked to evaluate and why they had been asked to do so:

“The core business of RORG, i.e. DE, turned out to be rather uncertain in the mind of the RORGs. DE in the North in general, and in Norway in particular, was a practice to which the participants from the South were not only totally unfamiliar with, but also fundamentally suspicious about. The decision to involve the South in an evaluation of the RORGs was also treated with a significant amount of suspicion” (van der Merwe, 2003: 23).

Both RORG and the Southern evaluators wisely recognised that these apparent obstacles were in fact an integral part of the process; they commented that ‘the learning that took place during this South-North process became a purpose in itself’ (van der Merwe, 2003: 11). 

The evaluation produced interesting recommendations, including a call for the development of a ‘pedagogy for the rich’ that would help to build real solidarity with the global South (van der Merwe, 2003: 24). The recommendations were quite critical of current RORG practices, but were offered in the understanding that ‘evaluation should facilitate and empower the evaluated’ to take steps towards positive change (van der Merwe, 2003: 12). RORG’s work in subsequent years indicates that the evaluation resulted in increased reflection and action regarding the role of the South in Norwegian DE. These changes include the development of position papers on the role of DE, Southern perspectives and co-operation with the South (RORG, 2004a; 2004b); the publication of a set of ‘ethical guidelines’ for practitioners working in the development NGO sector (RORG, 2006); and successful bids for government funding for DE projects with Southern partners.

The RORG evaluation should be recognised as an important milestone in DE’s journey towards North-South equality.  However, it would be unrealistic to presume that other DE groups could replicate the RORG model of evaluation.  On a practical level, few organisations have the time and resources for such a process.  More significantly, in today’s competitive funding climate, DE groups are under pressure to gloss over difficult issues and to use evaluation merely as a means of advertising the ‘success stories’ of their programmes.

The Global Educator in Residence Programme

A possible alternative to the RORG model would be to engage a Southern educator in the role of ‘critical friend’ to a Northern DE organisation. This year, the Irish Development Education Association (IDEA) ran a pilot programme titled Global Educator in Residence (GEIR). In this programme, educators from the global South, all of whom were familiar with Northern-based DE, were engaged to meet face-to-face with IDEA member organisations to discuss how the organisations might better incorporate Southern perspectives into their work.  

The overall aim of the GEIR programme was ‘to contribute towards creating a fair and equal dialogue between global North and global South’ (IDEA, 2010b: 3). The GEIR programme was not conceived as an evaluation or as a ‘Global South Driving Test’ for IDEA members. Instead, the programme was promoted as an opportunity to engage in reflection and dialogue with a Southern educator. It was left to each participating organisation to communicate with its assigned Global Educator in order to plan what they would do during their time together.  A pre-visit workshop, an online discussion forum and learning journal tasks helped participating organisations to shape questions and define critical issues prior to the visits. The supporting activities provided a safe space for exploring some of the more challenging ideas relating to power, voice and representation in the ‘Southern perspectives’ debate.

Yet even within this supportive framework, there were still some anxieties about inviting an outsider into the inner, vulnerable spaces of an organisation. As IDEA (2010b:9) points out, this anxiety ‘was not always conducive to the learning objectives established by the programme’. The Global Educators themselves expressed discomfort with the role of ‘expert adviser’. One of the Educators commented:

“The role of the Global Educator is not/should not be a provocateur; it is someone from a different network coming in to give advice and support…challenging and pushing the organisations further (like a mentor or a coach perhaps), not judging or forcing but working with the passions of the organisations” (IDEA, 2010b: 18).

At the end of the programme, one organisation commented that the Global Educator acted not as a judge but instead as a catalyst for learning, development and change.  For a number of organisations, the most valued aspect of the GEIR was the fact that the overall programme ‘provided them with a space for reflection and engagement with a highly controversial topic and that they were able to do this while sharing their learning with others’ (IDEA, 2010b: 7). In this way, the Southern educators empowered Northern DE organisations to formulate key questions to ask themselves. Hopefully, these questions will form the foundation for important internal evaluation work. In this light, the GEIR programme could be perceived as a viable alternative to a more structured and formal Southern evaluation. 

Conclusion

This article has shown that, although a clear rationale exists for the incorporation of Southern perspectives into the evaluation of Northern-based DE, practical implementation of this ideal presents many difficulties. Southern stakeholders/partners are not usually clearly defined in DE, and, even if an appropriate evaluation team were to be assembled, most organisations lack the time, resources, and most importantly, the incentive to undertake a South-North evaluation.

The ambitious RORG programme produced valuable results, but the evaluation process demanded an extremely high level of commitment and a willingness to face uncomfortable issues. The much smaller-scale GEIR programme demonstrated the value to Northern DE organisations of reflecting upon their work with the aid of a Southern ‘critical friend’.  

Further work needs to be done in this challenging area. School links, and other DE projects with clearly-defined Northern and Southern stakeholders, would benefit greatly from a collaborative approach to evaluation. In such evaluations, Southern participants would need to be involved not just in evaluation tasks, but also in setting the evaluation criteria and deciding what changes will take place post-evaluation. 

Other DE projects, with less obvious Southern stakeholders, would benefit from developing the ‘critical friend’ model. Ideally, a Southern critical friend would be introduced at the early stages of a project, and would to help to construct an evaluation framework that could be revisited periodically throughout the project lifecycle. 

Because the global South, in all of its complexity, plays many different roles in the wide variety of programmes that make up ‘development education’, there will never be a single formula for bringing Southern perspectives into DE evaluation.  Each DE organisation needs to look closely at how its particular programme relates to the global South, and then find creative ways of engaging Southern voices in a process of reflection and evaluation.       

Note: The author thanks Arnfinn Nygaard and Matthias Fiedler for their input into this article.

References

Andreotti, V (2006) ‘Theory without practice is idle, practice without theory is blind: the potential contributions of post-colonial theory to development education’, Development Education Journal, Vol. 12, No. 3, pp. 7-10.

Andreotti, V and de Souza, LM (2008) ‘Translating theory into practice and walking minefields: lessons from the ‘Through Other Eyes’ project’, International Journal of Development Education and Global Learning, Vol. 1, No. 1, pp. 23-36.

Burr, M (2008) Thinking about Linking, London: DEA.

Christie, C and Alkin, M (2008) ‘Evaluation theory tree re-examined’, Studies in Educational Evaluation, Vol. 34, pp. 131-135.

Dolan, A (1998) ‘A rationale for expanding the evaluation budget’, Development Education Journal Vol. 5, No. 1, pp. 8-10.

Guba, E and Lincoln, Y (1989) Fourth Generation Evaluation, London: Sage.

IDEA (2010a) ‘Defining Southern Perspectives’, IDEA discussion forum, available: http://www.ideaonline.ie/content/defining-southern-perspectives (accessed 20 July 2010).

IDEA (2010b) Report on Global Educator in Residence Programme, Dublin: IDEA.

Irish Aid (2003) Describing, Understanding, Challenging: The story of human development in today’s world, Dublin: Irish Aid.

RORG (2004a) ‘North-South Information: Communication for Global Justice’, Key DE Policy Papers, available: http://www.rorg.no/Artikler/1811.html (accessed 30 July 2010).

RORG (2004b) ‘Southern Perspectives and Co-operation with the South’, Key DE Policy Papers, available: http://www.rorg.no/Artikler/1811.html (accessed 30 July 2010).

RORG (2006) ‘Be Careful: Ethical Guidelines for North-South Information in Norway’, Key DE Policy Papers, available: http://www.rorg.no/Artikler/1811.html (accessed 30 July 2010).

Scriven, M (1991) Evaluation Thesaurus, London: Sage.

van der Merwe, S (2003) ‘South Evaluation of the RORG-network in Norway: A view from the South on Development Education in the North’, Oslo: RORG.

Susan Gallwey is the Education Officer at the Waterford One World Centre, and is Vice-Chairperson of IDEA. Her 2009 M.Ed. thesis explored evaluation issues in Ireland-South school links.