“Chapter 3 Investigating state behaviour towards refugees” in “Refugee Reception in Southern Africa”
Chapter 3 Investigating state behaviour towards refugees
This chapter explores the methodology and research methods used to investigate variations in state responses to the arrival of refugees in Southern Africa. Specifically, the chapter examines the book’s adopted methodology in a piecemeal way through the distinct phases of the project, from choosing the research design to collecting and analysing the data. In doing so, the intention is for the chapter to be a working model for future work by scholars developing and designing similar projects. This should be of particular interest for scholars studying refugee reception policy but also for those in related fields who are interested in research methods, triangulation and comparative case studies.
The chapter starts with an explanation of the overarching methodological approach of the book. It then broadly follows the steps set out by Hentschel (1998) in relation to producing a robust research design, by outlining the overall research design, then moving on to examine the approaches taken in respect of data collection, data analysis and data interpretation. At each stage, time is spent critically reflecting on the decisions made, and how certain approaches were selected over others. In the second half of the chapter, the timing of the research, positionality and potential limitations to the study are explored. Key ethical considerations inherent to a project aimed at understanding state policies related to refugees are also examined. Engaging with an emerging body of literature on this topic, the chapter considers relevant ethical concerns and how they were mitigated in the project. At the same time, the chapter does not purport to have solutions to all ethical challenges that emerge in forced migration research (Müller-Funk, 2021). There are no easy answers when addressing these issues. Rather, it remains a constant balancing act, with the principle of ‘do no harm’ needing to be the starting point for all decisions.
The book is predominantly based on three prolonged stays in Southern Africa that took place between 2016 and 2018 and draws from a range of sources (both oral and written). Initial library research, conducted at the University of the Witwatersrand in Johannesburg in late 2016 generated a focused literature review of key topics and a collection of published law and policy relating to refugee reception on the African continent and specifically concerning South Africa and Zambia. In addition, informal interviews were conducted with local academics, and meetings and events between state officials and civil society were attended. During this first visit, the design of the project was finalised.
In 2017 and 2018, two further extended visits were undertaken in South Africa and Zambia. The methods used for data collection were: (1) key informant interviews (KIIs); (2) informal interviews and attendance at state body and civil society meetings and events; and (3) review of national legal and policy documents. This amounted to over seventy hours of formal and informal interviews, attendance at more than twenty public meetings and events with government officials and/or civil society and the review of a considerable quantity of policy documentation. Between 2018 and 2022, three years of post-doctoral study were completed at the African Centre for Migration and Society (ACMS), University of the Witwatersrand. During this time, initial analysis was revised and updated to reflect political shifts in Southern Africa and recent developments in academic literature.
Overarching methodological stance
In line with the overarching epistemological position set out in the Introduction, the qualitative methodological approach that underpins this book is a broad constructivist one. In contrast to a positivist stance, which aims for objective and universal knowledge, this approach acknowledges the constructed nature of the social reality that is embedded in decision-making and power relations concerning state reception policies. Within this broad constructivist stance, there is the scope for both ‘thick descriptions’ of key actors’ experiences at the national and local level (Neimeyer and Levitt, 2001), as well as advancing knowledge via the process of theory-building (Gray, 2013). As proposed by Betts and Orchard (2014), this can be achieved through balancing the in-depth insights of ethnography with wider insights from political science.
The use of KIIs as the primary method of data collection closely fits this overall approach. The aim of KIIs is to uncover participants’ perceptions (Crouch and McKenzie, 2006) and specific motivations and behaviours relating to a particular topic (Kumar, 1989). Thus, the goal is to generate insights into the stakeholders’ involvement with the reception of refugees. Furthermore, under constructivism, contradictions between KIIs indicates differences of perspective and remain valid and insightful rather than indicative of inaccuracies. These insights were supplemented by quasi-ethnographic methods, namely residing on and off in the target region for over four years, undertaking informal interviews with local ‘experts’ and attending a large number of events, meetings and conferences relating to the reception and movement of migrants in the sub-region (referred to hereafter as ‘informal interviews and symposia’). The use of the term ‘quasi-ethnographic’ is due to the frequency of visits to the sub-region (Murtagh, 2007). Ethnographic studies have customarily involved the researcher being immersed in one setting for a long period of time (Bryman, 2004), whereas this book conducted a study in a small number of settings. Nevertheless, since the 1990s, ‘multi-sited ethnography’ has been widely accepted (Marcus, 1995). Consistent with Murtagh’s (2007) understanding of quasi-ethnography, while the use of multi-sites does have implications in terms of less time spent in each setting, it nevertheless facilitates the opportunity to explore refugee reception from two perspectives and to generate a rich array of data and processual connections.
Research design
An overarching explanatory research design was chosen because the book asks ‘why’ and ‘how’ questions about refugee reception (Gray, 2013). As explored in the previous chapter, knowledge already exists on specific aspects of the reception of refugees in Africa. Thus, a more exploratory research design was rejected (Robson, 2002).1 Instead, the book builds on previous work within the fields of refugee and forced migration studies to understand specifically why states respond to refugees in different ways.
To investigate the disparate state responses to the arrival of refugees in Africa, the book sets out to answer two key questions. First, how do we explain the diverse ways in which states receive refugees in their territories? Second, how do the refugee reception policies of host states shape a refugee’s ability to pursue their own personal and economic aims? The need to ask these ‘how’ questions has a bearing on all aspects of the research design, including the methods chosen for data collection (Crotty, 1998). Nevertheless, as examined next, the development of these research questions themselves was an iterative process, with findings during the framing exercise and initial stages of fieldwork being fed back into this process and informing the finalised questions above (Creswell, 2009). Lastly, when determining the research design, care was taken to make sure it also fitted with the chosen comparative case study model, as presented in the Introduction; the book’s conceptual framework, as set out in Chapter 1; and the overarching methodological approach, as discussed above.
The framing exercise, September 2016
The first visit to Southern Africa lasted for four months. It took place in 2016 and was based at ACMS. The purpose of this initial visit was to undertake a framing exercise. This amounted to the completion of several preliminary activities to determine the make-up of the research design. Firstly, six informal interviews were conducted with academics working in the field of migration studies in South Africa.2 Secondly, an affiliation with ACMS permitted access to the daily activities of the centre, which included a multitude of workshops and lectures relating to human mobility and migration in Southern Africa. Finally, the affiliation also allowed access to civil society meetings with government bodies, as well as to presentations and conferences hosted and run by the Department of Home Affairs in South Africa. The overarching aims of the framing exercise were to develop and refine original research questions; to develop a comprehensive literature review in order to locate an adequate conceptual framework through which to respond to the research questions; and to select the most appropriate methods to answer the research questions.3
The initial focus of the project (derived from a Masters’ dissertation completed in 2013) was on state attitudes towards the right to freedom of movement for refugees at the point of arrival using a socio-legal lens.4 The first iterations of the research questions were tentative, yet also useful as a tool for setting out the primary aims of the research (Agee, 2009). Through the framing exercise, the research questions evolved, becoming more focused on the rationale behind state-run reception policies and the potential implications for refugees. In this way, the concept of the ‘state’ became more layered in terms of the level of explicitness of attitudes, rationales and motivations, as well as geographical scale. Furthermore, as examined in the Introduction, the book investigates this topic from a political science perspective (rather than a socio-legal one), with the emphasis being on why states behave as they do, rather than on what they should be doing. Finally, as noted by Flick (2006:106), the reflexive process of formulating research questions helps ‘circumscribe a specific area of a more or less complex field, which you regard as essential’. Indeed, this reflexive approach continued throughout the fieldwork and analytical stages, with the research questions continuing to be developed and refined (Agee, 2009).
The framing exercise also developed a preliminary literature review by incorporating localised literature and interviews with local academics, in order to select and adapt a suitable conceptual framework for the book. As both the framework and research questions were formulated (although not finalised) during this scoping phrase, the research questions could be constructed so they explicitly made links with the theory (Agee, 2009). Moreover, similarly to the reflexive approach used for finalising the research questions, the conceptual framework adopted for the book was also amended during the research process. Specifically, this occurred when the initial data collected indicated that the ideational influence of national governance frameworks in the implementation of reception policies was far greater than initially proposed by Betts and Orchard’s (2014) theory of norm implementation. Conversely, this suggested that the role of the global refugee regime on reception policies in Southern Africa was potentially less influential than inferred from the original theory. Lastly, as proposed by Mackenzie et al. (2007), the informal dialogues conducted during the framing exercise were valuable in developing future questions for more structured interviews and drawing attention to new areas for investigation. Thus, the framing exercise itself became an integral part of the research design, with the informal interviews and attendance at key stakeholder symposia forming part of the analysis.
The finalised research design
Based on the framing exercise, an explanatory research design was finalised at the end of the first trip. KIIs were chosen as the main method for collecting data. Two further methods were then selected to augment and triangulate data in conjunction with this dominant approach. Firstly, the relevant national legal and policy documents were examined and secondly ‘Informal Interviews and Symposia’ were drawn upon. These are examined further below.
KIIs were selected as the main research method for several reasons. First and foremost, during the initial trip to South Africa, only a limited number of legal and policy documents relevant to refugee reception policy were located. During discussions with academics in South Africa and Zambia at this time, it became apparent that there was an assumption (which was subsequently confirmed during the second field trip) that this situation would be replicated in Zambia. Thus, an alternative method of collecting data was required to address the research questions. Interviewing as an approach was selected over other methods (such as surveys and questionnaires) due to its suitability for the explanatory nature of the research questions (Gray, 2013), time constraints and concerns regarding the availability of key decision-makers. In addition, the flexibility inherent to KIIs provided the room to explore and adapt in response to new ideas and concepts emerging from the interviews.
It was also decided that key informant interviews best fitted the state-focused analysis of the macro- and micro-level structures approach taken by the book. As Kumar (1989) observes, KIIs reveal the perspectives and motivations of persons involved with a specific issue or theme. As the information is obtained directly from knowledgeable people, these interviews can provide data that is not possible via other methods. For instance, KIIs can reveal the personal experiences, observations and underlying motivations and attitudes of interviewees (Kumar, 1989). They can show not only what people do but also why they do it. This means KIIs have the potential to add new insight into why reception policies at the national and local level are implemented (or contested) – even ones that appear to conflict with national legal frameworks. Thus, this method has the capacity to elicit new insights behind key decisions and actions that are unlikely to be found in the public domain.
The KIIs were then correlated and triangulated with insights gained from informal interviews with local academics and experts on the sub-region, and attendance at symposia, and the review of national legal and policy documents. The two additional methods were embraced to improve the validity and reliability of the overall project. As Kumar (1989) notes, the more that interview findings are correlated and triangulated with data from other sources, the more confidence can be placed on the findings. Finally, these approaches were also used to identify key areas or themes that were not apparent during the early stages of the KIIs.5
The data collection stage
This section considers core elements of the data collection stage of a successful research project, which included for this book, deciding who to interview (and who not to interview), the interviews themselves, and more ethnographical approaches, such as attending meetings between civil society and state officials. Data collection was ongoing throughout the three main stays in Southern Africa between 2016 and 2018. However, the KIIs did not start until the second visit in mid-2017 once the framing exercise had been completed.
Sampling for the key informant interviews
During the framing exercise conducted in 2016, appropriate ‘key informant’ groupings were identified (for example, ‘state officials’ and ‘civil society’). In line with the overall methodological approach, gaining new insight into refugee reception policies requires input from diverse perspectives (Flick, 2009). Nonetheless, the book adopts a state-focused perspective aimed at understanding how and why host states (and global agencies) respond to refugees in specific ways, and how these approaches shape refugees’ attempts at engaging with local communities and markets. As a result, the sampling process for this book was orientated to finding groups of participants whose perspectives on reception would be the most instructive for this form of analysis.6 Accordingly, the groups of key informants were decided according to which entities were directly involved in reception policies at the international, national and local level. In total, three distinct groups of key informants were identified: Group A: Government Officials (national and local level); Group B: INGO Officials; Group C: Local civil society, refugee/migrant leaders and refugee experts.7
Due to the overall state-focused approach taken, it was decided that the involvement of large numbers of refugees was not essential to answering the research questions. The groupings of key informants therefore initially stayed at the level of refugee leaders (or ‘above’). An exception was nevertheless made to this approach, with two ‘refugee experts’ being interviewed in Lusaka, Zambia. This was done due to a lack of street-level organisations and civil societies working on these issues in Zambia. After consultation with other refugee leaders and civil society, two refugees were selected as expert sources of information to discuss how processes work in both the settlements and within urban spaces.8 Care was taken to not simply ask for their ‘stories’, but to treat them like any other key informant interviewed on the topic of refugee reception (Reed and Schenck, 2023).
Turning to the sampling of individual participants, key informants from each grouping (A to C) were selected for interview by purposive sampling and snowball sampling. As a first step, purposive sampling was used to select the KIIs, with the key criteria for selection being that they possessed an intimate knowledge of the subject or theme on which they were being questioned. In turn, ‘intimate knowledge’ was based on the participant’s professional expertise or contribution to a specific project or programme relating to refugee reception.9 To locate these potential interviewees, local and international researchers and academics were consulted via the two affiliations in South Africa and Zambia during the framing exercise. These consultations generated a lengthy list of possible informants for each grouping within each country.
Due to the nature of KIIs, the possibility of additional snowball sampling was also built into the design of the project. Therefore, when key informants suggested other potential participants, there was sufficient time allocated in both locations to conduct additional interviews. Snowballing offers real benefits for locating difficult to reach or hidden populations (Atkinson and Flint, 2001) and this was especially useful for identifying refugee and migrant leaders and civil society in Johannesburg and Lusaka.
In total, sixty-four semi-structured interviews were conducted across the range of groupings identified.10 The sample ended up being larger than is often recommended in the literature for KIIs (Kumar, 1989; Rudestam and Newton, 2007). In part, this was to avoid selection bias and ensure sample accuracy and precision (Bernard, 2011).11 In addition, the decision to end the KIIs was taken when no new information was uncovered in each grouping. As Rudestam and Newton (2007) note, this type of sampling is done to saturate a concept. In other words, the interviews continue to the point where the researcher is comfortable that the specific issue (and its relationship with other concepts) has been comprehensively explored so that it becomes theoretically meaningful. However, total saturation may never fully occur because each new participant is likely to have something unique to provide (Josselson and Lieblich, 2003). Therefore, as Rudestam and Newton (2007) suggest, it is vital to collect sufficient data (hence the large sample) to represent the breadth and depth of the concepts being investigated.
The interview process
As most of the participants were high-level bureaucrats or elite members of relevant communities with demanding jobs or roles, there was an expectation of only obtaining one-off interviews with the key informants. Therefore, semi-structured interviews were selected as the most appropriate form of interview-style (Bernard, 2011). This type of interview allows the interviewer to frame open-ended questions around the central theme of the study, whilst also having the flexibility to allow for the conversation to shift to new areas or points of interest (Kvale, 1996).
Separate interview guides were created for each grouping of key informants (Bernard, 2011). These help to produce more systematic, reliable and comparable qualitative data (Sewell, 1998). There was no need, however, to note every item nor specific questions that would be asked in each interview (Sewell, 1998). Indeed, because the intention of KIIs is to investigate key topics in depth, the guides were deliberately concise to avoid the risk of interviews covering too many topics and resulting in superficiality (Kumar, 1989).
Interview questions arose from the interview guides and had a simple structure to them. The questions aimed to elicit detailed information on topics covering the reception of refugees. These included, for example, the role of different actors in the reception offered to refugees and the interaction between refugee movement in-country and state and UN structures. Thus, the questions remained open-ended and were designed to avoid simple yes or no answers. This allowed respondents to explain what they meant in their own words and produced a relaxed conversational flow (Schoenberger, 1991). Follow-up questions were asked to probe further on specific topics or themes.
The interviews were conducted in English and recorded digitally, subject to the interviewees’ written permission. In addition, notes were made throughout each interview. These included: (1) recording most answers given to the questions; and (2) any additional insights that occurred to the researcher during the interview (referred to hereafter as ‘field notes’). After each interview, field notes were kept separately from the responses of the interviewees and ultimately used as a source of supplementary information, helping shape additional enquiries in future interviews.12
KIIs in South Africa were undertaken during the second field trip between June 2017 and November 2017, in Johannesburg and Cape Town. Gaining access to networks of civil society and refugee and migrant groups in South Africa was relatively straightforward. Indeed, most organisations and community groups responded to the first email enquiry. In terms of state entities, city-level departments were interested in participating as well as entities with specific human rights mandates (such as the Human Rights Commission). In contrast, it was extremely difficult to gain access to the Department of Home Affairs, which has the national mandate to deal with refugee matters. After eventually gaining research approval from the department, interviews were able to take place.
A similar situation occurred with UNHCR, with the South African office generally unwilling to speak to academics. For example, after months of emails, a senior protection officer finally agreed to meet. However, upon arriving for the interview, the officer stated they would not answer questions about UNHCR’s role in South Africa or its relationship with the host state. This lack of access in and of itself says a great deal about the role of UNHCR in-country and will be examined further in later chapters. Alternative informants were ultimately found to gain an agency perspective – including UNHCR staff from the Southern Africa office.
The third trip involved spending five months in Zambia (from November 2017 to March 2018) based at Southern African Institute for Policy and Research (SAIPAR) in Lusaka. In contrast to South Africa, research looking at migration or refugee movement is relatively sparse in Zambia. Furthermore, civil society is small, especially in terms of migrant and refugee issues. As a result, obtaining interviews with key informants from these groupings was challenging. In part, these dynamics are due to the refugee settlements and their remote locations. However, it was also to do with the ruling political settlement moving towards more of an authoritarian rule, meaning that the political space for civil society and academics was rapidly shrinking.
Conversely, there remains a large network of international NGOs and agencies present in Zambia. These organisations have greater freedom in terms of discussing social and political issues compared with the situation in South Africa. Overall, they also have a good working relationship with the Zambian state. As a result, international organisations were very accommodating in response to interview requests. The major difficulty in obtaining interviews in Zambia arose due to communication issues, with civil society and state officials rarely responding to emails. It became apparent that WhatsApp was the best means of contacting key informants.
Interviews normally lasted between forty-five minutes and one hour, with most interviewees being accommodating and engaging. In rare instances, high-level bureaucrats within UN and state systems were uncooperative during the meetings. In these situations, the purpose of the study was re-emphasised and a commitment to gaining a non-biased and ‘complete’ understanding of refugee reception was made. On rare occasions (such as the UNHCR interview mentioned above) when all attempts at creating a genuine and open dialogue failed, the interview was ended early.
Legal and policy documents
The second method of data collection was the use of legal and policy documents originating from the selected countries. Due to the framing of the research questions, documents selected for review and analysis were limited to those produced by the national or local level governments. The existence of official documentation was unearthed either by prior desk-based research, through the framing exercise in South Africa in 2016 or via the KIIs. Documents were then obtained via the Internet or official channels. Legal and policy documentation was more accessible in South Africa than in Zambia, because in Zambia a great deal of refugee policy is not recorded publicly.
The obtained texts were utilised to examine the ‘status quo’ and official policy and procedures at the national level. In addition, they helped develop an understanding of the varying ideational factors that permeate at different government levels. For example, through an examination of the 2016 South African Green Paper on International Migration, it was possible to advance insight into how the Department of Home Affairs conceptualises forms of cross-border migration.13 To assess biases within the official documents, the authorship and intended readership of the documents was also considered and examined.14 Finally, other grey literature, such as parliamentary records and media reporting, were consulted to understand particular debates around key legislation and policy documents.
Informal interviews and symposia
This quasi-ethnographic method emerged from the framing exercise conducted in 2016. During this exercise, six informal interviews were carried out with academics and local experts, as well as attendance at numerous state and NGO meetings. This approach was developed and incorporated into the research design, becoming the third source of research data. Thus, during the second and third visits in 2017 and 2018 a further six informal interviews with academics or local experts were conducted. Furthermore, numerous local level and national level meetings between civil society and state officials were attended between 2016 and 2018. These meetings covered key issues relating to state responses to the arrival of refugees in both Zambia and South Africa. For example: the ability to obtain legal papers and negotiate the asylum processes successfully; the ability to move freely within urban areas; and solidarity initiatives with local communities. These events also touched on key material, ideational and institutional factors that were influencing state approaches to reception and were compared to the information obtained from KIIs.
The analysis stage
The analysis stage of qualitative research is traditionally the stage that either receives the least discussion, is left opaque or simply omitted (Nowell et al., 2017; Thorne, 2000). Yet, for qualitative research to be meaningful, all stages need to be transparent, rigorous and methodical (Attride-Stirling, 2001). For this book, a form of thematic (or content) analysis was adopted to examine and interpret the data. In essence, this is a method of identifying and analysing themes and patterns within qualitative data (Braun and Clarke, 2006). Under this broad approach, a theoretical thematic examination was adopted rather than a more inductive thematic style. This means that instead of themes emerging solely from the data (that is, purely data-driven or inductive), themes also emerged from the study’s overall theoretical stance. Thus, the formation of categories used in the coding of the empirical data emerged during the initial desk-based research, the framing exercise and the crafting of the original research questions.15 Subsequently, a systematic approach to the analysis was taken, whereby these categories and theoretical ideas were applied to the data gained from the interviews (Becker, 2009). However, the categories and concepts were also continuously assessed in light of the empirical data and modified if deemed appropriate.
The first analytical step was to transcribe all the interviews from the digital recordings. At this point, handwritten notes from the interviews were also reviewed to improve precision. The second step was to code the transcripts via a systematic recording of the data.16 The coding involved categories (and families of categories) that were devised during the framing exercise, with each interview first being individually analysed (vertical analysis). Coding categories were recorded on the transcripts, with a summary sheet for each interview being completed, which listed the identified categories and corresponding page numbers for that interview. As the interviews progressed and further understanding emerged, so new categories were also introduced, and others amended.
Once all the interviews had been conducted, the transcripts were compared against each other using the finalised categories (horizontal analysis).17 For a category or theme to be counted, there needed to be several instances of the theme across the data set. Nevertheless, quantity alone did not dictate the theme’s relative importance (Braun and Clarke, 2006). Thus, an exact number of instances across a data set was not needed for a theme to be ‘counted’. Indeed, the ‘keyness’ of a theme is not necessarily dependent on quantifiable measures. Rather, based on the researcher’s judgement, a theme or category becomes important ‘when it captures something important in relation to the overall research question’ (Gray, 2013:92).
A core goal of thematic analysis is to reduce the raw data to a manageable level (Flick, 2009). Therefore, once key categories and themes were confirmed, the material was then paraphrased (first reduction) both vertically (within each transcript) and horizontally (across transcripts). Similar paraphrases were then ‘bundled’ and summarised (second reduction) (Flick, 2009).18 In this way, at the end of the thematic (content) analysis, the data that emerged was manageable, and related directly to the research questions and themes that originated from the study’s overall theoretical stance.19 Nevertheless, the data sources (in the form of original interview recordings, transcripts and field notes) were continually revisited during the analysis and write-up stages to explore the ‘thick’ descriptive content. As Richards and Richards (1994) caution, analysis should never entirely depart from the data.20
Validity, ethics and reflexivity: conducting field research in Southern Africa
This final section moves to consider the more evaluative and reflexive components intrinsic within a successful research project. The section firstly investigates how to achieve validity and reliability in analysis, before turning to the positionality and the role that timing plays in the collection of data. The chapter ends with an examination of relevant ethical considerations and potential limitations of the methods used for this book.
Validity and reliability
This section outlines the steps taken in the preparation, implementation and analysis stages to ensure scientific rigour, specifically, the measures put in place to safeguard reliability and validity. Reliability in this context refers to the replication of a study under similar circumstances (Rudestam and Newton, 2007). To achieve this, Gray (2013) underlines the importance of providing an audit trail. As such, this chapter has produced a detailed record of the processes involved in conducting the research. This has included explaining the following:
- the configuration and role of the initial framing exercise;
- information on how interview guides were created;
- the format and types of questions asked in the KIIs;
- the use of national and local documentation;
- the additional ethnographic elements of the fieldwork, including the use of informal interviews, field notes and attendance at local workshops, conferences and meetings;
- how the raw data was analysed, as well as the approach taken to coding; and
- the reflexive elements involved, such as the amending of research questions and key themes during the progression of the data collection and analytical stages.
The validity of a specific study is examined in two ways, by looking at internal and external validity. The internal validity of this book (that is, having sufficient evidence to show that the findings are supported by what was observed) is outlined extensively below. In contrast, due to the design and overall approach, the external validity of the study (or the ‘generalisability’ of the findings) receives less attention.21 As this is a comparative investigation of two case studies, the book is focused on the context of specific settings (namely South Africa, Zambia and Southern Africa). As Rudestam and Newton (2007:113) argue, ‘generalization is the task of the reader rather than the author of qualitative studies’. Thus, any generalisations made to other settings in this book are intentionally modest.
By comparison, significant weight has been attached to demonstrating the internal validity of the book’s findings. Put simply, internal validity is a way to measure whether research is rigorous or not. Gray (2013) outlines several techniques, which were followed for this book, to demonstrate internal validity during the analysis and presentation stages. Firstly, the internal validity is enhanced by the time spent in both locations. Indeed, I lived on and off in Southern Africa for over four years during the life of this extended project, which permitted the time and access to check for any distortions in the data and to explore the topic (why states adopt specific reception policies) in sufficient detail (Rudestam and Newton, 2007).
Secondly, to further reduce the risk of confirmation bias and to improve the validity of the study, interpretations of the data were checked with local academics and experts in both countries.22 To this end, academic meetings and seminars were held in both South Africa and Zambia during the analysis stage. At meetings with local academics (via the two affiliations), initial findings were presented, and discussions ensued in relation to the data collection and data interpretation. In addition, elements of the findings were presented in public seminars at local universities in both Johannesburg and Lusaka. These occurred during the second and third main visits and before the write-up stage of the project began. In these seminars, the research design, the methods and the initial findings were presented. Following the presentation of the research, academics, experts and civil society representatives asked questions, raised critiques and suggested rival explanations for the researcher to consider. Thus, throughout these stages, the core theoretical assertions made in the book were scrutinised, questioned and explored in the face of divergent ideas from academics and experts.23
The final technique used to establish the internal validity of the research project was the use of methodological triangulation.24 This is achieved by cross-checking and corroborating the data via multiple sources (Rudestam and Newton, 2007). In terms of application, it is important to combine methodological approaches which are distinct in their focus and in the data they provide (Flick, 2009). This was achieved through a varied approach to data collection. Indeed, data was collected from a range of different sources including the perspectives of key informants and academics; primary source documentation set out the official approach of states and observations and impressions gained during public meetings between key informant groupings. Thus, throughout the visits, and during the analysis stage, methodological triangulation was used to cross-check extensive amounts of diverse data.25
Positionality
A key issue that pertains to the validity of the data is positionality. When conducting research, it is imperative for the researcher to be aware of how their perspective and interpretations of what is happening can be affected by their own positionality (Clark-Kazak, 2022). As observed by Maher and Tetreault (1994:118), ‘knowledge is valid when it includes an acknowledgement of the knower’s specific position in any context’. Thus, for this book, it was not possible for me as the researcher to completely separate out my positionality as a white, British male, from the quality and validity of the data.26 As noted by Kuch (2016), the value system I have internalised from living in London (that can broadly be defined as secular and liberal) impacts on the questions I ask, my perceptions of people and events, and the arguments I develop. Similarly, the interviewee’s perceptions about race, gender, Western values or the role of the British in the history of their country/ host country may all influence their attitudes towards me as the researcher.
Substantial time was, therefore, spent reflecting on these complexities, particularly on the positionality of the researcher and the respondents, and the specific power dynamics that could emerge within the interview setting (Fedyuk and Zentai, 2018). When crafting interview questions, consideration was given to how an inquiry might position the researcher in relation to the interviewee, and what the implications of this might be for the interviewee’s life (Agee, 2009). In addition, there is a danger when designing and conducting interviews that they merely end up aligning with the individual researcher’s interests or preconceived ideas. To reduce this risk, interviews with refugee leaders, experts and civil society representatives were made as relaxed as possible, with participants given sufficient space to raise their own issues in relation to the broader themes of the study.
In actuality, during the interview process identities (ascribed and assumed) were generally fluid rather than fixed. Consequently, the notion that the researcher is always the one in the position of power did not consistently transpire. Depending on the participant, the researcher was perceived in different ways, for instance as an expert, a student, an outsider with little understanding of the local context, a colleague or ‘comrade’ or as a link to a wider audience. Indeed, during the interviews with refugee leaders and local civil society members, this full spectrum of identities was often observed. Likewise, the interviewees often presented themselves in varied ways within the same interview, for instance as refugee, expert, victim, guest, host and information source.
Research on migration and forced migration also ‘takes place in contexts of global inequalities’ (Clark-Kazak, 2022:17). I was aware when conducting the research that as a researcher from London with a UK passport, I had access to resources, including time and money. Within this context, charges of extractivism have rightly been levied against researchers from the minority world. Building on this, Bilotta (2020) suggests that when working within related migration fields, the principle of ‘do no harm’ is not sufficient. I take this to mean that as privileged researchers we need to be doing more, either in terms of contributing to ending suffering (however modest that might be) (Clark-Kazak, 2021), or saying no to conducting certain forms of research (Hagen et al., 2023). These are difficult conversations for the forced migration field, and I do not purport to have the answers. Taking a lead from Mayblin and Turner (2020:36), perhaps a first step is the need to ‘sit with the discomfort’ of these charges of extractivism, and take them seriously. In an attempt to address some of these concerns (as it is not possible to remove all of them), the intention was always to publish this book with open access. Further, additional time and assistance was given during prolonged visits with refugee groups, and forms of compensation given to refugee experts and groups who offered their time for the interviews. Nevertheless, it is essential this topic remains an ongoing conversation for researchers and the wider field. More collaborative work is needed, with the development of alternative methods and new ways to reduce inequalities within global research agendas (Chatzipanagiotidou and Murphy, 2022; Grabska and Clark-Kazak, 2022).
Timing of the research
The majority of the research for this book was carried out in South Africa and Zambia between 2016 and 2018, which was a particularly notable time to conduct work in this region. During this time, both countries witnessed their ruling political settlements engaging in high-level discussions involving fundamental shifts in the states’ response to refugees, while simultaneously dealing with outbreaks of cholera, xenophobic violence and increased influxes of refugees and other forced migrants.
Zambia had recently signed up to the CRRF in 2016, with the now former President Lungu attending UN meetings in New York and committing to considering opening up the urban space for refugees. If implemented, this would see a move away from the country’s traditional settlement approach to reception. This meant that the issue of refugees was a hotly debated topic within specific government bodies and international NGOs (INGOs). In addition, during the period of interviewing in 2017, two relevant events had recently surfaced and were ongoing. Firstly, renewed unrest in neighbouring DRC meant that considerable numbers of refugees were arriving at the border with Zambia, with emergency transit centres having to be set up. Secondly, an outbreak of cholera was declared on 6 October 2017 in Lusaka. Thus, these events added to the impression that migrant and refugee issues were – at least at that moment – a high priority for international agencies and some government departments.
At the same time, the national government in South Africa was going through the process of publishing the Green and White Papers on International Migration (DHA, 2016a, 2017). These two policy documents set out concrete proposals for moving all asylum-seekers to the processing centres at the border, thereby restricting access to the urban space for forced migrants. These publications generated a great deal of debate between Home Affairs, UNHCR, civil society and academia. Furthermore, after a relatively quiet 2016, incidences of xenophobic attacks on migrants and refugees in large urban areas started to increase throughout 2017 and 2018 (BBC, 2019).
The combination of these varied large-scale events and the high-level policy discussions taking place in both countries indelibly influenced and shaped the context of the conducted research. Inevitably, if the research had been carried out in less fraught times certain interviews might have been less rushed or politically charged. Nevertheless, the timing of the project is ultimately understood as being advantageous (if a little fortuitous). There was sufficient time in both countries to develop key insights into the maintenance of long-term and entrenched approaches to refugee reception as well as an examination of the reasoning behind potential shifts in policy.
Ethical considerations relating to the adopted methods
Research that focuses on migration issues retains specific ethical challenges (Clark-Kazak, 2021). In part, this is because migration, and especially forced migration, can result in unstable or dangerous situations, including precarious or temporary legal status (Anderson, 2010). Equally, migration across the globe is increasingly criminalised and/or vilified by states and local populations (Clark-Kazak, 2021). Thus, projects that engage with these populations need to be held up to relevant ethical principles (Müller-Funk, 2021).
In November 2016, during the first trip to South Africa and following the finalisation of the research design, ethical clearance was gained from the University of London. In addition, the project followed the ethical guidelines for conducting interviews as set out by the Refugee Studies Centre, University of Oxford (Refugee Studies Centre, 2007). These include acknowledging the responsibilities of the researcher towards participants, host communities and governments and the wider society when carrying out research. Furthermore, gaining ethical approval and adhering to guidelines is one part of the responsibility undertaken by a researcher when conducting empirical research. As highlighted below, it is important to consider existing and emerging ethical concerns throughout the entire research process (Miller and Bell, 2002).
Several core ethical considerations relating to refugees and local communities were identified during the design stage and as such were scrutinised and considered throughout the life of the project.27 Firstly, informed consent is widely seen as the ‘cornerstone of ethical practice in research that involves human participants or personal data’ (LSE, 2019:1). Following the example of Corti et al. (2000) and Sin (2005), when obtaining informed consent from each participant, several key points were highlighted verbally. These included informing participants that they could terminate the interview or renegotiate consent at any point during the research process, and that they had the opportunity to opt out of the project entirely. The steps taken to ensure confidentiality and anonymity were also made explicit. In addition, before each interview formally started, participants were given a participation form (which set out the goals of the project) and a consent form. On the consent form, participants were asked to agree to the recording of the interview and for the content of the interview to be used in the study. After explaining the content of each form and responding to any queries, each participant was asked to sign to indicate their consent to these arrangements.
Secondly, with regards to confidentiality, the consent form gave participants the option of remaining entirely anonymous or allowing their name to be used in the book. While some participants asked to be named, ultimately, I decided to anonymise all the participants’ names.28 Given this desire for self-determination by certain participants (Clark-Kazak, 2017), this decision, while not taken lightly, was not ideal. The reasoning was based on the tense political climate surrounding refugee and migrant reception within the two countries at the time of the research. Indeed, this has only escalated further in South Africa since the initial collection of data was completed, with renewed xenophobic attacks in cities such as Johannesburg and Cape Town occurring in 2019 (BBC, 2019).
Limitations of the book’s research design
Firstly, it can be argued that the use of KIIs risks privileging the perspectives of states and international organisations over other actors such as refugees and civil society. A concerted effort was therefore made to find interviewees from a wide pool of informants in both countries, with the range of KII ‘groupings’ ultimately including local civil society members and migrant and refugee leaders. Nevertheless, the lack of migrant community groups and civil society members working on migrant issues in Zambia, in contrast to the abundance of international organisations in that state, can be considered a constraint in terms of the final data set.
Following on from this point, the role of refugees and migrants in this research project warrants reflection. Within the academic field, there are ongoing discussions and critical self-reflections surrounding the form and role that refugees and migrants should play in research (Krause, 2017; Hagen et al., 2023). A key outcome of this introspection has centred on the idea of active collaboration and participation with the target population during the design and investigation stages (Fiddian-Qasmiyeh, 2020). Thus, a limitation of this research project is the absence of refugee voices. This is particularly relevant when attempting to gain new understanding of how reception policies may affect this population on the ground.
The lack of refugee participation is a valid constraint and was considered during the design stage. Ultimately it was decided that due to the state focus and ethical concerns, the engagement of large numbers of refugees was not necessary to answer the book’s central questions. The primary focus of the project is on key stakeholders at the international, national and local level who are involved in the creation and implementation of reception policies. Due to this approach, it was not seen as justified to request considerable time from populations who are living in precarious situations. As Turner observes, a balance needs to be struck between the value placed on a refugee’s time and privacy versus a need to include their views on a given topic (in Hagen et al., 2023). This thinking was further crystallised during the initial period of fieldwork in South Africa, where a general sense of research fatigue was found amongst refugee groups within Johannesburg.29
Linked to this overall approach to refugee participation was the decision not to conduct research directly in the two settlements in Zambia. Sufficient data was collected through the interviews with key informants who work in the settlements or who travel there regularly. This information was also supplemented by available existing research that had been conducted within the settlements. As this research did not involve directly interviewing refugees, there were ethical concerns about visiting people’s homes and communities in the absence of an obvious immediate benefit of the research for them. The next chapter now turns to present the results of this methodology through the case studies of Zambia and then South Africa.
Notes
1. Exploratory research is conducted when little is known about a phenomenon and the ‘problem’ is unable to be defined succinctly (Boru, 2018; Saunders et al., 2007).
2. These amounted to what Spradley (1979) sees as ethnographic interviews, whereby they are not formal interviews but are elevated above ‘friendly conversations’.
3. Framing or scoping exercises such as informal interviews should be done as part of ‘the process of observing a social setting of interest’ (Cohen and Crabtree, 2006:331).
4. See Maple (2016).
5. For example, if key informant understandings differed from official documentation or past findings, these disparities raised additional queries as the study progressed.
6. See Flick (2009).
7. Participants from Set C had travel expenses reimbursed and were offered a non-monetary form of thanks, that is, a phone card or lunch. See Bernard (2011).
8. The refugee experts were individuals (female and male) who had lived in Zambia for over five years and had lived for a substantial amount of time in both the settlements and Lusaka.
9. See Kumar (1989).
10. There was a broad balance between the three groups, although as examined below, there were differences and some limitations within each case study.
11. See Atkinson and Flint (2001).
12. See Rudestam and Newton (2007).
13. See DHA (2016a).
14. See Grant (2018).
15. See Flick (2009).
16. See Kumar (1989).
17. Inevitably, these forms of horizontal analysis informally started once interviews began, with connections and contradictions being observed and retained (Spiggle, 1994).
18. Nevertheless, key quotations from transcripts were kept as this process of reduction was carried out.
19. See Braun and Clarke (2006).
20. See also Veroff (2010a, 2010b).
21. See Rudestam and Newton (2007).
22. The findings of the study were also compared to previous literature relating to the reception of refugees (Rudestam and Newton, 2007).
23. See Rudestam and Newton (2007).
24. Not specifically set out by Gray (2013), but by many others (Hentschel, 1998).
25. See Pratt and Loizos (2003).
26. See Kuch (2016).
27. See Pittaway et al. (2010); Mackenzie et al. (2007).
28. Renzetti and Lee (1993); Sin (2005).
29. See Omata (2019).
We use cookies to analyze our traffic. Please decide if you are willing to accept cookies from our website. You can change this setting anytime in Privacy Settings.