Drawing from her research, in this blog PhD candidate Heidi Maiberg evaluates measurement issues around extremism deradicalisation and disengagement programmes.
In recent years, the number of deradicalisation and disengagement programmes to support extremists in their journey of leaving hate behind has increased significantly, and worldwide. To understand their impact, it is necessary to test programmes’ methods. This is especially important, because vague or untested assumptions might prove counter-productive in practice. For instance, they can contribute to further radicalisation or recidivism instead. While there are various risk assessment tools and types of evaluation, practitioners lack ways to understand the intervention's core impact and evaluate the program's efficiency. Literature shows that most deradicalisation and prevention programmes do not even analyse their effectiveness.
In my PhD research, I explore how impact is assessed in tertiary prevention programme interventions in Europe and North America. Through 19 interviews with intervention providers and former extremists, I investigate questions such as 'Is impact assessed?', 'Have organisations defined the aim of their programme and how they understand 'success'?' and 'How do the organisations understand when the participant is ready to leave the programme?' By analysing intervention providers' perspectives, my research offers insight into the practices of contemporary programmes and into dilemmas that the exit workers are facing. More than this, it offers practical suggestions that, if put into practice, could be used to solve current bottlenecks in the system and change how the field operates today.
Why Practitioners Don’t Measure
In November, I had presented my initial findings at the AVERT Research Network's Research Symposium: "The Space Between: New Directions in Countering Violent Extremism (CVE) Interventions” at Deakin University in Melbourne, Australia. My presentation focused on some concerning gaps evident in my research. The most fundamental is that 'Impact assessment' is vaguely defined among intervention providers. Similar to other terms in terrorism studies (‘terrorism’, ‘extremism’, ‘radicalisation’), there are a plethora of interpretations for the terms ‘impact' and 'assessment'. For instance, 'impact' can be seen as the participant's development during the process. Some organisations discuss 'impact' in terms of funder evaluations of their programmes. For others, impact is related to participants who drop out of the programme.
Programmes also face several dilemmas in terms of measuring the impact of their work. The majority of the intervention providers struggle for instance with creating clarity, systems and generalisation in this often very individual, tailor-made world. Only a small number of intervention providers have defined 'deradicalisation' or what can be considered 'success' or 'failure'. One exit worker emphasised that it is difficult to come up with strict definitions when every individual has their own path and one's success is another's failure.
Because of the mentioned dilemmas, the majority of intervention providers I spoke to do not evaluate or assess their initiatives with existing tools. While some simply do not see the need, other organisations identified a need and decided not to evaluate anyway. They had four main reasons for this. First, assessing or evaluating the process (in front of, with the participant or in general) might unnecessarily institutionalise them, making the exit workers resemble prison, police or probation officers. This is something they aim to avoid. Their clients fulfil enough scales, questionnaires and paperwork with other institutions; exit workers do not want to lose the close relation that they have with their clients/mentees. Second, there are no suitable tools. Most practitioners were aware of risk assessment tools like VERA-2R, ERG-22+ or TRAP-18 that entail a list of indicators that support the professional judgement of risk assessment and risk management of terrorists and violent extremists. However, the exit workers believe that risk assessment tools do not grasp the complexities of the deradicalisation process. While risk assessment tools could theoretically be used in some phases, instruments that cover the interdisciplinarity of the process are needed.
Third, professionals see that even if impact assessment is possible, nothing can guarantee that the individual they are working with will not fall down the rabbit hole, again. One exit worker said “we cannot ever say that we know exactly what the outcome would be. And it is the same thing with risk assessment ... There is no guaranteed way to assess will somebody [revert to].. recidivism or not.” Because of that, most practitioners do not want to provide information about the exact outcome of their work.
Last, most intervention providers work based on their ‘gut feeling’ and do not see the need to change their current practice. Or, even if they want to do it, the above-mentioned dilemmas and reasons make it difficult for them to alter their way of work.
What to Measure
Nevertheless, some exit workers do measure the impact of their work. For instance, they monitor whether the client is fulfilling the set agreements and goals. Also, many organisations stay in touch with the participants and check in on them regularly. They see that the individual is ready once 'other' life takes over. The client has found new friends, work opportunities, a home, and other ‘things’ they have wished for.
In summary, since there is no consensus on definitions, it is very difficult to standardise the field in terms of terminology, work objectives and methods. While the spread of tailor-made approaches is positive, such a paradigm does not easily support impact assessment or programme evaluation. The findings show that exit workers agree with the need for evidence-based approaches, however, they believe they do not have any suitable tools. In the next phases of my research, I focus on two main questions. First, considering the current practice of exit workers, how much should the field be standardised regarding terminology and work methods? Second, what are the appropriate tools for measuring the efficiency of deradicalisation and disengagement programmes? And if there are none, is it time to abandon the aspiration of evidence-based assessment to evaluate these programmes?
Heidi Maiberg is a PhD candidate in Criminology. Her dissertation titled “However you feed a wolf she will always look towards the forest? Dissertation on the reasons and the extent of using impact assessment practices of tertiary prevention programme interventions“ focuses on impact assessment and evaluation practices of deradicalisation and disengagement programmes. Her research and participation at AVERT were funded by the Republic of Estonia Education and Youth Board's Kristjan Jaak Scholarship for degree studies abroad.
