One phrase appears with remarkable consistency in impact assessment reports: “A mixed-methods approach was adopted.”
It sounds reassuring. It signals methodological sophistication. It suggests that numbers and narratives have been combined to produce a richer understanding of impact.
But has it?
In many reports, “mixed methods” simply means that surveys were conducted alongside a few interviews or case studies. Quantitative findings are presented, followed by a handful of quotations, and the study is described as mixed methods.
Yet mixed methods are not methods mixed.
Collecting qualitative data does not automatically make a study qualitative. Likewise, conducting interviews does not make a study mixed method. If interview responses are merely converted into frequencies—how many agreed, disagreed, or expressed satisfaction—they are still being analysed quantitatively.
Methodological rigour is not determined by the number of methods used. It is determined by how those methods are designed, analysed, integrated, and interpreted.
Part of the confusion comes from equating data collection methods with research design.
Surveys are assumed to represent quantitative research. Interviews are assumed to represent qualitative research. Use both together, and the study is labelled “mixed methods.”
But mixed methods is not defined by using multiple tools. It is defined by how different forms of evidence work together to answer the evaluation question.
The integration is the methodology. The methods are simply the means.
A genuine mixed-methods study combines qualitative and quantitative approaches because each contributes something the other cannot.
The relationship between the two is intentional from the outset—not something added while writing the report.
The mixed-methods literature identifies several recognised research designs. Three are particularly relevant to impact assessment.
Quantitative and qualitative data are collected during the same phase of the evaluation and analysed independently before being brought together.
The evaluator then asks:
Imagine a CSR programme designed to improve school attendance.
Survey results show attendance increasing from 72% to 91%.
Interviews reveal that attendance improved not simply because classrooms were renovated, but because mothers' self-help groups began monitoring absenteeism and encouraging children to attend school.
The quantitative findings establish what changed.
The qualitative findings explain why it changed.
Together they produce a richer understanding than either could provide alone.
Sometimes the numbers raise questions.
This design begins with quantitative analysis, followed by qualitative research that explains the patterns observed.
Suppose survey findings show that farmers with smaller landholdings experienced substantial income gains while larger farmers saw little improvement.
Rather than merely reporting the difference, the evaluator investigates it through interviews.
Those conversations may reveal that smaller farmers adopted new agricultural practices more readily because they depended more heavily on extension services, whereas larger farmers continued traditional cultivation methods.
Here, qualitative research explains what quantitative analysis alone could not.
Sometimes the evaluator begins with very little understanding of the context.
In these situations, qualitative exploration comes first.
Interviews, observations and focus group discussions help identify important issues, refine programme theory and generate hypotheses.
These insights then shape the quantitative study.
For example, an evaluation of a women's entrepreneurship programme may discover through interviews that confidence, family support and mobility are stronger determinants of business success than access to finance alone.
The subsequent survey measures these newly identified factors across a larger population.
Rather than measuring only what is easy to count, the evaluation measures what actually matters.
There is no universally superior mixed-methods design.
The choice depends entirely on the evaluation question.
The important point is that the choice is intentional—not accidental.
Poorly designed mixed-methods studies create an illusion of rigour.
They generate more data. They do not necessarily generate more understanding.
Well-designed mixed-methods studies do something very different. They:
The value of mixed methods lies not in collecting more evidence, but in producing better explanations.
As the impact assessment ecosystem matures, expectations around methodological quality must mature with it.
Mixed methods should never become a label attached to an impact report simply because interviews were conducted.
It should represent a deliberate research strategy where qualitative and quantitative evidence are intentionally brought together to answer questions neither could answer independently.
Rigour is not about employing more methods.
It is about choosing the right methods, for the right reasons, and integrating them in ways that deepen understanding.
Because mixed methods is not “methods mixed.” It is the thoughtful integration of different ways of knowing—so that impact assessment moves beyond documenting change to explaining it, learning from it, and ultimately helping organisations design interventions that create greater and more sustainable impact.
Impact Assessment Accreditation Forum