Traditional versus novel ethical considerations when using CAQDAS
António Pedro Costa, University of Aveiro (Portugal)
Researcher at the Research Centre on Didactics and Technology in the Education of Trainers (CIDTFF), Department of Education and Psychology, University of Aveiro, and collaborator at the Laboratory of Artificial Intelligence and Computer Science (LIACC), Faculty of Engineering, University of Porto.
Martin Tolich
Martin Tolich’s first degrees were from Auckland University and his Ph.D. in Sociology was from University of California, Davis. He is currently Associate Professor in Sociology at Otago University, New Zealand. Martin has authored and co-authored numerous books on Research Methods and Research Ethics for Pearson, Oxford University Press, Routledge and Sage.
Abstract: In the last 30 years, numerous IT solutions have arisen to support researchers in almost all research project phases, reducing manual work, although more mechanical or routine. How have those innovative solutions affected social science methodological and ethical considerations? This paper focuses on qualitative research in its various forms, highlighting the emergent and iterative epistemological features of qualitative data collection and analysis. Do qualitative researchers using IT follow different research practice paths than more traditional pen-and-paper analysis? The article finds IT social science analysis remarkably similar to traditional qualitative research analysis but highlights four procedures where IT-generated analysis differs from the traditional pen-and-paper counterpart. The article ends by explaining ethical solutions for these four procedures.
What Induces Us
Examining the research ethics considerations practised in qualitative analysis with and without computer-assisted software, such as webQDA (Costa, Moreira, & Souza, 2019), can reveal how the use of these tools may impact the primacy of methodological priorities.
Â
This article is a conversation between an exponent of computer-assisted analysis and an expert in qualitative research ethics. The expert wants to know how methodological and ethical considerations change using computer-assisted qualitative data analysis (CAQDAS). We begin the paper with five fundamental contrasts. First, the essential difference between quantitative research ethics using a questionnaire and qualitative research ethics using face-to-face interviews. Part 2 distinguishes between procedural ethics and ethics in practice. Part 3 examines qualitative research ethics on paper as if the analysis was conducted with pen and paper. This establishes what benchmarks for qualitative research ethics are. Part 4 considers methodological differences between qualitative research conducted with pen and paper and a CAQDAS, such as webQDA (Costa, Moreira, & Souza, 2019). Part 5 compares and contrasts ethical considerations practice between a pen-and-paper ethics exercise and a similar research design analysed by computer-assisted programs. Before beginning, we define the priority of the method as follows:
In quantitative research, the instrument (the questionnaire) is the method’s priority. In qualitative research, the researcher’s priority is the researcher, who simultaneously collects and analyses research data in situ. The research question is the method’s priority in mixed methods research.
1. Part 1: Quantitative and Qualitative Research Ethics
Quantitative research ethics using questionnaires are more straightforward than qualitative research ethics using one-on-one interviews. The research design of questionnaires involves an enormous amount of time, thought and energy. All ethical issues have been considered and operationalised when sending the survey to the respondent. The most important exemplar of this research design is the absence of a consent form that a respondent signs, agreeing to the ethical considerations offered by the researcher. The sentence, “filling out this questionnaire implies your informed consent,” is the total of the consent process.
Â
Qualitative research provides a participant information sheet and has the participants sign a consent form stating that they agree with the ethical considerations offered. When examining this consent process, there are Significant differences between quantitative and qualitative research ethics. “Filling out this questionnaire implies informed consent” captures a fleeting moment. Once the questionnaire is submitted to the researcher, anonymity is the norm. Even if the respondents wanted to withdraw their information after the submission, this is not possible in most cases, as the researcher cannot single out any submitted questionnaire. Â
Â
Participants signing a qualitative research consent form do not experience a fleeting moment. On the contrary, the participant and the qualitative researcher have an embedded relationship. For example, if the participant chooses to have second thoughts about participating and decides to withdraw from the project, this is quickly done. A second key difference between the face-to-face interview and the questionnaire is anonymity and confidentiality assurances. The term anonymity means “the situation in which someone’s name is not given or known.” This term is useless in qualitative research because the participants are always known in face-to-face interviews. The confidentiality assurance given means the participant is known to the researcher and that identity will not be shared with other people. What the participant said will be shared publicly, but no other person can link the person to the quotation.
2. Part 2: Procedural Ethics and Ethics in Practice
A domain assumption used in this paper is that the research discussed is conducted in countries where ethics review is mandatory. Rather than using a confusing array of abbreviations (IRB, HREC, REB) to denote ethics committees, this paper adopts Guillimen and Gillam’s (2018) classic distinction between procedural ethics and ethics in practice. Procedural ethics refers to a formal review by a committee that approves research.
Â
Guillimen and Gillam’s (2018) distinction is essential for qualitative research as it highlights an epistemological anomaly with qualitative research. Whereas the quantitative questionnaire discussed above corralled its ethical considerations within the research design stage of the study, qualitative research does not operate in such a linear, deductive manner. A qualitative researcher may apply their research ethics to procedural ethics, but the unique epistemology in qualitative research makes the ethical considerations change. This stems from four characteristics:
- The researcher seeks to understand an unknown world. Their problem has an aim, not a hypothesis;
- The research question is emergent. It materialises from the analysis of participant interview transcripts;
- The research question is iterative – whereas a questionnaire gains its reliability from a standardised set of questions, a qualitative researcher interview guide is informed by the participants;
- The qualitative researcher is a learner, and the participant is the expert.
These four characteristics create constant change in qualitative research design, and this uncertainty produces what Guillimen and Gillam (2018) call “ethics in practice”. Rather than corralling ethical issues in the review process, qualitative researchers must take responsibility for the ethical issues that develop in the field.
Â
Tolich and Fitzgerald (2006) capture a problem that any procedural ethics (committee) process has with qualitative research. They characterise procedural ethics for qualitative researchers as trying to fit a round peg into a square hole. The following four questions expose the likelihood of ethics in practice developing in the field and beyond the scrutiny of procedural ethics. The ethics committee could ask:
1. Tell us about your research,
2. Tell us about the ethical issues that arise from this research,
3. Tell us how you’re going to address these ethical issues.
Â
These three questions are standard in procedural ethics. The fourth question is not standard, yet based on the emergent epistemology above, this produces an iterative, ever-evolving research question. The question not asked in procedural ethics is:
Â
4. Tell us what you will do when the research question evolves, raising ethical issues not considered in procedural ethics. Rather than this development being aberrant, it is the norm.
3. Part 3: Ethical considerations for a pen and paper qualitative research project
Respect for persons (autonomy/diminished autonomy), beneficence, and non-maleficence are fundamental to all research ethics. How they manifest in qualitative research is unique, as demonstrated in the example above, contrasting anonymity and confidentiality; it suggested that anonymity was not something qualitative researchers could aspire to. If they want to anonymise their data, they cannot; they know the person and promise not to tell others. What they can do instead is de-identify the data. For qualitative researchers to promise anonymity is both unachievable and unethical.Â
Â
Any procedural ethics review of qualitative research should focus on how participants were recruited.  We create a hypothetical case to examine the ethics of this pen-and-paper exercise for purely contextual reasons. The topic researches bullying among high school students in multiple schools in a city of 200,000 people. Integral to this recruitment is the production of a participant information sheet and (a signed) consent form that demonstrates the following:
- The participants were informed about the nature of the project and what they were expected to do.
- The participant consented to taking part in the research. This is the norm, but in the hypothetical case of the high school students, they are all under 18 and considered to have diminished autonomy. These high school students can “assent” to the research only after their parent or guardian has given “consent” to approach them.
- Participation by the parent or high school student is voluntary. The researcher has used no coercion, including any form of inducement that would undermine the voluntary nature of the research.
Voluntary—informed—consent is comprised of three microprocessors in one. They are the hallmark of sound ethical considerations in qualitative research. However, ethics can modify this consent process. For example, if the research design were to be approved in procedural ethics, it could change if the researcher learns more about the complexity of the research question. In these circumstances, does the initial voluntary informed consent hold? Many qualitative researchers, Carolyn Ellis being the most prominent, understand the changeable nature of the consent process and employ “process consent”. Process consent is consent gained not only at the beginning of the research project but throughout the project as the research question evolves. That is best practice.
Beneficence is straightforward in procedural ethics. The qualitative researcher outlines the project’s aim and justifies recruiting persons into the study. A helpful guideline for beneficence is the statement, “Without benefit, there can be no risk”. That is, it is incumbent upon the researcher to establish benefits before recruiting anyone into their study. “This research will allow me to get a degree” does not justify beneficence. Beneficence is about how the research will benefit society and the participants, not themselves.
Non-maleficence requires the researcher to establish that risks have been identified and minimised. For example, in any study of bullying, it is incumbent on the researcher to plan for adverse events. When a participant is asked about their experience with bullying in their school, the high school student may reveal to the researcher not only their opinion or experience of bullying but also the emotional impact it had on them. An ethical researcher should have predicted what Guillimen and Gillam (2018) call “big ethical moments” like this and have a protocol to address this, i.e. refer the student to a counsellor.
4. Part 4: Are the methodological considerations in Computer Assisted Analysis of Qualitative Data the same as in the pen and paper exercise above?
The following three questions are methodological. How is the methodology different from a pen-and-paper study? It seems essential to decipher methodology first before considering ethical questions.
- Where does a literature search sit in webQDA? Van den Hoonaard (2002) suggests that a literature review is not essential in qualitative research. Here, he suggests that qualitative research is inductive. This is an important point. He suggests talking to people about bullying, establishing a research question, and then going to the library to conduct a literature search once the question is formulated. He also advises that you shouldn’t let the literature bias your study. So, this is perhaps an extreme version of induction. Where does a literature search sit in webQDA? Must it happen first?
- Coding. How are the first and second transcripts coded? Is this done on pen and paper or the computer? Some background. I often tell my students that the first and second interviews are usually burnt. This means the interviews are not expected to produce meaningful data. The first and second interviews aim to see if participants answer the questions if they understand them, and if the questions are sufficiently open-ended to allow the participant to be the expert and the researcher to be the learner. So, when and how are the first and second interviews coded?
- Interview guide: the hallmark of a quantitative questionnaire is that each respondent is asked to answer the same questions. The hallmark of the interview guide in qualitative research is that the research question is emergent, and the questions asked are iterative. For the first few interviews, different sets of interview questions are asked. This happens because the researcher learns new information and seeks to gather information from future participants on these new topics. So, the interview guide is constantly changing; it is iterative. Does this happen in webQDA, or are interview questions standardised? If there are 20 participants, do all 20 get asked the same questions?
In other words, before considering CAQDAS’s ethical considerations, we must understand how CAQDAS (such as webQDA) affects qualitative methodology.
5. Final Remarks
The transition from traditional pen-and-paper qualitative analysis to computer-assisted qualitative data analysis software (CAQDAS) tools like webQDA introduces several critical ethical considerations that researchers must navigate. While the fundamental principles of qualitative research ethics – respect for persons, beneficence, and non-maleficence – remain constant, the specific ways these principles are applied can shift when utilising technological solutions.
Â
One key challenge is maintaining qualitative research’s iterative, emergent nature within the more structured CAQDAS environment. For example, qualitative interview guides’ flexible, evolving nature may be more problematic to replicate when using software that encourages more standardised coding structures. Researchers must be vigilant to ensure that the affordances of the CAQDAS tool do not inadvertently constrain the organic development of research questions and data analysis. Careful consideration should be given to how literature searches, initial coding, and other methodological elements are integrated to preserve the inductive spirit of qualitative inquiry.
Â
Consent, confidentiality, and the researcher-participant relationship also require adaptation when moving to CAQDAS. The inability to truly anonymise qualitative data means that concepts of confidentiality must be redefined, with researchers focusing more on de-identification techniques. Furthermore, the “process consent” model advocated by scholars like Carolyn Ellis becomes even more crucial as analytical software amplifies the evolving nature of qualitative research. Researchers must proactively revisit consent throughout the study, particularly as new ethical considerations emerge.
Â
As qualitative researchers continue to embrace CAQDAS tools, the broader research community must engage in ongoing discussions and develop best practices. Potential areas for future exploration include the impact of CAQDAS on researcher reflexivity, the ethical implications of automated or semi-automated analysis functions, and the challenges of ensuring participant privacy and data security within digital research environments.
Â
Ultimately, adopting CAQDAS represents both an opportunity and a responsibility for qualitative researchers. While these tools can enhance efficiency and rigour, they must be accompanied by a steadfast commitment to preserving the foundational ethical principles underpinning qualitative inquiry. By proactively addressing the evolving ethical landscape, the qualitative research community can ensure that technological advancements strengthen, rather than compromise, the discipline’s core values and practices.
References
Brent, E. (1984). Qualitative computing: Approaches and issues. Qualitative Sociology, 7(1–2), 34–60. https://doi.org/10.1007/BF00987106
Â
Costa, A. P., & Amado, J. (2018). Content Analysis Supported by Software (1a). Oliveira de Azem̩is РAveiro РPORTUGAL: Ludomedia.
Â
Costa, A. P., Moreira, A., & Souza, F. N. de. (2019). webQDA – Qualitative Data Analysis. Aveiro – Portugal: Aveiro University and MicroIO. Retrieved from www.webqda.net
Â
Gibbs, G. R., Friese, S., & Mangabeira, W. C. (2002). The Use of New Technology in Qualitative Research. Introduction to Issue 3(2) of FQS. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research; Vol 3, No 2 (2002): Using Technology in the Qualitative Research Process, 3(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/847
Â
Gillam, L., & Guillemin, M. (2018). Reflexivity: Overcoming mistrust between research ethics committees and researchers. The Sage Handbook of Qualitative Research Ethics, London: Sage, pp263-275.
Â
Lage, M. C., & Godoy, A. S. (2008). O uso do computador na análise de dados qualitativos: questões emergentes. RAM. Revista de Administração Mackenzie, 9(4), 75–98. https://doi.org/10.1590/S1678-69712008000400006
Â
Minayo, M. C. de S., & Costa, A. P. (2019). Techniques that Use Speech, Observation and Empathy: Qualitative Research in Action (1st ed.). Oliveira de Azem̩is РAveiro РPORTUGAL: Ludomedia.
Â
Spannagel, C., Glaser-Zukuda, M., & Schroeder, U. (2005). Application of Qualitative Content Analysis in User-Program Interaction Research. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 6(2).
Â
Tolich, M. and Fitzgerald, M. H. (2006) ‘If Ethics Committees were Designed for Ethnography’, Journal of Empirical Research on Human Research Ethics, 1(2), pp. 71–78. doi: 10.1525/jer.2006.1.2.71.
Â
Van den Hoonaard, W. (ed.) (2002) Walking the Tightrope: Ethical issues for qualitative researchers, Toronto: University of Toronto Press.
Related Publications
