The importance of Critical Thinking when using a CAQDAS
António Pedro Costa, University of Aveiro (Portugal)
Researcher at the Research Centre on Didactics and Technology in the Education of Trainers (CIDTFF), Department of Education and Psychology, University of Aveiro, and collaborator at the Laboratory of Artificial Intelligence and Computer Science (LIACC), Faculty of Engineering, University of Porto.
Computer-Assisted Qualitative Data Analysis Software (CAQDAS), such as webQDA or ATLAS.ti, are software packages that enable users to analyze their qualitative data (text, image, video, and audio) through features that allow them to organize, code, categorize, and question the data to answer the research questions. Currently, CAQDAS can include several characteristics such as:
- Automatic recognition and transcription mechanisms for data in audio, video, and image formats into text;
- Dynamic import of bulk data from various sources;
- Creation of parallel aggregation mechanisms for multimedia data (audio, video, and image) with their respective segments of automatic transcription;
- Mechanisms for bulk analysis code creation and automatic coding;
- Enhancement of coding comparison processes for multiple users within one or more coding systems;
- Methods for generating qualitative and/or quantitative visualization models (i.e., conceptual maps, matrix charts);
- Creation of tools to support decision-making in the process of comparing classification and coding by various users;
- Promoting interaction between researchers through the implementation of Collaboration and Cooperation, Coordination and Communication functionalities.
The decision-making process regarding which software to use should begin well before its actual exploration. Ideally, it would be important for the user to have prior knowledge of what they aim to achieve with their exploration before selecting a CAQDAS. Here, we focus on the digital or computational skills of the user. These skills, in conjunction with theoretical, epistemological, and methodological knowledge of qualitative studies, allow for the characterization of three behavioral approaches:
- Method behavior dominates software behavior: method behavior can dominate software behavior when CAQDAS is used to perform existing analytical techniques in new ways;
- Software behavior complements method behavior: software behavior can complement method behavior when CAQDAS facilitates the development of new research practices, such as query coding and data auto-coding;
- Software behavior dominates method behavior: software behavior can dominate method behavior by determining the methods adopted by researchers or influencing analytical outputs.
With the rapid advancement of Generative Artificial Intelligence tools, critical thinking emerges as a pillar to leverage the unlimited potential of CAQDAS without getting lost in its immense possibilities or being disoriented by the inherent complexities of these tools.
This articulation between software behavior and methodological application highlights the essentiality of critical thinking, or the ability to evaluate information and arguments reflectively and methodically, with the aim of forming well-founded inferences. With the rapid advancement of Generative Artificial Intelligence tools, critical thinking emerges as a pillar to leverage the unlimited potential of CAQDAS without getting lost in its immense possibilities or being disoriented by the inherent complexities of these tools. In this context, we share some examples where this competence is pervasive throughout the qualitative data analysis process (see Figure 1):
Figure 1: Critical Thinking in Qualitative Data Analysis
- Data Selection/Organization: How to choose the most relevant data for analysis? Careful data selection strengthens the analysis, avoiding the temptation to favour what appears to have immediate visibility to the detriment of potentially more significant information.
- Coding and Categorization: The importance of coding consistently and avoiding over-interpretation. Vital for an impartial interpretation, it is necessary to maintain consistency in assigning codes, avoiding the tendency to read too much into data that can offer multiple interpretations.
- Pattern Identification: How to recognize genuine patterns without forcing interpretations.
- Interpretation of Results: Analyze the results with a critical eye, considering different perspectives.
Challenges and pitfalls of qualitative data analysis:
- Personal Biases: Recognizing and mitigating the influence of our own beliefs and experiences.
- Selective Interpretation: Avoid interpreting data selectively to confirm biases.
- Software Limitations: Understanding the limitations of CAQDAS and not blindly relying on the results.
In 2023, the European Commission outlined a set of competencies (38 skills divided into 7 dimensions) for researchers (European Competence Framework for Researcher). Even without specific data to consolidate this point, I conjecture that researchers, regardless of their career stage, may not be able to develop most of the competencies presented. Thus, the formation of multidisciplinary teams or, when not possible, the use of experts in a particular area, favors higher-quality analyses. The exchange of ideas and specific competencies broadens analytical perspectives and reduces the incidence of unidimensional analytical overlaps.
In the case of CAQDAS, features such as coding comparison enable an enriching dialogue between users, while support tools strengthen the integrity of the decision-making process. Sustainable interaction between researchers establishes fertile ground for innovation and cross-validation, framing the qualitative analysis process in a robust and integrated set of research strategies.
Establishing clear boundaries between technology and methodology, actively reflecting on decisions throughout the analysis, and choosing a CAQDAS not only for its technological capabilities but also for its suitability within the context and scope of the research project, serves to scientifically mark the use of CAQDAS. Therefore, critical thinking consolidates as the indispensable foundation when choosing and handling such tools, ensuring the necessary discernment to use technology with purpose and direction while maintaining the primacy of scientific integrity in qualitative research techniques and results.
Bibliography
Almeida, C. A., Freitas, F., Costa, A. P., & Moreira, A. (2019). WEBQDA: The Quest for a Place in the Competitive World of CAQDAS. International Conference in Engineering Applications (ICEA), 1–7. https://doi.org/10.1109/CEAP.2019.8883456
Carter, S. M., & Little, M. (2007). Justifying Knowledge, Justifying Method, Taking Action: Epistemologies, Methodologies, and Methods in Qualitative Research. Qualitative Health Research, 17(10), 1316–1328. https://doi.org/10.1177/1049732307306927
Chen, S.-Y. (2023). Generative AI, learning and new literacies. Journal of Educational Technology Development and Exchange, 16(2), 1 – 19. https://doi.org/10.18785/jetde.1602.01
Costa, A. P. (2016). Cloud Computing em Investigação Qualitativa: Investigação Colaborativa através do software webQDA. Fronteiras: Journal of Social, Technological and Environmental Science, 5(2), 153–161. https://doi.org/10.21664/2238-8869.2016v5i2.p153-161
Freitas, F., Costa, A. P., & Moreira, A. (2022). In the Wonderful World of Computer-Assisted Qualitative Data Analysis Software (CAQDAS). In A. Rosario & S. Wa-Mbaleka (Eds.), The SAGE Handbook of Qualitative Research in the Asian Context (pp. 477–491). Sage Publications.
Freitas, F., Ribeiro, J., Brandão, C., de Almeida, C. A., de Souza, F. N., & Costa, A. P. (2019). How do we like to learn qualitative data analysis software? The Qualitative Report, 24(13).
García‐Horta, J. B., & Guerra‐Ramos, M. T. (2009). The use of CAQDAS in educational research: some advantages, limitations and potential risks. International Journal of Research & Method in Education, 32(2), 151–165. https://doi.org/10.1080/17437270902946686
Vieira, R. M., & Tenreiro-Vieira, C. (2021). Pensamento Crítico e Criativo na Educação em Ciências: percursos de investigação e proposta de referencial. In N. Kiouranis, R. M. Vieira, C. Tenreiro-Vieira, & V. Calixto, (Orgs.), Pensamento crítico na educação em ciências (pp. 17-42). Editora Livraria da Física.
Woods, M., Macklin, R., & Lewis, G. K. (2016). Researcher reflexivity: exploring the impacts of CAQDAS use. International Journal of Social Research Methodology, 19(4), 385–403. https://doi.org/10.1080/13645579.2015.1023964
