AI in interpretative qualitative research
The hybrid interpretation group is a new approach to integrating AI into interpretative qualitative research – Krähnke, Dresing, Pehl (in preparation). It enables an insight-evoking dialog between several AIs (Large Language Models / LLMs) and the researcher, with the aim of a profound and comprehensible interpretation of the text. This article describes the practical implementation of this idea, which can be implemented largely free of charge via a browser. (as at 18.09.2024)
Previously widespread approaches to the use of AI in qualitative social research, such as the use of ChatGPT, often only deliver superficial results (keyword “make a summary”). In addition, strategies of elaborate input to the LLM (prompt engineering) are pursued, which, however, require a high level of development expertise. Nevertheless, automatically generated interpretations by individual prompts are usually not particularly differentiated.
But there is a better way: With the hybrid interpretation group described here, we have found that we can carry out high-quality and differentiated analyses with AI. We ourselves are always surprised and delighted by the depth and sophistication of the interpretation suggestions. This awakens in us the desire to analyze and interpret again and again.
Here we describe the features of the approach and how anyone interested can try it out – including with free tools.
6 reasons that make this approach so special
The approach reduces the need for elaborate prompt design and instead enables a natural discussion language when dealing with AI. This makes the method more accessible and less technically demanding.
Three different LLMs (currently ChatGPT-4o, Claude 3.5 Sonnet and Gemini 1.5 pro) are integrated together in the research process. Studies show that the interlinking of different LLMs leads to an improvement in the quality of the output of each individual LLM. This significantly increases the overall quality of the analysis. This variance increases the diversity of perspectives due to the different “bias” of the LLMs involved. The confrontation with different points of view encourages the models to provide more differentiated answers and mutual reference, which leads to an improvement in quality.
The researcher takes on an orchestrating role as a moderator and remains actively involved in the interpretation process. She critically examines the interpretations offered, asks specific questions, gives instructions and reflects on the contributions of the AI models. This active control ensures that the analysis is targeted and in line with the research interest.
The analysis takes place in several rounds, with each round imitating the style of a lively group discussion. The AI models are confronted with the answers of the other LLMs and the assessments of the researcher. This process encourages the models to deepen their analyses, argue in a more differentiated way and justify their positions more strongly. This iterative approach results in an increasingly refined and multi-layered interpretation of the research material.
The complete documentation of the process addresses the often criticized “black box” problem of AI usage. This enables an intersubjective comprehensibility of the interpretation and makes the individual work performance of the researchers in the context of the orchestration of the work process clear. The researcher’s examination of the various interpretative approaches is documented transparently. It can be understood and assessed in terms of plausibility and quality, which increases the validity of the research.
You can throw students in at the deep end with this format. This allows them to gain their first practical experience with qualitative interpretation. The LLMs integrated into the interpretation group act as a kind of sparring partner for the interpretation work.
The hybrid interpretation group supports students and researchers in getting to know, applying and critically evaluating different perspectives of text interpretation. You do not simply receive a finished result or a pre-coded text.
By actively engaging with different approaches to interpretation, they are encouraged to develop their own, well-founded positions. This process not only promotes analytical thinking, but also the ability to synthesize and evaluate complex interpretations.
First, the user sets up (partly) free accounts with all three LLMs and selects a short, non-GDPR-relevant text excerpt. A simple, open starting question is formulated for the AI interpretation group to initiate the analysis process.
In the first round of analysis, the three LLMs examine the text from different perspectives. The researcher then reflects on the results and derives the next steps. This is followed by several iterative discussion rounds in which the LLMs and the researcher deepen and discuss the interpretations. This process is controlled by interim comments and reflections from the user until the user determines the conclusion of the analysis.
The entire discussion process is carefully documented, structured and commented on in an f4 project. The researcher identifies key aspects and develops their own interpretative perspective. Finally, a conclusion is drawn up that summarizes the interpretation developed. To ensure traceability, the entire process is exported as a Word, PDF or HTML file.
We have selected the following three LLMs. We have had by far the best experience with these in our test runs in terms of the quality of the interpretation suggestions. To use these, each person needs their own account, initially free of charge. You can simply click on the links and register.
The full use of all three LLMs can therefore be associated with costs of up to 60 euros per month. Fortunately, this is not necessary for a test run as part of a course or for moderate use.
To work completely free of charge, use Claude and ChatGPT in the limited but free versions and Gemini in the trial month. This means that you can only use Gemini for a maximum of 4 weeks and Claude only to a limited extent during this time.
Unfortunately, the free version of Claude limits the amount of text that can be edited and output within 5 hours. This is achieved with just a few analysis runs, as Claude not only counts the output, but also the input word quantity. You must organize yourself in such a way that you carry out one or two analysis runs within 5-hour time slots. Then cancel Google’s Gemini again in good time. ChatGPT also has an upper volume limit, but this is not reached so quickly.
Shorter analysis text sequences and shorter LLM responses enable more iterations. Optimization by restricting the response length (in the start prompt) significantly reduces the total amount of text. However, longer answers are often better argued. This needs to be weighed up.
The implementation of this working proposal is not yet GDPR-compliant, as all data is transferred to the respective providers outside the scope of the GDPR. Therefore, do not use any material that is critical under data protection law and contains personal data. Instead, use extracts from publicly available data, simulated data or data for which you have explicit written consent for this use.
How exactly does it work?
You can copy all the material generated in the following steps into this text to document the rest of the text and the entire process.
Write your individual overall conclusion on the interpretation of the text passage in the yellow comment field under the text “AI interpretation group”. Argue which and why you choose the favored perspective and which others you do not find plausible and therefore reject.
Example
You are currently viewing a placeholder content from Vimeo. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.
More InformationInterviewer: To begin with, exactly how long have you been involved with move2035 here in Marburg?
Participants: The first meeting took place in September. That was in ’23. And since then they have been very active.
Interviewer: And how would you describe your work?
Participants: My job is actually to coordinate this whole meeting. And the attempt to move it forward.
We often discussed this fictitious passage in our courses initially without any specific methodology or background knowledge. At first, people usually notice content such as coordination, meeting, time and place, as well as words such as “actually” and “attempt”, which are usually interpreted as a critical note. In most cases, there is then the view that “there is nothing more to be found in the text section”. While a content-analytical perspective might be satisfied with this, it is precisely the hermeneutic interpretation that opens up differentiated insights. And this succeeds with our hybrid interpretation group. In the following, we summarize the insights gained in each round. If you would like to find out more, the full course of the conversation can be found in the f4 project below.
The LLMs noticed that the participant used “they” instead of “we”, which was interpreted as distance from the group. The exact memory of the start time shows the significance of this moment. The self-description as a coordinator indicates a mediating role. The expression “trying to move it forward” implies challenges or resistance.
We wondered why the participant said “actually”. There may be a discrepancy between official role and actual activity.
The LLMs deepened the analysis and suggested that the participant vacillates between insider and outsider roles – a possible “professional dilemma”. The word “actually” could indicate additional, unnamed tasks. As coordinator, he may have limited decision-making power, and the “experiment” shows that progress does not depend on him alone.
We considered how this positioning affects the effectiveness of his role.
The LLMs further nuanced their interpretation. They saw distance as a strategy to maintain professional objectivity, allowing him to both understand and lead the group. His commitment despite challenges indicates intrinsic motivation. The awareness of the limits of his influence is reflected in the word “attempt”.
The participant moves in a complex role identity between professional distance and personal commitment to move2035. The use of “they” instead of “we” signals a deliberate distancing in order to maintain objectivity. “Actually” indicates a reflection on his role, perhaps there are more facets to his activity than officially known. The “attempt to move it forward” shows his commitment, his intrinsic motivation, but also his awareness of possible obstacles. The precise recollection of the starting point underlines the personal significance of the project for him, while his distanced language suggests a reflective attitude.
Initially, we only recognized superficial content: Coordination, meetings, timings and critical words. The hybrid interpretation group allowed us to dive deeper and understand the complex role identity of the participant. We discovered the tension between closeness and distance, the challenges of his position and his motivation. This iterative process enabled us to draw a multi-layered picture from a simple statement that would have remained hidden without this method.
Here you can download the current version (11.09.2024) of the f4 project file.
This contains the start prompts described above, which can simply be copied and pasted. Make sure to switch on the “Copy without citation” setting in f4, this makes copying easier. (see instructions above under “Preparation in f4”).
Finally, a blank text is prepared in which you insert the AI answers one by one, thus documenting the whole process. The prepared codes can be used to color-code the entries.
You will also find an example of an interpretation group in the project file. Here you can follow an entire discussion to analyze a fictitious interview passage.
Citation of this article
Dresing, T., Pehl, T., Krähnke, U. (2024). The hybrid interpretation group with dialogic-moderated LLMs: practical guidance for AI-supported qualitative research. audiotranskription.de. https://www.audiotranskription.de/hybride-interpretationsgruppe
This blog post is based on the article in progress: Krähnke, U., Dresing, T., Pehl, T. (in preparation). The hybrid interpretation group with dialogic-moderated LLMs. FQS.
further literature:
Lieder, F. R., & Schäffer, B. (2023). Teaching and learning reconstructive research methods with generative language models in hybrid research workshops? Journal of Psychology, 31(2), 131-154. https://doi.org/10.30820/0942-2285-2023-2-131.
(This article structures and explains specific prompts for qualitative research and mentions the idea of a “hybrid research workshop”, which we have expanded and concretized with our hybrid interpretation group approach).
We are curious about your experiences with our proposal. How do you manage the discussion process and how do you rate the quality of the answers and the added value of this approach? Any feedback helps us to improve and further develop this approach. We therefore kindly ask you to send us your comments by e-mail or telephone.
Newsletter
Never miss anything again
