AI in interpretative qualitative research
Hybrid interpretation is an elaborate method for integrating AI into interpretative qualitative research – Krähnke, Dresing, Pehl (in preparation). It enables an insight-evoking dialog between several AIs (Large Language Models / LLMs) and the researcher, with the aim of a profound and comprehensible interpretation of the text. This article describes the practical implementation of this procedure, which can be implemented largely free of charge via a browser.
This podcast was automatically generated from the written guide using Google Notebook LM to make the content accessible in a conversational format. The content is based on this description on the website and the PDF document (both by Dresing, Pehl & Krähnke (2024)). Entertaining and amazingly good, automatically generated presentation. But from about minute 12 it becomes nonsensical and has nothing to do with our proposal 🙂
Previously widespread approaches to the use of AI in qualitative social research, such as the use of ChatGPT, often only deliver superficial results (keyword “make a summary”). In addition, strategies of elaborate input to the LLM (prompt engineering) are pursued, which, however, require a high level of development expertise. Nevertheless, automatically generated interpretations by individual prompts are usually not particularly differentiated.
With the hybrid interpretation described here, we have found that we can carry out high-quality and differentiated analyses with AI. We ourselves are always surprised and delighted by the depth and sophistication of the interpretation suggestions. This awakens in us the desire to analyze and interpret again and again.
Here we describe the features of the approach and provide instructions on how anyone interested can try it out for themselves – including using tools that are available free of charge.
6 reasons that make this approach so special
The approach reduces the need for elaborate prompt design and instead enables a natural discussion language when dealing with AI. This makes the method more accessible and less technically demanding.
Three different LLMs (currently ChatGPT, Claude and Gemini) are integrated into the research process and moderated by the researcher. Studies show that the interlinking of different LLMs leads to an improvement in the quality of the output of each individual LLM. This significantly increases the overall quality of the analysis. This variance increases the diversity of perspectives due to the different “bias” of the LLMs involved. The confrontation with different points of view encourages more differentiated answers and mutual reference, which increases the diversity of perspectives.
The researcher takes on an orchestrating role as a moderator and remains actively involved in the interpretation process. She critically examines the interpretations offered, asks specific questions, gives instructions and reflects on the contributions of the AI models. This active control ensures that the analysis is targeted and in line with the research interest.
The analysis takes place in several rounds, with each round imitating the style of a lively group discussion. The AI models are confronted with the answers of the other LLMs and the assessments of the researcher. In the process, the analyses are deepened, argued in a more differentiated way and their interpretations are examined. This iterative approach results in an increasingly refined and multi-layered interpretation of the research material.
The complete documentation of the process addresses the often criticized “black box” problem of AI usage. This enables an intersubjective comprehensibility of the interpretation and makes the individual work performance of the researchers in the context of the orchestration of the work process clear. The researcher’s examination of the various interpretative approaches is documented transparently. It can be retraced in terms of plausibility and quality, which increases the validity of the research.
Hybrid interpretation allows students to gain their first practical experience of qualitative interpretation. The integrated LLMs act as a kind of sparring partner for the interpretation work.
Hybrid interpretation helps students to get to know, apply and critically evaluate different perspectives on text interpretation. You do not simply receive a finished result or automatic coding, but rather material for your own examination of the text.
By actively engaging with different approaches to interpretation, they are encouraged to develop their own, well-founded positions. This process not only promotes analytical thinking, but also the ability to synthesize and evaluate complex interpretations.
First of all, the user needs (partially) free accounts with all three LLMs and selects a short, non-GDPR-relevant text excerpt. A simple, open starting question is formulated for the AI interpretation group to initiate the analysis process.
In the first round of analysis, the three LLMs describe the text from different perspectives. The researcher then reflects on the statements and derives the next steps.
This is followed by several iterative discussion rounds in which the researcher deepens the interpretations and discusses them with the LLMs. This process is moderated and controlled by interim comments and reflections from the user until the user determines the conclusion of the analysis.
The researcher identifies key aspects and develops their own interpretative perspective. Finally, a conclusion is drawn up that summarizes the interpretation developed.
To ensure traceability, the entire course of the discussion is documented and commented on in Word or an f4 project.
During our test runs, we had by far the best experience with the following three LLMs in terms of the appealing quality of the interpretation suggestions. To use these, each person needs their own account, initially free of charge. You can simply click on the links and register. (as at Dec 2024)
The full use of all three LLMs can therefore be associated with costs of up to 60 euros per month. Fortunately, this is not necessary for a test run as part of a course or for moderate use.
To work completely free of charge, use Claude and ChatGPT in the limited but free versions and Gemini in the trial month. This means that you can only use Gemini for a maximum of 4 weeks and Claude only to a limited extent during this time.
Unfortunately, the free version of Claude limits the amount of text that can be edited and output within 5 hours. This is achieved with just a few analysis runs, as Claude not only counts the output, but also the input word quantity. You must organize yourself in such a way that you carry out one or two analysis runs within 5-hour time slots. Then cancel Google’s Gemini again in good time. ChatGPT also has an upper volume limit, but this is not reached so quickly.
Shorter analysis text sequences and shorter LLM responses enable more iterations. Optimization by restricting the response length (in the start prompt) significantly reduces the total amount of text. However, longer answers are often better argued. This needs to be weighed up.
The implementation of this working proposal is not yet GDPR-compliant, as all data is transferred to the respective providers outside the scope of the GDPR. Therefore, do not use any material that is critical under data protection law and contains personal data. Instead, use extracts from publicly available data, simulated data or data for which you have explicit written consent for this use.
How exactly does it work?
The necessary start prompt to the LLM consists of three components: Role assignment, work order and data material. These three elements are copied together to form a complete text. This complete text is the starting prompt with which the interpretation round can begin.
Write your individual overall conclusion on the interpretation of the passage. Argue which and why you choose the favored perspective and which others you do not find plausible and therefore reject.
Here you can download the current version (11.09.2024) of the start prompts described above, which can simply be copied and pasted.
You will also find an example of hybrid interpretation in the project file. Here you can follow an entire discussion to analyze a fictitious interview passage.
Citation of this article
Dresing, T., Pehl, T., Krähnke, U. (2024). The hybrid interpretation group with dialogic-moderated LLMs: practical guidance for AI-supported qualitative research. audiotranskription.de. https://www.audiotranskription.de/hybride-interpretationsgruppe
This blog post is based on the article in progress: Krähnke, U., Dresing, T., Pehl, T. (in preparation). Hybrid interpretation of qualitative data with dialogically integrated LLMs. On the use of generative AI in qualitative research
further literature:
Lieder, F. R., & Schäffer, B. (2023). Teaching and learning reconstructive research methods with generative language models in hybrid research workshops? Journal of Psychology, 31(2), 131-154. https://doi.org/10.30820/0942-2285-2023-2-131.
(This article structures and explains specific prompts for qualitative research and mentions the idea of a “hybrid research workshop”, which we have taken up and expanded with our approach of hybrid interpretation).
Example
You are currently viewing a placeholder content from Vimeo. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.
More InformationInterviewer: To begin with, exactly how long have you been involved with move2035 here in Marburg?
Participants: The first meeting took place in September. That was in ’23. And since then they have been very active.
Interviewer: And how would you describe your work?
Participants: My job is actually to coordinate this whole meeting. And the attempt to move it forward.
We often discussed this fictitious passage in our courses initially without any specific methodology or background knowledge. At first, people usually notice content such as coordination, meeting, time and place, as well as words such as “actually” and “attempt”, which are usually interpreted as a critical note. In most cases, there is then the view that “there is nothing more to be found in the text section”. While a content-analytical perspective might be satisfied with this, it is precisely the hermeneutic interpretation that opens up differentiated insights. And this succeeds with our hybrid interpretation group. In the following, we summarize the insights gained in each round. If you would like to follow it in more detail: the full course of the conversation can be found in the f4 project below.
The LLMs identify several significant linguistic patterns: The participant’s use of the personal pronoun “they” instead of “we” is interpreted as a possible indicator of a distanced attitude towards the group. The precise timing of the start of the project indicates a particular emotional or organizational significance of this moment. The self-characterization as a coordinator manifests a mediating function. The phrase “trying to move it forward” could indicate existing obstacles or resistance in the process.
These linguistic observations led us to critically reflect on the participant’s use of the modal particle “actually”. This could indicate a potential discrepancy between the formally assigned role and the actual function performed.
The LLMs identify a characteristic ambivalence in the positioning of the participant, which manifests itself in the oscillation between insider and outsider roles – a phenomenon that is classified as a “professional dilemma”. The use of the modal particle “actually” could indicate a latent spectrum of non-explicit tasks. The position of coordinator may imply limited decision-making authority. A specific LLM accentuates the significance of the lexeme “attempt”, which could indicate a shared or externalized process responsibility.
These observations lead to the analytical question of the interdependence between this complex (self-)positioning and the motivational structures of the actor.
The LLMs elaborate a differentiated interpretation of distancing as a potential strategy for maintaining professional objectivity. This perspective enables a methodological duality: it simultaneously facilitates the empathic understanding of group dynamics and the exercise of leadership functions. In this theoretical conceptualization, distance can be reinterpreted as a specific form of manifestation of engagement. The persistence of commitment evident in the material despite adverse conditions indicates intrinsic motivational structures. In this analytical framework, the use of the lexeme “attempt” can be interpreted as a reflection on the limitations of one’s own scope of action.
The participant navigates a complex structure of multiple role identities, characterized by the dialectic between professional detachment and personal involvement in the context of move2035. The preferred use of the personal pronoun “they” over “we” can be interpreted as an intentional distancing strategy to maintain professional objectivity. The modal particle “actually” indicates a pronounced process of reflection regarding one’s own role conception and possibly points to unexplained facets of the job configuration. The phrase “trying to make progress” simultaneously manifests intrinsic motivational structures and a pronounced awareness of potential restrictions in the field of action. The precise temporal localization of the project initiation suggests a considerable personal significance of the project, while the distanced use of language refers to an elaborate reflexive competence.
Initially, we only recognized superficial content: Coordination, meetings, timings and critical words. The hybrid interpretation group allowed us to dive deeper and understand the complex role identity of the participant. We discovered the tension between closeness and distance, the challenges of his position and his motivation. This iterative process enabled us to draw a multi-layered picture from a simple statement that would have remained hidden without this method.
We are curious about your experiences with our proposal. How do you manage the discussion process and how do you rate the quality of the answers and the added value of this approach? Any feedback helps us to improve and further develop this approach. We therefore kindly ask you to send us your comments by e-mail or telephone.
Newsletter
Never miss anything again
