qualitative research about health

• Technical Support
• Find My Rep

You are here

Qualitative Health Research

Preview this book

• Description
• Aims and Scope
• Editorial Board
• Abstracting / Indexing
• Submission Guidelines

Qualitative Health Research provides an international, interdisciplinary forum to enhance health and health care and further the development and understanding of qualitative health research. The journal is an invaluable resource for researchers and academics, administrators and others in the health and social service professions, and graduates, who seek examples of studies in which the authors used qualitative methodologies. Each issue of Qualitative Health Research provides readers with a wealth of information on conceptual, theoretical, methodological, and ethical issues pertaining to qualitative inquiry. A Variety of Perspectives We encourage submissions across all health-related areas and disciplines. Qualitative Health Research understands health in its broadest sense and values contributions from various traditions of qualitative inquiry. As a journal of SAGE Publishing, Qualitative Health Research aspires to disseminate high-quality research and engaged scholarship globally, and we are committed to diversity and inclusion in publishing. We encourage submissions from a diverse range of authors from across all countries and backgrounds. There are no fees payable to submit or publish in Qualitative Health Research .

Original, Timely, and Insightful Scholarship Qualitative Health Research aspires to publish articles addressing significant and contemporary health-related issues. Only manuscripts of sufficient originality and quality that align with the aims and scope of Qualitative Health Research will be reviewed. As part of the submission process authors are required to warrant that they are submitting original work, that they have the rights in the work, that they have obtained, and that can supply all necessary permissions for the reproduction of any copyright works not owned by them, and that they are submitting the work for first publication in the Journal and that it is not being considered for publication elsewhere and has not already been published elsewhere. Please note that Qualitative Health Research does not accept submissions of papers that have been published elsewhere. Sage requires authors to identify preprints upon submission (see https://us.sagepub.com/en-us/nam/preprintsfaq ). This Journal is a member of the Committee on Publication Ethics (COPE) .

This Journal recommends that authors follow the Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals formulated by the International Committee of Medical Journal Editors (ICMJE).

Qualitative Health Research is an international, interdisciplinary, refereed journal for the enhancement of health care and to further the development and understanding of qualitative research methods in health care settings. We welcome manuscripts in the following areas: the description and analysis of the illness experience, health and health-seeking behaviors, the experiences of caregivers, the sociocultural organization of health care, health care policy, and related topics. We also seek critical reviews and commentaries addressing conceptual, theoretical, methodological, and ethical issues pertaining to qualitative enquiry.

• Applied Social Sciences Index & Abstracts (ASSIA)
• CAB Abstracts (Index Veterinarius, Veterinary Bulletin)
• CABI: Abstracts on Hygiene and Communicable Diseases
• CABI: CAB Abstracts
• CABI: Global Health
• CABI: Nutrition Abstracts and Reviews Series A
• CABI: Tropical Diseases Bulletin
• Clarivate Analytics: Current Contents - Physical, Chemical & Earth Sciences
• Combined Health Information Database (CHID)
• Corporate ResourceNET - Ebsco
• Current Citations Express
• EBSCO: Vocational & Career Collection
• EMBASE/Excerpta Medica
• Family & Society Studies Worldwide (NISC)
• Health Business FullTEXT
• Health Service Abstracts
• Health Source Plus
• MasterFILE - Ebsco
• ProQuest: Applied Social Science Index & Abstracts (ASSIA)
• ProQuest: CSA Sociological Abstracts
• Psychological Abstracts
• Rural Development Abstracts
• SRM Database of Social Research Methodology
• Social Sciences Citation Index (Web of Science)
• Social Services Abstracts
• Standard Periodical Directory (SPD)
• TOPICsearch - Ebsco

Manuscript submission guidelines:

Qualitative Health Research (QHR)  has specific guidelines! While  Sage Publishing has general guidelines , all manuscripts submitted to QHR must follow our specific guidelines (found below). Once you have reviewed these guidelines, please visit QHR ’s  submission site  to upload your manuscript.   Please note that manuscripts not conforming to these guidelines will be returned and/or encounter delays in peer review.   Remember you can log in to the submission site at any time to check on the progress of your manuscript throughout the peer review process.

1. Deciding whether to submit a manuscript to QHR

1.1 Aims & scope

1.2 Article types

2. Review criteria

2.1  Original research studies

2.2 Pearls, Piths, and Provocations

2.3 Common reasons for rejection

3.  Preparing your manuscript

3.1 Title page

3.2 Abstract

3.3 Manuscript

3.4 Tables, Figures, Artwork, and other graphics

3.5 Supplemental material

4. Submitting your manuscript

5. Editorial Policies

5.1 Peer review policy

5.2 Authorship

5.3 Acknowledgments

5.4 Funding

5.5 Declaration of conflicting interests

5.6 Research ethics and participant consent

6. Publishing Policies

6.1 Publication ethics

6.2 Contribtor's publishing agreement

6.3 Open access and author archiving

1. Deciding whether to submit a manuscript to QHR 

QHR  provides an international, interdisciplinary forum to enhance health and health care and further the development and understanding of qualitative health research. The journal is an invaluable resource for researchers and academics, administrators and others in the health and social service professions, and graduates, who seek examples of studies in which the authors used qualitative methodologies.  Each issue of QHR provides readers with a wealth of information on conceptual, theoretical, methodological, and ethical issues pertaining to qualitative inquiry.

Rather than send query letters to the Editor regarding article fit, QHR asks authors to make their own decision regarding the suitability of their manuscript for QHR   by  asking: Does your proposed submission make a meaningful and strong contribution to qualitative health research literature? Is it useful to readers and/or practitioners?

The following manuscript types are considered for publication.

• Original Research Studies : These are fully developed qualitative research studies. This may include mixed method studies in which the major focus/portion of the study is qualitative research. Please read  Maintaining the Integrity of Qualitatively Driven Mixed Methods: Avoiding the “ This Work is Part of a Larger Study”  Syndrome . 
• Pearls, Piths, and Provocations : These manuscripts should foster discussion and debate about significant issues, enhance communication of methodological advances, promote and discuss issues related to the teaching of qualitative approaches in health contexts, and/or encourage the discussion of new and/or provocative ideas. They should also make clear what the manuscript adds to the existing body of knowledge in the area.
• Editorials : These are generally invited articles written by editors/editorial board members associated with QHR.

Please note, QHR does NOT publish pilot studies. We do not normally publish literature reviews unless they focus on qualitative research studies elaborating methodological issues and developments. Review articles should be submitted to the Pearls, Piths, and Provocations section. They are reviewed according to criteria in 2.2.

Back to top

2. Review criteria 

2.1 Original research

Reviewers are asked to consider the following areas and questions when making recommendations about research manuscripts:

• Importance of submission : Does the manuscript make a significant contribution to qualitative health research literature?  Is it original? Relevant? In depth? Insightful? Is it useful to the reader and/or practitioner?
• Methodological considerations : Is the overall study design clearly explained including why this design was an appropriate one? Are the methodology/methods/approaches used in keeping with that design? Are they appropriate given the research question and/or aims? Are they logically articulated? Clarity in design and presentation? Data adequacy and appropriateness? Evidence of rigor?
• Ethical Concerns : Are relevant ethical concerns discussed and acknowledged? Is enough detail given to enable the reader to understand how ethical issues were navigated? Has formal IRB approval (when needed) and consent from participants been obtained?
• Data analysis, findings, discussion : Does the analysis of data reflect depth and coherence? In-depth descriptive but also interpretive dimensions? Creative and insightful analysis? Are results linked to existing literature and theory, as appropriate? Is the contribution of the research clear including its relevance to health disciplines and their practice?
• Manuscript style and format : Is the manuscript organized in a clear and concise manner? Has sufficient attention been paid to word choice, spelling, grammar, and so forth? Did the author adhere to APA guidelines? Do diagrams/illustrations comply with guidelines? Is the overall manuscript aligned with QHR guidelines in relation to formatting?
• Scope:  Does the article fit with QHR ’s publication mandate? Has the author cited the major work in the area, including those published in QHR ? 

The purpose of papers in this section is to raise and discuss issues pertinent to the development and advancement of qualitative research in health-related arenas.  As the name Pearls, Piths, and Provocations suggests, we are looking for manuscripts that make a significant contribution to areas of dialogue, development, experience sharing and debate relevant to the scope of QHR in this section of the journal.  Reviewers are asked to consider the following questions when making recommendations about  articles in the Pearls, Piths, and Provocations section.

• Significance :   Does the paper highlight issues that have the potential to advance, develop, and/or challenge thinking in qualitative health related research?
• Clarity :   Are the arguments clearly presented and well supported? 
• Rigor :   Is there the explicit use of/interaction with methodology and/or theory and/or empirical studies (depending on the focus of the paper) that grounds the work and is coherently carried throughout the arguments and/or analysis in the manuscript? Put another way, is there evidence of a rigorously constructed argument?
• Engagement :   Does the paper have the potential to engage the reader to ‘think differently’ by raising questions, suggesting innovative directions for qualitative health research, and/or stimulating critical reflection?   Are the implications of the paper for the practice of either qualitative research and/or health clear? 
• Quality of the writing :   Is the main argument of the paper clearly articulated and presented with few grammatical or typographical issues? Are terms and concepts key to the scholarship communicated clearly and in sufficient detail? 

QHR  most commonly turns away manuscripts that fall outside the journal’s scope, do not make a novel contribution to the literature, lack substantive and/or interpretative depth, require extensive revisions, and/or do not adequately address ethical issues that are fundamental to qualitative inquiry. Submissions of the supplementary component of mixed methods studies often are rejected as the findings are difficult to interpret without the findings of the primary study. For additional information on this policy, please read  Maintaining the Integrity of Qualitatively Driven Mixed Methods: Avoiding the “ This Work is Part of a Larger Study”  Syndrome . 

3. Preparing your manuscript for submission 

We strongly encourage all authors to review previously published articles in QHR for style prior to submission.

QHR  journal practices include double anonymization. All identifying information MUST be removed completely from the Abstract, Manuscript, Acknowledgements, Tables, and Figure files prior to submission. ONLY the Title Page and Cover Letter may contain identifying information. See  Sage’s general submission guidelines  for additional guidance on making an anonymous submission.

Preferred formats for the text and tables of your manuscript are Word DOC or PDF. The text must be double-spaced throughout with standard 1-inch margins (APA formatting). Text should be standard font (i.e., Times New Roman) 12-point. 

3.1 Title page 

• The title page should be uploaded as a separate document containing the following information: Author names; Affiliations; Author contact information; Contribution list; Acknowledgements; Ethical statement; Funding Statement; Conflict of Interest Statements; and, Grant Number. Please know that the Title Page is NOT included in the materials sent out for Peer Review.
• Ethical statement: An ethical statement must include the following: the full name of the ethical board that approved your study; the approval number given by the ethical board; and, confirmation that all your participants gave informed consent. Authors are also required to state in the methods section whether participants provided informed consent, whether the consent was written or verbal, and how it was obtained and by whom. For example: “Our study was approved by The Mercy Health Research Ethics Committee (approval no. XYZ123). All participants provided written informed consent prior to enrollment in the study.” If your study did not need ethical approval (often manuscripts in the Pearls, Piths, and Provocations may not), we still need a statement that states that your study did not need approval and an explanation as to why. For example: “Ethical Statement: Our study did not require an ethical board approval because it did not directly involve humans or animals.” 

3.2 Abstract and Keywords

• The Abstract should be unstructured, written in narrative form. Maximum of 250 words. This should be on its own page, appearing as the first page of the Main Manuscript file.
• The keywords should be included beneath the abstract on the Main Manuscript file. 
• Length: 8,000 words or less excluding the abstract, list of references, and acknowledgements. This applies to both Original Research and Pearls, Piths, and Provocations. Please note that text from Tables and Figures is included in the word count limits. On-line supplementary materials are not included in the word limit. 
• Structure: While many authors will choose to use headings of Background, Methods, Results, and Discussion to organize their manuscript, it is up to authors to choose the most appropriate terms and structure for their submission. It is the expectation that manuscripts contain detailed reflections on methodological considerations.
• Ethics: In studies where data collection or other methods present ethical challenges, the authors should explicate how such issues were navigated including how consent was gained and by whom. An anonymized version of the ethical statement should be included in the manuscript (in addition to appearing on the title page).
• Participant identification: Generally, demographics should be described in narrative form or otherwise reported as a group. Quotations may be linked to particular participants and/or demographic features provided measures are taken to ensure anonymity of participants (e.g., use of pseudonyms).
• Use of checklists: Authors should not include qualitative research checklists, such as COREQ (COnsolidated criteria for REporting Qualitative research).  Generally, authors should use a narrative approach to describe the processes used to enhance the rigor of their study. For additional information on this policy, please read  Why the Qualitative Health Research (QHR) Review Process Does Not Use Checklists
• References: APA format. While there is no limit to the number of references, authors are recommended to use pertinent references only, including literature previously published in QHR . References should be on a separate page.   QHR adheres to the APA 7 reference style. View the APA guidelines to ensure your manuscript conforms to this reference style. Please ensure you check carefully that both your in-text references and list of references are in the correct format.
• Authors are required to disclose the use of generative Artificial Intelligence (such as ChatGPT) and other technologies (such as NVivo, ATLAS. Ti, Quirkos, etc.), whether used to conceive ideas, develop study design, generate data, assist in analysis, present study findings, or other activities formative of qualitative research. We suggest authors provide both a description of the technology, when it was accessed, and how it was used (see  https://uk.sagepub.com/en-gb/eur/chatgpt-and-generative-ai ).
• Manuscripts that receive favorable reviews will not be accepted until any formatting and copy-editing required has been done. 
• Tables, Figures, Artwork, and other graphics should be submitted as separate files rather than incorporated into the main manuscript file. Within the manuscript, indicate where these items should appear (i.e. INSERT TABLE 1 HERE).
• TIFF, JPED, or common picture formats accepted. The preferred format for graphs and line art is EPS.
• Resolution: Rasterized based files (i.e. with .tiff or .jpeg extension) require a resolution of at least 300 dpi (dots per inch). Line art should be supplied with a minimum resolution of 800 dpi.
• Dimension: Check that the artworks supplied match or exceed the dimensions of the journal. Images cannot be scaled up after origination.
• Figures supplied in color will appear in color online regardless of whether or not these illustrations are reproduced in color in the printed version. For specifically requested color reproduction in print, you will receive information regarding the costs from Sage after receipt of your accepted article. 
• Core elements of the manuscript should not be included as supplementary material.
• QHR  is able to host additional materials online (e.g., datasets, podcasts, videos, images etc.) alongside the full-text of the article. For more information please refer to Sage’s general  guidelines on submitting supplemental files .

4. Submitting your manuscript 

QHR  is hosted on Sage Track, a web based online submission and peer review system powered by ScholarOne™ Manuscripts. Visit  https://mc.manuscriptcentral.com/QHR  to login and submit your article online. 

IMPORTANT:  Please check whether you already have an account in the system before trying to create a new one. If you have reviewed or authored for the Journal in the past year it is likely that you will have had an account created.  For further guidance on submitting your manuscript online please visit  ScholarOne Online Help . 

5. Editorial policies 

QHR  adheres to a rigorous double-anonymized reviewing policy in which the identities of both the reviewer and author are always concealed from both parties.

Sage does not permit the use of author-suggested (recommended) reviewers at any stage of the submission process, be that through the web-based submission system or other communication. Reviewers should be experts in their fields and should be able to provide an objective assessment of the manuscript. Our policy is that reviewers should not be assigned to a manuscript if:

•  The reviewer is based at the same institution as any of the co-authors

•  The reviewer is based at the funding body of the manuscript

•  The author has recommended the reviewer

•  The reviewer has provided a personal (e.g. Gmail/Yahoo/Hotmail) email account and an institutional email account cannot be found after performing a basic Google search (name, department and institution). 

Qualitative Health Research  is committed to delivering high quality, fast peer-review for your manuscript, and as such has partnered with Web of Science. Web of Science is a third-party service that seeks to track, verify and give credit for peer review. Reviewers for Qualitative Health Research can opt in to Web of Science in order to claim their reviews or have them automatically verified and added to their reviewer profile. Reviewers claiming credit for their review will be associated with the relevant journal, but the article name, reviewer’s decision, and the content of their review is not published on the site. For more information visit the  Web of Science  website.

The Editor or members of the Editorial Team or Board may occasionally submit their own manuscripts for possible publication in the Journal. In these cases, the peer review process will be managed by alternative members of the Editorial Team or Board and the submitting Editor Team/Board member will have no involvement in the decision-making process. 

Manuscripts should only be submitted for consideration once consent is given by all contributing authors. Those submitting manuscripts should carefully check that all those whose work contributed to the manuscript are acknowledged as contributing authors. The list of authors should include all those who can legitimately claim authorship. This is all those who meet all of the following criteria:

(i)   Made a substantial contribution to the design of the work or acquisition, analysis, interpretation, or presentation of data,  (ii)  Drafted the article or revised it critically for important intellectual content,  (iii) Approved the version to be published,  (iv) Each author should have participated sufficiently in the work to take public responsibility for appropriate portions of the content.

Acquisition of funding, collection of data, or general supervision of the research group alone does not constitute authorship, although all contributors who do not meet the criteria for authorship should be listed in the Acknowledgments section. Please refer to the  International Committee of Medical Journal Editors (ICMJE) authorship guidelines   for more information on authorship.

Authors are required to disclose the use of generative Artificial Intelligence (such as ChatGPT) and other technologies (such as NVivo, ATLAS. Ti, Quirkos, etc.), whether used to conceive ideas, develop study design, generate data, assist in analysis, present study findings, or other activities formative of qualitative research. We suggest authors provide both a description of the technology, when it was accessed, and how it was used. This needs to be clearly identified within the text and acknowledged within your Acknowledgements section. Please note that AI bots such as ChatGPT should not be listed as an author. For more details on this policy, please visit  ChatGPT and Generative AI . 

5.3 Acknowledgements

All contributors who do not meet the criteria for authorship should be listed in an Acknowledgements section. Examples of those who might be acknowledged include a person who provided purely technical help, or a department chair who provided only general support.

Please supply any personal acknowledgements separately to the main text to facilitate anonymous peer review. 

Per  ICMJE recommendations , it is best practice to obtain consent from non-author contributors who you are acknowledging in your manuscript.

1.3.1 Writing assistance

Individuals who provided writing assistance, e.g., from a specialist communications company, do not qualify as authors and so should be included in the Acknowledgements section. Authors must disclose any writing assistance – including the individual’s name, company and level of input – and identify the entity that paid for this assistance. It is not necessary to disclose use of language polishing services. 

Qualitative Health Research   requires all authors to acknowledge their funding in a consistent fashion under a separate heading.  Please visit the  Funding Acknowledgements   page on the Sage Journal Author Gateway to confirm the format of the acknowledgment text in the event of funding, or state that: This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. 

It is the policy of  Qualitative Health Research to require a declaration of conflicting interests from all authors enabling a statement to be carried within the paginated pages of all published articles.

Please ensure that a ‘Declaration of Conflicting Interests’ statement is included at the end of your manuscript, after any acknowledgements and prior to the references. If no conflict exists, please state that ‘The Author(s) declare(s) that there is no conflict of interest’. For guidance on conflict of interest statements, please see the ICMJE recommendations  here . 

Research involving participants must be conducted according to the  World Medical Association Declaration of Helsinki

Submitted manuscripts should conform to the  ICMJE Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals :

All manuscripts must state that the relevant Ethics Committee or Institutional Review Board provided (or waived) approval. Please ensure that you blind the name and institution of the review committee until such time as your article has been accepted. The Editor will request authors to replace the name and add the approval number once the article review has been completed. Please note that in itself, simply stating that Ethics Committee or Institutional Review was obtained is not sufficient.  Authors are also required to state in the methods section whether participants provided informed consent, whether the consent was written or verbal, and how it was obtained and by whom.

Please do not submit the participant’s informed consent documents with your article, as this in itself breaches the participant’s confidentiality. The Journal requests that you confirm to us, in writing, that you have obtained informed consent recognizing the documentation of consent itself should be held by the authors/investigators themselves (for example, in a participant’s hospital record or an author’s institution’s archives).

Please also refer to the  ICMJE Recommendations for the Protection of Research Participants . 

6. Publishing Policies 

Sage is committed to upholding the integrity of the academic record. We encourage authors to refer to the Committee on Publication Ethics’  International Standards for Authors  and view the Publication Ethics page on the  Sage Author Gateway .

6.1.1 Plagiarism

Qualitative Health Research  and Sage take issues of copyright infringement, plagiarism or other breaches of best practice in publication very seriously. The Committee on Publication Ethics (COPE) defines plagiarism as: “When somebody presents the work of others (data, words or theories) as if they were his/her own and without proper acknowledgment.” We seek to protect the rights of our authors and we always investigate claims of plagiarism or misuse of published articles. Equally, we seek to protect the reputation of the journal against malpractice. Submitted articles may be checked with duplication-checking software. Where an article, for example, is found to have plagiarised other work or included third-party copyright material without permission or with insufficient acknowledgement, or where the authorship of the article is contested, we reserve the right to take action including, but not limited to: publishing an erratum or corrigendum (correction); retracting the article; taking up the matter with the head of department or dean of the author's institution and/or relevant academic bodies or societies; or taking appropriate legal action.

6.1.2 Prior publication

If material has been previously published it is not generally acceptable for publication in a Sage journal. However, there are certain circumstances where previously published material can be considered for publication. Please refer to the guidance on the  Sage Author Gateway  or if in doubt, contact the Editor at the address given below. 

6.2 Contributor's publishing agreement

Before publication, Sage requires the author as the rights holder to sign a Journal Contributor’s Publishing Agreement. Sage’s Journal Contributor’s Publishing Agreement is an exclusive licence agreement which means that the author retains copyright of the work but grants Sage the sole and exclusive right and licence to publish for the full legal term of copyright. Exceptions may exist where an assignment of copyright is required or preferred by a proprietor other than Sage. In this case copyright in the work will be assigned from the author to the society. For more information please visit the  Sage Author Gateway . 

Qualitative Health Research  offers optional open access publishing via the Sage Choice programme and Open Access agreements, where authors can publish open access either discounted or free of charge depending on the agreement with Sage. Find out if your institution is participating by visiting Open Access Agreements at Sage . For more information on Open Access publishing options at Sage please visit Sage Open Access . For information on funding body compliance, and depositing your article in repositories, please visit Sage’s Author Archiving and Re-Use Guidelines and Publishing Policies .

• Read Online
• Sample Issues
• Current Issue
• Email Alert
• Permissions
• Foreign rights
• Reprints and sponsorship
• Advertising

Individual Subscription, E-access

Individual Subscription, Print Only

Institutional Backfile Purchase, E-access (Content through 1998)

Institutional Subscription, E-access

Institutional Subscription & Backfile Lease, E-access Plus Backfile (All Online Content)

Institutional Subscription, Print Only

Institutional Subscription, Combined (Print & E-access)

Institutional Subscription & Backfile Lease, Combined Plus Backfile (Current Volume Print & All Online Content)

Individual, Single Print Issue

Institutional, Single Print Issue

To order single issues of this journal, please contact SAGE Customer Services at 1-800-818-7243 / 1-805-583-9774 with details of the volume and issue you would like to purchase.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Qualitative research in healthcare: an introduction to grounded theory using thematic analysis

Affiliation.

• 1 AL Chapman, Department of Infectious Diseases, Monklands Hospital, Airdrie ML6 0JS, UK. Email [email protected].
• PMID: 26517098
• DOI: 10.4997/JRCPE.2015.305

In today's NHS, qualitative research is increasingly important as a method of assessing and improving quality of care. Grounded theory has developed as an analytical approach to qualitative data over the last 40 years. It is primarily an inductive process whereby theoretical insights are generated from data, in contrast to deductive research where theoretical hypotheses are tested via data collection. Grounded theory has been one of the main contributors to the acceptance of qualitative methods in a wide range of applied social sciences. The influence of grounded theory as an approach is, in part, based on its provision of an explicit framework for analysis and theory generation. Furthermore the stress upon grounding research in the reality of participants has also given it credence in healthcare research. As with all analytical approaches, grounded theory has drawbacks and limitations. It is important to have an understanding of these in order to assess the applicability of this approach to healthcare research. In this review we outline the principles of grounded theory, and focus on thematic analysis as the analytical approach used most frequently in grounded theory studies, with the aim of providing clinicians with the skills to critically review studies using this methodology.

Keywords: grounded theory; healthcare; inductive analysis; qualitative research; quality improvement; thematic analysis.

PubMed Disclaimer

Similar articles

• Grounded theory as a method for research in speech and language therapy. Skeat J, Perry A. Skeat J, et al. Int J Lang Commun Disord. 2008 Mar-Apr;43(2):95-109. doi: 10.1080/13682820701437245. Int J Lang Commun Disord. 2008. PMID: 17852520 Review.
• Participants' views of telephone interviews within a grounded theory study. Ward K, Gott M, Hoare K. Ward K, et al. J Adv Nurs. 2015 Dec;71(12):2775-85. doi: 10.1111/jan.12748. Epub 2015 Aug 10. J Adv Nurs. 2015. PMID: 26256835
• Applications of Grounded Theory Methodology to Investigate Hearing Loss: A Methodological Qualitative Systematic Review With Developed Guidelines. Ali Y, Wright N, Charnock D, Henshaw H, Morris H, Hoare DJ. Ali Y, et al. Ear Hear. 2024 May-Jun 01;45(3):550-562. doi: 10.1097/AUD.0000000000001459. Epub 2024 Apr 14. Ear Hear. 2024. PMID: 38608196 Free PMC article.
• The grounded theory method and maternal-infant research and practice. Marcellus L. Marcellus L. J Obstet Gynecol Neonatal Nurs. 2005 May-Jun;34(3):349-57. doi: 10.1177/0884217505276053. J Obstet Gynecol Neonatal Nurs. 2005. PMID: 15890834 Review.
• Is it really theoretical? A review of sampling in grounded theory studies in nursing journals. McCrae N, Purssell E. McCrae N, et al. J Adv Nurs. 2016 Oct;72(10):2284-93. doi: 10.1111/jan.12986. Epub 2016 Apr 26. J Adv Nurs. 2016. PMID: 27113800 Review.
• Nurse rostering: understanding the current shift work scheduling processes, benefits, limitations, and potential fatigue risks. Booker LA, Mills J, Bish M, Spong J, Deacon-Crouch M, Skinner TC. Booker LA, et al. BMC Nurs. 2024 Apr 29;23(1):295. doi: 10.1186/s12912-024-01949-2. BMC Nurs. 2024. PMID: 38685019 Free PMC article.
• Role of family medicine physicians in providing nutrition support to older patients admitted to orthopedics departments: a grounded theory approach. Ohta R, Nitta T, Shimizu A, Sano C. Ohta R, et al. BMC Prim Care. 2024 Apr 19;25(1):121. doi: 10.1186/s12875-024-02379-4. BMC Prim Care. 2024. PMID: 38641569 Free PMC article.
• Implementation of a critical care outreach team in a children's hospital. Mehta S, Galligan MM, Lopez KT, Chambers C, Kabat D, Papili K, Stinson H, Sutton RM. Mehta S, et al. Resusc Plus. 2024 Apr 11;18:100626. doi: 10.1016/j.resplu.2024.100626. eCollection 2024 Jun. Resusc Plus. 2024. PMID: 38623378 Free PMC article.
• Early Impressions and Adoption of the AtriAmp for Managing Arrhythmias Following Congenital Heart Surgery. Leopold SM, Brown DH, Zhang X, Nguyen XT, Al-Subu AM, Olson KR. Leopold SM, et al. Res Sq [Preprint]. 2024 Mar 21:rs.3.rs-4125331. doi: 10.21203/rs.3.rs-4125331/v1. Res Sq. 2024. Update in: Pediatr Cardiol. 2024 Jul 6. doi: 10.1007/s00246-024-03573-y. PMID: 38562710 Free PMC article. Updated. Preprint.
• Developing an innovative national ACP-OSCE program in Taiwan: a mixed method study. Wu YL, Hsieh TY, Hwang SF, Lin YY, Chu WM. Wu YL, et al. BMC Med Educ. 2024 Mar 23;24(1):333. doi: 10.1186/s12909-024-05294-5. BMC Med Educ. 2024. PMID: 38521917 Free PMC article.

Publication types

• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• Enlighten: Publications, University of Glasgow

Research Materials

• NCI CPTC Antibody Characterization Program

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• Open access
• Published: 27 May 2020

How to use and assess qualitative research methods

• Loraine Busetto   ORCID: orcid.org/0000-0002-9228-7875 1 ,
• Wolfgang Wick 1 , 2 &
• Christoph Gumbinger 1  

Neurological Research and Practice volume  2 , Article number:  14 ( 2020 ) Cite this article

762k Accesses

337 Citations

89 Altmetric

Metrics details

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 , 8 , 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 , 10 , 11 , 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 , 25 , 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 , 32 , 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 , 38 , 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Availability of data and materials

Not applicable.

Abbreviations

Endovascular treatment

Randomised Controlled Trial

Standard Operating Procedure

Standards for Reporting Qualitative Research

Philipsen, H., & Vernooij-Dassen, M. (2007). Kwalitatief onderzoek: nuttig, onmisbaar en uitdagend. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Qualitative research: useful, indispensable and challenging. In: Qualitative research: Practical methods for medical practice (pp. 5–12). Houten: Bohn Stafleu van Loghum.

Chapter   Google Scholar  

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches . London: Sage.

Kelly, J., Dwyer, J., Willis, E., & Pekarsky, B. (2014). Travelling to the city for hospital care: Access factors in country aboriginal patient journeys. Australian Journal of Rural Health, 22 (3), 109–113.

Article   Google Scholar  

Nilsen, P., Ståhl, C., Roback, K., & Cairney, P. (2013). Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Science, 8 (1), 1–12.

Howick J, Chalmers I, Glasziou, P., Greenhalgh, T., Heneghan, C., Liberati, A., Moschetti, I., Phillips, B., & Thornton, H. (2011). The 2011 Oxford CEBM evidence levels of evidence (introductory document) . Oxford Center for Evidence Based Medicine. https://www.cebm.net/2011/06/2011-oxford-cebm-levels-evidence-introductory-document/ .

Eakin, J. M. (2016). Educating critical qualitative health researchers in the land of the randomized controlled trial. Qualitative Inquiry, 22 (2), 107–118.

May, A., & Mathijssen, J. (2015). Alternatieven voor RCT bij de evaluatie van effectiviteit van interventies!? Eindrapportage. In Alternatives for RCTs in the evaluation of effectiveness of interventions!? Final report .

Google Scholar  

Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299 (10), 1182–1184.

Article   CAS   Google Scholar  

Christ, T. W. (2014). Scientific-based research and randomized controlled trials, the “gold” standard? Alternative paradigms and mixed methodologies. Qualitative Inquiry, 20 (1), 72–80.

Lamont, T., Barber, N., Jd, P., Fulop, N., Garfield-Birkbeck, S., Lilford, R., Mear, L., Raine, R., & Fitzpatrick, R. (2016). New approaches to evaluating complex health and care systems. BMJ, 352:i154.

Drabble, S. J., & O’Cathain, A. (2015). Moving from Randomized Controlled Trials to Mixed Methods Intervention Evaluation. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 406–425). London: Oxford University Press.

Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science : IS, 8 , 117.

Hak, T. (2007). Waarnemingsmethoden in kwalitatief onderzoek. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Observation methods in qualitative research] (pp. 13–25). Houten: Bohn Stafleu van Loghum.

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating qualitative research. Australian and New Zealand Journal of Psychiatry, 36 , 717–732.

Yanow, D. (2000). Conducting interpretive policy analysis (Vol. 47). Thousand Oaks: Sage University Papers Series on Qualitative Research Methods.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

van der Geest, S. (2006). Participeren in ziekte en zorg: meer over kwalitatief onderzoek. Huisarts en Wetenschap, 49 (4), 283–287.

Hijmans, E., & Kuyper, M. (2007). Het halfopen interview als onderzoeksmethode. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [The half-open interview as research method (pp. 43–51). Houten: Bohn Stafleu van Loghum.

Jansen, H. (2007). Systematiek en toepassing van de kwalitatieve survey. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Systematics and implementation of the qualitative survey (pp. 27–41). Houten: Bohn Stafleu van Loghum.

Pv, R., & Peremans, L. (2007). Exploreren met focusgroepgesprekken: de ‘stem’ van de groep onder de loep. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Exploring with focus group conversations: the “voice” of the group under the magnifying glass (pp. 53–64). Houten: Bohn Stafleu van Loghum.

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41 (5), 545–547.

Boeije H: Analyseren in kwalitatief onderzoek: Denken en doen, [Analysis in qualitative research: Thinking and doing] vol. Den Haag Boom Lemma uitgevers; 2012.

Hunter, A., & Brewer, J. (2015). Designing Multimethod Research. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 185–205). London: Oxford University Press.

Archibald, M. M., Radil, A. I., Zhang, X., & Hanson, W. E. (2015). Current mixed methods practices in qualitative research: A content analysis of leading journals. International Journal of Qualitative Methods, 14 (2), 5–33.

Creswell, J. W., & Plano Clark, V. L. (2011). Choosing a Mixed Methods Design. In Designing and Conducting Mixed Methods Research . Thousand Oaks: SAGE Publications.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320 (7226), 50–52.

O'Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine : Journal of the Association of American Medical Colleges, 89 (9), 1245–1251.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality and Quantity, 52 (4), 1893–1907.

Moser, A., & Korstjens, I. (2018). Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. European Journal of General Practice, 24 (1), 9–18.

Marlett, N., Shklarov, S., Marshall, D., Santana, M. J., & Wasylak, T. (2015). Building new roles and relationships in research: A model of patient engagement research. Quality of Life Research : an international journal of quality of life aspects of treatment, care and rehabilitation, 24 (5), 1057–1067.

Demian, M. N., Lam, N. N., Mac-Way, F., Sapir-Pichhadze, R., & Fernandez, N. (2017). Opportunities for engaging patients in kidney research. Canadian Journal of Kidney Health and Disease, 4 , 2054358117703070–2054358117703070.

Noyes, J., McLaughlin, L., Morgan, K., Roberts, A., Stephens, M., Bourne, J., Houlston, M., Houlston, J., Thomas, S., Rhys, R. G., et al. (2019). Designing a co-productive study to overcome known methodological challenges in organ donation research with bereaved family members. Health Expectations . 22(4):824–35.

Piil, K., Jarden, M., & Pii, K. H. (2019). Research agenda for life-threatening cancer. European Journal Cancer Care (Engl), 28 (1), e12935.

Hofmann, D., Ibrahim, F., Rose, D., Scott, D. L., Cope, A., Wykes, T., & Lempp, H. (2015). Expectations of new treatment in rheumatoid arthritis: Developing a patient-generated questionnaire. Health Expectations : an international journal of public participation in health care and health policy, 18 (5), 995–1008.

Jun, M., Manns, B., Laupacis, A., Manns, L., Rehal, B., Crowe, S., & Hemmelgarn, B. R. (2015). Assessing the extent to which current clinical research is consistent with patient priorities: A scoping review using a case study in patients on or nearing dialysis. Canadian Journal of Kidney Health and Disease, 2 , 35.

Elsie Baker, S., & Edwards, R. (2012). How many qualitative interviews is enough? In National Centre for Research Methods Review Paper . National Centre for Research Methods. http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf .

Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18 (2), 179–183.

Sim, J., Saunders, B., Waterfield, J., & Kingstone, T. (2018). Can sample size in qualitative research be determined a priori? International Journal of Social Research Methodology, 21 (5), 619–634.

Download references

Acknowledgements

no external funding.

Author information

Authors and affiliations.

Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120, Heidelberg, Germany

Loraine Busetto, Wolfgang Wick & Christoph Gumbinger

Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Wolfgang Wick

You can also search for this author in PubMed   Google Scholar

Contributions

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

Corresponding author

Correspondence to Loraine Busetto .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Busetto, L., Wick, W. & Gumbinger, C. How to use and assess qualitative research methods. Neurol. Res. Pract. 2 , 14 (2020). https://doi.org/10.1186/s42466-020-00059-z

Download citation

Received : 30 January 2020

Accepted : 22 April 2020

Published : 27 May 2020

DOI : https://doi.org/10.1186/s42466-020-00059-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Mixed methods
• Quality assessment

Neurological Research and Practice

ISSN: 2524-3489

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Center for Qualitative Studies in Health and Medicine

• Our Research
• Qualitative Courses

Dissertation Enhancement Awards

Center for qualitative studies in health and medicine.

The mission of the Center for Qualitative Studies in Health and Medicine (CQSHM) is to build capacity and community around the conduct of high-quality qualitative research across the Johns Hopkins schools of Public Health, Medicine and Nursing.

Founded in 2011, CQSHM serves to connect qualitative researchers—faculty and students—with faculty in need of qualitative and mixed methods expertise for their research. The center organizes qualitative research symposia, provides small grants to support qualitative doctoral dissertations, and has initiated a  two-part summer institute training program  in qualitative interviewing and analysis.

Dr. Katherine Clegg Smith serves as director of the CQSHM, and Dr. Shannon Frattaroli and Dr. Caitlin Kennedy are co-directors. Currently, the center has more than 20 affiliated faculty and research staff from across the East Baltimore campus of the Johns Hopkins University.

Center Leadership and Affiliated Faculty

Qualitative course offerings, research activities, cqshm mailing list.

Please contact  Susan Hannum , the CQSHM center coordinator, to be added to the list. 

• Open access
• Published: 13 December 2018

Using qualitative Health Research methods to improve patient and public involvement and engagement in research

• Danielle E. Rolfe 1 ,
• Vivian R. Ramsden 2 ,
• Davina Banner 3 &
• Ian D. Graham 1  

Research Involvement and Engagement volume  4 , Article number:  49 ( 2018 ) Cite this article

44k Accesses

64 Citations

23 Altmetric

Metrics details

Plain English summary

Patient engagement (or patient and public involvement) in health research is becoming a requirement for many health research funders, yet many researchers have little or no experience in engaging patients as partners as opposed to research subjects. Additionally, many patients have no experience providing input on the research design or acting as a decision-making partner on a research team. Several potential risks exist when patient engagement is done poorly, despite best intentions. Some of these risks are that: (1) patients’ involvement is merely tokenism (patients are involved but their suggestions have little influence on how research is conducted); (2) engaged patients do not represent the diversity of people affected by the research; and, (3) research outcomes lack relevance to patients’ lives and experiences.

Qualitative health research (the collection and systematic analysis of non-quantitative data about peoples’ experiences of health or illness and the healthcare system) offers several approaches that can help to mitigate these risks. Several qualitative health research methods, when done well, can help research teams to: (1) accurately incorporate patients’ perspectives and experiences into the design and conduct of research; (2) engage diverse patient perspectives; and, (3) treat patients as equal and ongoing partners on the research team.

This commentary presents several established qualitative health research methods that are relevant to patient engagement in research. The hope is that this paper will inspire readers to seek more information about qualitative health research, and consider how its established methods may help improve the quality and ethical conduct of patient engagement for health research.

Research funders in several countries have posited a new vision for research that involves patients and the public as co-applicants for the funding, and as collaborative partners in decision-making at various stages and/or throughout the research process. Patient engagement (or patient and public involvement) in health research is presented as a more democratic approach that leads to research that is relevant to the lives of the people affected by its outcomes. What is missing from the recent proliferation of resources and publications detailing the practical aspects of patient engagement is a recognition of how existing research methods can inform patient engagement initiatives. Qualitative health research, for example, has established methods of collecting and analyzing non-quantitative data about individuals’ and communities’ lived experiences with health, illness and/or the healthcare system. Included in the paradigm of qualitative health research is participatory health research, which offers approaches to partnering with individuals and communities to design and conduct research that addresses their needs and priorities.

The purpose of this commentary is to explore how qualitative health research methods can inform and support meaningful engagement with patients as partners. Specifically, this paper addresses issues of: rigour (how can patient engagement in research be done well?); representation (are the right patients being engaged?); and, reflexivity (is engagement being done in ways that are meaningful, ethical and equitable?). Various qualitative research methods are presented to increase the rigour found within patient engagement. Approaches to engage more diverse patient perspectives are presented to improve representation beyond the common practice of engaging only one or two patients. Reflexivity, or the practice of identifying and articulating how research processes and outcomes are constructed by the respective personal and professional experiences of researchers and patients, is presented to support the development of authentic, sustainable, equitable and meaningful engagement of patients as partners in health research.

Conclusions

Researchers will need to engage patients as stakeholders in order to satisfy the overlapping mandate in health policy, care and research for engaging patients as partners in decision-making. This paper presents several suggestions to ground patient engagement approaches in established research designs and methods.

Peer Review reports

Patient engagement (or patient and public involvement) in research involves partnering with ‘patients’ (a term more often used in Canada and the US, that is inclusive of individuals, caregivers, and/or members of the public) to facilitate research related to health or healthcare services. Rather than research subjects or participants, patients are engaged as partners in the research process. This partnership is intended to be meaningful and ongoing, from the outset of planning a research project, and/or at various stages throughout the research process. Engagement can include the involvement of patients in defining a research question, identifying appropriate outcomes and methods, collecting and interpreting data, and developing and delivering a knowledge translation strategy [ 1 ].

The concept of engaging non-researchers throughout the research process is not new to participatory health researchers, or integrated knowledge translation researchers, as the latter involves ongoing collaboration with clinicians, health planners and policy makers throughout the research process in order to generate new knowledge [ 2 , 3 ]. Patients, however, are less frequently included as partners on health research teams, or as knowledge users in integrated knowledge translation research teams compared to clinicians, healthcare managers and policy-makers, as these individuals are perceived as having “the authority to invoke change in the practice or policy setting.” (p.2) [ 2 ] Recent requirements for patient engagement by health research funders [ 4 , 5 , 6 ], ,and mandates by most healthcare planners and organizations to engage patients in healthcare improvement initiatives, suggest that it would be prudent for integrated knowledge translation (and indeed all) health researchers to begin engaging patients as knowledge users in many, if not all, of their research projects.

Training and tools for patient engagement are being developed and implemented in Canada via the Canadian Institutes for Health Research (CIHR) Strategy for Patient Oriented Research (SPOR) initiative, in the US via Patient Centered Outcomes Research Institute (PCORI), and very practical resources are already available from the UK’s more established INVOLVE Advisory Group [ 5 , 6 , 7 ]. What is seldom provided by these ‘get started’ guides, however, are rigorous methods and evidence-based approaches to engaging diverse patient perspectives, and ensuring that their experiences, values and advice are appropriately incorporated into the research process.

The purpose of this commentary is to stimulate readers’ further discussion and inquiry into qualitative health research methods as a means of fostering the more meaningfully engagement of patients as partners for research. Specifically, this paper will address issues of: rigour (how do we know that the interpretation of patients’ perspectives has been done well and is applicable to other patients?); representation (are multiple and diverse patient perspectives being sought?); and, reflexivity (is engagement being done ethically and equitably?). This commentary alone is insufficient to guide researchers and patient partners to use the methods presented as part of their patient engagement efforts. However, with increased understanding of these approaches and perhaps guidance from experienced qualitative health researchers, integrated knowledge translation and health researchers alike may be better prepared to engage patients in a meaningful way in research that has the potential to improve health and healthcare experiences and outcomes.

What can be learned from methods utilized in qualitative health research?

There is wide variation in researchers’ and healthcare providers’ openness to engaging patients [ 8 ]. Often, the patients that are engaged are a select group of individuals known to the research team, sometimes do not reflect the target population of the research, are involved at a consultative rather than a partnership level, and are more likely to be involved in the planning rather than the dissemination of research [ 9 , 10 , 11 ]. As a result, patient engagement can be seen as tokenistic and the antithesis of the intention of most patient engagement initiatives, which is to have patients’ diverse experiences and perspectives help to shape what and how research is done. The principles, values, and practices of qualitative health research (e.g., relativism, social equity, inductive reasoning) have rich epistemological traditions that align with the conceptual and practical spirit of patient engagement. It is beyond the scope of this commentary, however, to describe in detail the qualitative research paradigm, and readers are encouraged to gain greater knowledge of this topic via relevant courses and texts. Nevertheless, several qualitative research considerations and methods can be applied to the practice of patient engagement, and the following sections describe three of these: rigour, representation and reflexivity.

Rigour: Interpreting and incorporating patients’ experiences into the design and conduct of research

When patient engagement strategies go beyond the inclusion of a few patient partners on the research team, for example, by using focus groups, interviews, community forums, or other methods of seeking input from a broad range of patient perspectives, the diversity of patients’ experiences or perspectives may be a challenge to quickly draw conclusions from in order to make decisions about the study design. To make these decisions, members of the research team (which should include patient partners) may discuss what they heard about patients’ perspectives and suggestions, and then unsystematically incorporate these suggestions, or they may take a vote, try to achieve consensus, implement a Delphi technique [ 12 ], or use another approach designed specifically for patient engagement like the James Lind Alliance technique for priority setting [ 13 ]. Although the information gathered from patients is not data (and indeed would require ethical review to be used as such), a number of qualitative research practices designed to increase rigour can be employed to help ensure that the interpretation and incorporation of patients’ experiences and perspectives has been done systematically and could be reproduced [ 14 ]. These practices include member checking , dense description , and constant comparative analysis . To borrow key descriptors of rigour from qualitative research, these techniques improve “credibility” (i.e., accurate representations of patients’ experiences and preferences that are likely to be understood or recognized by other patients in similar situations – known in quantitative research as internal validity), and “transferability” (or the ability to apply what was found among a group of engaged patients to other patients in similar contexts – known in quantitative research as external validity) [ 15 ].

Member checking

Member checking in qualitative research involves “taking ideas back to the research participants for their confirmation” (p. 111) [ 16 ]. The objective of member checking is to ensure that a researcher’s interpretation of the data (whether a single interview with a participant, or after analyzing several interviews with participants) accurately reflects the participants’ intended meaning (in the case of a member check with a single participant about their interview), or their lived experience (in the case of sharing an overall finding about several individuals with one or more participants) [ 16 ]. For research involving patient engagement, member checking can be utilized to follow-up with patients who may have been engaged at one or only a few time points, or on an on-going basis with patient partners. A summary of what was understood and what decisions were made based on patients’ recommendations could be used to initiate this discussion and followed up with questions such as, “have I understood correctly what you intended to communicate to me?” or “do you see yourself or your experience(s) reflected in these findings or suggestions for the design of the study?”

Dense description

As with quantitative research, detailed information about qualitative research methods and study participants is needed to enable other researchers to understand the context and focus of the research and to establish how these findings relate more broadly. This helps researchers to not only potentially repeat the study, but to extend its findings to similar participants in similar contexts. Dense description provides details of the social, demographic and health profile of participants (e.g., gender, education, health conditions, etc.), as well as the setting and context of their experiences (i.e., where they live, what access to healthcare they have). In this way, dense description improves the transferability of study findings to similar individuals in similar situations [ 15 ]. To date, most studies involving patient engagement provide limited details about their engagement processes and who was engaged [ 17 ]. This omission may be done intentionally (e.g., to protect the privacy of engaged patients, particularly those with stigmatizing health conditions), or as a practical constraint such as publication word limits. Nonetheless, reporting of patient engagement using some aspects of dense description of participants (as appropriate), the ways that they were engaged, and recommendations that emanated from engaged patients can also contribute to greater transferability and understanding of how patient engagement influenced the design of a research study.

Constant comparative analysis

Constant comparative analysis is a method commonly used in grounded theory qualitative research [ 18 ]. Put simply, the understanding of a phenomenon or experience that a researcher acquires through engaging with participants is constantly redeveloped and refined based on subsequent participant interactions. This process of adapting to new information in order to make it more relevant is similar to processes used in rapid cycle evaluation during implementation research [ 19 ]. This method can be usefully adapted and applied to research involving ongoing collaboration and partnership with several engaged patient partners, and/or engagement strategies that seek the perspectives of many patients at various points in the research process. For example, if, in addition to having ongoing patient partners, a larger group of patients provides input and advice (e.g., a steering or advisory committee) at different stages in the research process, their input may result in multiple course corrections during the design and conduct of the research processes to incorporate their suggestions. These suggestions may result in refinement of earlier decisions made about study design or conduct, and as such, the research process becomes more iterative rather than linear. In this way, engaged patients and patient partners are able to provide their input and experience to improve each step of the research process from formulating an appropriate research question or objective, determining best approaches to conducting the research and sharing it with those most affected by the outcomes.

Representation: Gathering diverse perspectives to design relevant and appropriate research studies

The intention of engaging patients is to have their lived experience of health care or a health condition contribute to the optimization of a research project design [ 20 ]. Development of a meaningful and sustainable relationship with patient partners requires considerable time, a demonstrated commitment to partnership by both the patient partners and the researcher(s), resources to facilitate patient partners’ engagement, and often, an individual designated to support the development of this relationship [ 17 , 21 ]. This may lead some research teams to sustain this relationship with only one or two patients who are often previously known to the research team [ 17 ]. The limitation of this approach is that the experiences of these one or two individuals may not adequately reflect the diverse perspectives of patients that may be affected by the research or its outcomes. The notion of gaining ‘ the patient perspective’ from a single or only a few individuals has already been problematized [ 22 , 23 ]. To be sure, the engagement of a single patient is better than none at all, but the engagement of a broader and diverse population of patients should be considered to better inform the research design, and to help prevent further perpetuation of health disparities. Key issues to be considered include (1) how engagement can be made accessible to patients from diverse backgrounds, and (2) which engagement strategies (e.g., ranging from a community information forum to full partnership on the research team) are most appropriate to reach the target population [ 24 ].

Making engagement accessible

Expecting patient partner(s) to attend regular research team meetings held during working hours in a boardroom setting in a hospital, research institute or university limits the participation of many individuals. To support the participation and diversity of engaged patients, effort should be made to increase the accessibility and emotional safety of engagement initiatives [ 25 ]. A budget must be allocated for patient partners’ transportation, childcare or caregiving support, remuneration for time or time taken off work and, at the very least, covering expenses related to their engagement. Another consideration that is often made by qualitative health researchers is whether brief counselling support can be provided to patients should the sharing of their experiences result in emotional distress. There are some resources that can help with planning for costs [ 26 ], including an online cost calculator [ 27 ].

Engagement strategies

Patient partners can be coached to consider the needs and experiences of people unlike them, but there are other methods of engagement that can help to gain a more fulsome perspective of what is likely a diverse patient population that is the focus of the research study. In qualitative health research, this is known as purposeful or purposive sampling: finding people who can provide information-rich descriptions of the phenomenon under study [ 28 ]. Engagement may require different approaches (e.g., deliberative group processes, community forums, focus groups, and patient partners on the research team), at different times in the research process to reach different individuals or populations (e.g., marginalized patients, or patients or caregivers experiencing illnesses that inhibit their ability to maintain an ongoing relationship with the research team). Engagement strategies of different forms at different times may be required. For example, ongoing engagement may occur with patient partners who are members of the research team (e.g., co-applicants on a research grant), and intermittent engagement may be sought from other patients through other methods that may be more time-limited or accessible to a diverse population of patients (e.g., a one-time focus group, community forum, or ongoing online discussion) to address issues that may arise during various stages of the research or dissemination processes. The result of this approach is that patients are not only consulted or involved (one-time or low commitment methods), but are also members of the research team and have the ability to help make decisions about the research being undertaken.

Engagement can generate a wealth of information from very diverse perspectives. Each iteration of engagement may yield new information. Knowing when enough information has been gathered to make decisions with the research team (that includes patient partners) about how the research may be designed or conducted can be challenging. One approach from qualitative research that can be adapted for patient engagement initiatives is theoretical saturation [ 29 ], or “the point in analysis when…further data gathering and analysis add little new to the conceptualization, though variations can always be discovered.” (p. 263) [ 18 ]. That is, a one-time engagement strategy (e.g., a discussion with a single patient partner) may be insufficient to acquire the diverse perspectives of the individuals that will be affected by the research or its outcomes. Additional strategies (e.g., focus groups or interviews with several individuals) may be initiated until many patients identify similar issues or recommendations.

Engagement approaches should also consider: how patients are initially engaged (e.g., through known or new networks, posted notices, telephone or in-person recruitment) and whether involvement has been offered widely enough to garner multiple perspectives; how patients’ experiences are shared (e.g., community forums, formal meetings, individual or group discussions) and whether facilitation enables broad participation; and finally, how patients’ participation and experiences are incorporated into the research planning and design, with patients having equal decision-making capacity to other research team members. Several publications and tools are available that can help guide researchers who are new to processes of engaging patients in research [ 24 , 30 , 31 , 32 , 33 , 34 ], but unfortunately few address how to evaluate the effectiveness of engagement [ 35 ].

Reflexivity: Ensuring meaningful and authentic engagement

In qualitative research, reflexivity is an ongoing process of “the researcher’s scrutiny of his or her research experience, decisions, and interpretations in ways that bring the researcher into the process and allow the reader to assess how and to what extent the researcher’s interests, positions, and assumptions influenced inquiry. A reflexive stance informs how the researcher conducts his or her research, relates to the research participants, and represents them in written reports,” (p.188–189) [ 16 ]. The concept of reflexivity can be applied to research involving patient engagement by continually and explicitly considering how decisions about the research study were made. All members of the research team must consider (and perhaps discuss): (1) how patient partners are invited to participate in research planning and decision-making; (2) how their input is received relative to other team members (i.e., do their suggestions garner the same respect as researchers’ or providers’?); and, (3) whether engaged patients or patient partners feel sufficiently safe, able and respected to share their experiences, preferences and recommendations with the research team.

Ideally, reflexivity becomes a practice within the research team and may be operationalized through regular check-ins with patients and researchers about their comfort in sharing their views, and whether they feel that their views have been considered and taken onboard. Power dynamics should also be considered during patient engagement initiatives. For example, reflecting on how community forums, focus groups or interviews are to be facilitated, including a consideration of who is at the table/who is not, who speaks/who does not, whose suggestions are implemented/whose are not? Reflexivity can be practiced through informal discussions, or using methods that may allow more candid responses by engaged patients (e.g., anonymous online survey or feedback forms). At the very least, if these practices were not conducted throughout the research process, the research team (including patient partners) should endeavor to reflect upon team dynamics and consider how these may have contributed to the research design or outcomes. For example, were physicians and researchers seen as experts and patients felt less welcome or able to share their personal experiences? Were patients only engaged by telephone rather than in-person and did this influence their ability to easily engage in decision-making? Reflexive practices may be usefully supplemented by formal evaluation of the process of patient engagement from the perspective of patients and other research team members [ 36 , 37 ], and some tools are available to do this [ 35 ].

A note about language

One way to address the team dynamic between researchers, professional knowledge users (such as clinicians or health policy planners) and patients is to consider the language used to engage with patients in the planning of patient engagement strategies. That is, the term ‘patient engagement’ is a construction of an individual’s identity that exists only within the healthcare setting, and in the context of a patient-provider dynamic. This term does not consider how people make decisions about their health and healthcare within a broader context of their family, community, and culture [ 22 , 38 ]. This may be why research communities in some countries (e.g., the United Kingdom) use the term ‘patient and public involvement’. Additionally, research that involves communities defined by geography, shared experiences, cultural or ethnic identity, as is the case with participatory health research, may refer to ‘community engagement.’ Regardless of the term used, partnerships with patients, the public, or with communities need to be conceived instead as person-to-person interactions between researchers and individuals who are most affected by the research. Discussions with engaged patients should be conducted early on to determine how to best describe their role on the team or during engagement initiatives (e.g., as patient partners, community members, or people with lived experience).

Tokenism is the “difference between…the empty ritual of participation and having the real power needed to affect the outcome,” (p.2) [ 39 ]. Ongoing reflection on the power dynamic between researchers and engaged patients, a central tenet of critical qualitative health research [ 40 , 41 ], can increase the likelihood that engagement involves equitable processes and will result in meaningful engagement experiences by patients rather than tokenism [ 36 , 42 ]. Patient engagement initiatives should strive for “partnership” amongst all team members, and not just reflect a patient-clinician or researcher-subject dynamic [ 43 ]. To develop meaningful, authentic and sustainable relationships with engaged patients, methods used for participatory, action or community-based research (approaches that fall under the paradigm of qualitative inquiry) provide detailed experiential guidance [ 44 ]. For example, a realist review of community-based participatory research projects reported that gaining and maintaining trust with patient or community partners, although time-intensive, is foundational to equitable and sustainable partnerships that benefit communities and individuals [ 45 , 46 ]. Additionally, Chapter Nine of the Canadian Tri-Council Policy Statement on Research involving Humans, which has to date been applied to research involving First Nations, Inuit and, Métis Peoples in Canada [ 47 ], provides useful information and direction that can be applied to working with patient partners on research [ 48 ].

Authentic patient engagement should include their involvement at all stages of the research process [ 49 , 50 ], but this is often not the case [ 10 ]. .Since patient partners are not research subjects or participants, their engagement does not (usually) require ethics approval, and they can be engaged as partners as early as during the submission of grant applications [ 49 ]. This early engagement helps to incorporate patients’ perspectives into the proposed research before the project is wedded to particular objectives, outcomes and methods, and can also serve to allocate needed resources to support patient engagement (including remuneration for patient partners’ time). Training in research for patient partners can also support their meaningful engagement by increasing their ability to fully engage in decision-making with other members of the research team [ 51 , 52 ]. Patient partners may also thrive in co-leading the dissemination of findings to healthcare providers, researchers, patients or communities most affected by the research [ 53 ].

Patient engagement has gained increasing popularity, but many research organizations are still at the early stages of developing approaches and methods, many of which are based on experience rather than evidence. As health researchers and members of the public will increasingly need to partner for research to satisfy the overlapping mandate of patient engagement in health policy, healthcare and research, the qualitative research methods highlighted in this commentary provide some suggestions to foster rigorous, meaningful and sustained engagement initiatives while addressing broader issues of power and representation. By incorporating evidence-based methods of gathering and learning from multiple and diverse patient perspectives, we will hopefully conduct better patient engaged research, live out the democratic ideals of patient engagement, and ultimately contribute to research that is more relevant to the lives of patients; as well as, contribute to the improved delivery of healthcare services. In addition to the references provided in this paper, readers are encouraged to learn more about the meaningful engagement of patients in research from several key texts [ 54 , 55 , 56 ].

Abbreviations

Canadian Institutes for Health Research

Patient Centered Outcomes Research Institute

Strategy for Patient Oriented Research

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research. 2014. http://www.cihr-irsc.gc.ca/e/documents/spor_framework-en.pdf . Accessed 27 Sep 2017.

Google Scholar  

Kothari A, Wathen CN. Integrated knowledge translation: digging deeper, moving forward. J Epidemiol Community Health. 2017;71:619–23. https://doi.org/10.1136/jech-2016-208490 .

Article   PubMed   Google Scholar  

Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci. 2016;11:38. https://doi.org/10.1186/s13012-016-0399-1 .

Article   PubMed   PubMed Central   Google Scholar  

Canadian Institutes of Health Research. Strategy for Patient-Oriented Research - CIHR. http://www.cihr-irsc.gc.ca/e/41204.html . Accessed 20 Sep 2017.

Patient-Centered Outcomes Research Institute. Patient-Centered Outcomes Research Institute website. https://www.pcori.org/ . Accessed 20 Sep 2017.

National Institute for Health Research. INVOLVE | INVOLVE Supporting public involvement in NHS, public health and social care research. http://www.invo.org.uk/ . Accessed 27 Sep 2017.

Canadian Institutes of Health Research. SPOR SUPPORT Units - CIHR. http://www.cihr-irsc.gc.ca/e/45859.html . Accessed 27 Sep 2017.

Ocloo J, Matthews R. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf. 2016;25:626–32.

Hearld KR, Hearld LR, Hall AG. Engaging patients as partners in research: factors associated with awareness, interest, and engagement as research partners. SAGE open Med. 2017;5:2050312116686709. https://doi.org/10.1177/2050312116686709.

Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89. https://doi.org/10.1186/1472-6963-14-89 .

Concannon TW, Fuster M, Saunders T, Patel K, Wong JB, Leslie LK, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med. 2014;29:1692–701.

Oostendorp LJM, Durand M-A, Lloyd A, Elwyn G. Measuring organisational readiness for patient engagement (MORE): an international online Delphi consensus study. BMC Health Serv Res. 2015;15:61. https://doi.org/10.1186/s12913-015-0717-3 .

Boddy K, Cowan K, Gibson A, Britten N. Does funded research reflect the priorities of people living with type 1 diabetes? A secondary analysis of research questions. BMJ Open. 2017;7:e016540. https://doi.org/10.1136/bmjopen-2017-016540 .

Mays N, Pope C. Rigour and qualitative research. BMJ Br Med J. 1995;311:109–12.

Article   CAS   Google Scholar  

Krefting L. Rigor in qualitative research: the assessment of trustworthiness. Am J Occup Ther. 1991;45:214–22.

Article   CAS   PubMed   Google Scholar  

Charmaz K. Constructing grounded theory : a practical guide through qualitative analysis. London: Sage Publications; 2006.

Wilson P, Mathie E, Keenan J, McNeilly E, Goodman C, Howe A, et al. ReseArch with patient and public invOlvement: a RealisT evaluation – the RAPPORT study. Heal Serv Deliv Res. 2015;3:1–176. https://doi.org/10.3310/hsdr03380 .

Corbin JM, Strauss AL, Strauss AL. Basics of qualitative research : techniques and procedures for developing grounded theory. Thousand Oaks: Sage Publications; 2008.

Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the consolidated framework for implementation research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci. 2017;12:15. https://doi.org/10.1186/s13012-017-0550-7 .

Esmail L, Moore E. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res. 2015;4:133–45. https://doi.org/10.2217/cer.14.79 .

Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7:387–95.

Rowland P, McMillan S, McGillicuddy P, Richards J. What is “the patient perspective” in patient engagement programs? Implicit logics and parallels to feminist theories. Heal An Interdiscip J Soc Study Heal Illn Med. 2016. https://doi.org/10.1177/1363459316644494 .

Article   Google Scholar  

Martin GP. “Ordinary people only”: knowledge, representativeness, and the publics of public participation in healthcare. Sociol Health Illn. 2008;30:35–54. https://doi.org/10.1111/j.1467-9566.2007.01027.x .

Snow B, Tweedie K, Pederson A, Shrestha H, Bachmann L. Patient Engagement: Heard and valued. 2013. http://www.cfhi-fcass.ca/sf-docs/default-source/patient-engagement/awesome_workbook-fraserhealth.pdf . Accessed 24 Jan 2018.

Montesanti SR, Abelson J, Lavis JN, Dunn JR. Enabling the participation of marginalized populations: case studies from a health service organization in Ontario. Canada Health Promot Int. 2017;32(4):636–49. https://doi.org/10.1093/heapro/dav118.

The Change Foundation. Should Money Come into It? A Tool for Deciding Whether to Pay Patient- Engagement Participants. Toronto; 2015. http://www.changefoundation.ca/site/wp-content/uploads/2016/05/Should-money-come-into-it.pdf . Accessed 18 June 2018

INVOLVE. Involvement Cost Calculator | INVOLVE. http://www.invo.org.uk/resource-centre/payment-and-recognition-for-public-involvement/involvement-cost-calculator/ . Accessed 11 Oct 2017.

Patton MQ. Qualitative research and evaluation methods. 3rd ed. Thousand Oaks: Sage Publications; 2002.

Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269–81. https://doi.org/10.1586/erp.10.30 .

Canadian Institutes of Health Research. CIHR’S Citizen Engagement Handbook. http://www.cihr-irsc.gc.ca/e/documents/ce_handbook_e.pdf . Accessed 18 June 2018.

Elliott J, Heesterbeek S, Lukensmeyer CJ, Slocum N. Participatory and deliberative methods toolkit how to connect with citizens: a practitioner’s manual. King Baudouin Foundation; 2005. https://www.kbs-frb.be/en/Virtual-Library/2006/294864 . Accessed 28 Sep 2017.

Robbins M, Tufte J, Hsu C. Learning to “swim” with the experts: experiences of two patient co-investigators for a project funded by the Patient-Centered Outcomes Research Institute. Perm J. 2016;20:85–8.

PubMed   PubMed Central   Google Scholar  

Gauvin F-P, Abelson J, Lavis JN. Strengthening public and patient engagement in health technology assessment in Ontario. 2014. https://www.mcmasterforum.org/docs/default-source/product-documents/evidence-briefs/public-engagement-in-health-technology-assessement-in-ontario-eb.pdf?sfvrsn=2 . Accessed 18 June 2018.

Abelson J, Montesanti S, Li K, Gauvin F-P, Martin E. Effective strategies for interactive public engagement in the development of healthcare policies and Programs. 2010; https://www.cfhi-fcass.ca/sf-docs/default-source/commissionedresearch-reports/Abelson_EN_FINAL.pdf?sfvrsn=0. . Accessed 16 Nov 2018.

Abelson J. Public and patient engagement evaluation tool (version 1.0), vol. 0; 2015. https://www.fhs.mcmaster.ca/publicandpatientengagement/ppeet_request_form.html ! Accessed 18 June 2018

Johnson DS, Bush MT, Brandzel S, Wernli KJ. The patient voice in research—evolution of a role. Res Involv Engagem. 2016;2:6. https://doi.org/10.1186/s40900-016-0020-4 .

Crocker JC, Boylan AM, Bostock J, Locock L. Is it worth it? Patient and public views on the impact of their involvement in health research and its assessment: a UK-based qualitative interview study. Health Expect. 2017;20(3):519–28. https://doi.org/10.1111/hex.12479 .

Renedo A, Marston C. Healthcare professionals’ representations of “patient and public involvement” and creation of “public participant” identities: implications for the development of inclusive and bottom-up community participation initiatives. J Community Appl Soc Psychol. 2011;21:268–80. https://doi.org/10.1002/casp.1092 .

Hahn DL, Hoffmann AE, Felzien M, LeMaster JW, Xu J, Fagnan LJ. Tokenism in patient engagement. Fam Pract. 2017;34(3):290–5. https://doi.org/10.1093/fampra/cmw097.

Hart C, Poole JM, Facey ME, Parsons JA. Holding firm: power, push-Back, and opportunities in navigating the liminal space of critical qualitative Health Research. Qual Health Res. 2017;27:1765–74. https://doi.org/10.1177/1049732317715631 .

Eakin JM. Educating critical qualitative health researchers in the land of the randomized controlled trial. Qual Inq. 2016;22:107–18. https://doi.org/10.1177/1077800415617207 .

Safo S, Cunningham C, Beckman A, Haughton L, Starrels JL. “A place at the table:” a qualitative analysis of community board members’ experiences with academic HIV/AIDS research. BMC Med Res Methodol. 2016;16:80. https://doi.org/10.1186/s12874-016-0181-8 .

Thompson J, Bissell P, Cooper C, Armitage CJ, Barber R, Walker S, et al. Credibility and the “professionalized” lay expert: reflections on the dilemmas and opportunities of public involvement in health research. Heal Interdiscip J Soc Study Heal Illn Med. 2012;16:602–18. https://doi.org/10.1177/1363459312441008 .

Burke JG, Jones J, Yonas M, Guizzetti L, Virata MC, Costlow M, et al. PCOR, CER, and CBPR: alphabet soup or complementary fields of Health Research? Clin Transl Sci. 2013;6:493–6. https://doi.org/10.1111/cts.12064 .

Jagosh J, Bush PL, Salsberg J, Macaulay AC, Greenhalgh T, Wong G, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health. 2015;15:725. https://doi.org/10.1186/s12889-015-1949-1 .

Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012;90:311–46. https://doi.org/10.1111/j.1468-0009.2012.00665.x .

Government of Canada IAP on RE. TCPS 2 - Chapter 9 Research Involving the First Nations, Inuit and Métis Peoples of Canada. http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/chapter9-chapitre9/ . Accessed 16 Oct 2017.

Ramsden VR, Crowe J, Rabbitskin N, Rolfe D, Macaulay A. Authentic engagement, co-creation and action research. In: Goodyear-smith F, Mash B, editors. How to do primary care research. Boca Raton: CRC press, medicine, Taylor & Francis Group; 2019.

Ramsden VR, Salsberg J, Herbert CP, Westfall JM, LeMaster J, Macaulay AC. Patient- and community-oriented research: how is authentic engagement identified in grant applications? Can Fam Physician. 2017;63:74–6.

Woolf SH, Zimmerman E, Haley A, Krist AH. Authentic engagement of patients and communities can transform research, practice, and policy. Health Aff. 2016;35:590–4.

Parkes JH, Pyer M, Wray P, Taylor J. Partners in projects: preparing for public involvement in health and social care research. Health Policy. 2014;117:399–408. https://doi.org/10.1016/j.healthpol.2014.04.014 .

Norman N, Bennett C, Cowart S, Felzien M, Flores M, Flores R, et al. Boot camp translation: a method for building a community of solution. J Am Board Fam Med. 2013;26:254–63. https://doi.org/10.3122/jabfm.2013.03.120253 .

Ramsden VR, Rabbitskin N, Westfall JM, Felzien M, Braden J, Sand J. Is knowledge translation without patient or community engagement flawed. Fam Pract. 2016:1–3.

Denzin NK, Lincoln YS. SAGE Handbook of qualitative research. 3rd ed. Thousand Oaks: Sage Publications; 2005.

Pearce V, Baraitser P, Smith G, Greenhalgh T. Experience-based co-design. In: User involvement in health care. Oxford: Wiley-Blackwell; 2010. p. 28–51. https://doi.org/10.1002/9781444325164.ch3 .

Chapter   Google Scholar  

Hulatt I, Lowes L. Involving service users in health and social care research: Routledge; 2005. https://doi.org/10.4324/9780203865651.

Book   Google Scholar  

Download references

Acknowledgements

This paper was drafted in response to a call for concept papers related to integrated knowledge translation issued by the Integrated Knowledge Translation Research Network (CIHR FDN #143237).

This paper was commissioned by the Integrated Knowledge Translation Network (IKTRN). The IKTRN brings together knowledge users and researchers to advance the science and practice of integrated knowledge translation and train the next generation of integrated knowledge translation researchers. Honorariums were provided for completed papers. The IKTRN is funded by a Canadian Institutes of Health Research Foundation Grant (FDN #143247).

Availability of data and materials

Not applicable.

Author information

Authors and affiliations.

School of Epidemiology and Public Health, University of Ottawa, 307D- 600 Peter Morand Crescent, Ottawa, ON, K1G 5Z3, Canada

Danielle E. Rolfe & Ian D. Graham

Department of Academic Family Medicine, University of Saskatchewan, West Winds Primary Health Centre, 3311 Fairlight Drive, Saskatoon, SK, S7M 3Y5, Canada

Vivian R. Ramsden

School of Nursing, University of Northern British Columbia, 3333 University Way, Prince George, BC, V2K4C6, Canada

Davina Banner

You can also search for this author in PubMed   Google Scholar

Contributions

DR conceived and drafted this paper. All authors were involved in critiquing and revising the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Danielle E. Rolfe .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Rolfe, D.E., Ramsden, V.R., Banner, D. et al. Using qualitative Health Research methods to improve patient and public involvement and engagement in research. Res Involv Engagem 4 , 49 (2018). https://doi.org/10.1186/s40900-018-0129-8

Download citation

Received : 18 June 2018

Accepted : 09 November 2018

Published : 13 December 2018

DOI : https://doi.org/10.1186/s40900-018-0129-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative methods
• Qualitative health research
• Patient-oriented research
• Integrated knowledge translation
• Patient engagement
• Patient partners
• Patient and public involvement

Research Involvement and Engagement

ISSN: 2056-7529

• General enquiries: [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Comprehensive criteria for reporting qualitative research (ccqr): reporting guideline for global health qualitative research methods.

1. Introduction

2.1. search strategy, 2.2. eligibility criteria, 2.3. study selection process, 2.4. quality assessment of articles, 4. discussion, 5. conclusions, supplementary materials, author contributions, conflicts of interest.

• Malterud, K. Developing and promoting qualitative methods in general practice research: Lessons learnt and strategies convened. Scand. J. Public Health 2022 , 50 , 1024–1033. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tenny, S.; Brannan, J.M.; Brannan, G.D. Qualitative Study ; StatPearls—NCBI Bookshelf: Petersburg, FL, USA, 2022. [ Google Scholar ]
• Renjith, V.; Yesodharan, R.; Noronha, J.A.; Ladd, E.; George, A. Qualitative methods in health care research. Int. J. Prev. Med. 2021 , 12 , 20. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Im, D.; Pyo, J.; Lee, H.; Jung, H.; Ock, M. Qualitative research in healthcare: Data analysis. J. Prev. Med. Public Health 2023 , 56 , 100. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Aspers, P.; Corte, U. What is qualitative in qualitative research. Qual. Sociol. 2019 , 42 , 139–160. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Bryman, A. Social Research Methods ; Oxford University Press: Oxford, UK, 2012. [ Google Scholar ]
• Polit, D.F.; Beck, C.T. Nursing Research: Generating and Assessing Evidence for Nursing Practice , 11th ed.; Wolters Kluwer: Philadelphia, PA, USA, 2021. [ Google Scholar ]
• Jennifer, R.; Gray, J.R.; Grove, S.K. Burns and Grove’s The Practice of Nursing Research—E-Book: Appraisal, Synthesis, and Generation of Evidence , 9th ed.; Elsevier: St. Louis, MO, USA, 2021. [ Google Scholar ]
• Grossoehme, D.H. Overview of qualitative research. J. Health Care Chaplain. 2014 , 20 , 109–122. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Brandão, C.; Ribeiro, J.; Costa, A.P. Qualitative Research: Where Do We Stand Now? SciELO: São Paulo, Brasil, 2018; Volume 23, p. 4. [ Google Scholar ]
• Peditto, K. Reporting qualitative research: Standards, challenges, and implications for health design. HERD Health Environ. Res. Des. J. 2018 , 11 , 16–19. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lewin, S.; Glenton, C. Are we entering a new era for qualitative research? Using qualitative evidence to support guidance and guideline development by the World Health Organization. Int. J. Equity Health 2018 , 7 , 126. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mays, N.; Pope, C. Quality in qualitative research. Qual. Res. Health Care 2020 , 188 , 211–233. [ Google Scholar ]
• Johnson, J.L.; Adkins, D.; Chauvin, S. A review of the quality indicators of rigor in qualitative research. Am. J. Pharm. Educ. 2020 , 84 , 7120. [ Google Scholar ] [ CrossRef ]
• Dossett, L.A.; Kaji, A.H.; Cochran, A. SRQR and COREQ reporting guidelines for qualitative studies. JAMA Surg. 2021 , 156 , 875–876. [ Google Scholar ] [ CrossRef ]
• Godinho, M.A.; Gudi, N.; Milkowska, M.; Murthy, S.; Bailey, A.; Nair, N.S. Completeness of reporting in Indian qualitative public health research: A systematic review of 20 years of literature. J. Public Health 2019 , 41 , 405–411. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Flemming, K.; Booth, A.; Hannes, K.; Cargo, M.; Noyes, J. Cochrane Qualitative and Implementation Methods Group guidance series—Paper 6: Reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J. Clin. Epidemiol. 2018 , 97 , 79–85. [ Google Scholar ] [ CrossRef ]
• MacCarthy, A.; Kirtley, S.; de Beyer, J.A.; Altman, D.G.; Simera, I. Reporting guidelines for oncology research: Helping to maximise the impact of your research. Br. J. Cancer 2018 , 118 , 619–628. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tai, J.; Ajjawi, R. Undertaking and reporting qualitative research. Clin. Teach. 2016 , 13 , 175–182. [ Google Scholar ] [ CrossRef ]
• Buus, N.; Perron, A. The quality of quality criteria: Replicating the development of the Consolidated Criteria for Reporting Qualitative Research (COREQ). Int. J. Nurs. Stud. 2020 , 102 , 103452. [ Google Scholar ] [ CrossRef ]
• Tong, A.; Sainsbury, P.; Craig, J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int. J. Qual. Health Care 2007 , 19 , 349–357. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• O’Brien, B.C.; Harris, I.B.; Beckman, T.J.; Reed, D.A.; Cook, D.A. Standards for reporting qualitative research: A synthesis of recommendations. Acad. Med. 2014 , 89 , 1245–1251. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hannes, K.; Heyvaert, M.; Slegers, K.; Vandenbrande, S.; Van Nuland, M. Exploring the potential for a consolidated standard for reporting guidelines for qualitative research: An argument Delphi approach. Int. J. Qual. Methods 2015 , 14 , 1609406915611528. [ Google Scholar ] [ CrossRef ]
• Garcia, M.; Daugherty, C.; Khallouq, B.B.; Maugans, T. Critical assessment of pediatric neurosurgery patient/parent educational information obtained via the Internet. J. Neurosurg. Pediatr. 2018 , 21 , 535–541. [ Google Scholar ] [ CrossRef ]
• Research Guides: Evaluating Sources: The CRAAP Test. Available online: https://researchguides.ben.edu/source-evaluation (accessed on 9 June 2024).
• France, E.F.; Cunningham, M.; Ring, N.; Uny, I.; Duncan, E.A.; Jepson, R.G.; Maxwell, M.; Roberts, R.J.; Turley, R.L.; Booth, A. Improving reporting of meta-ethnography: The eMERGe reporting guidance. BMC Med. Res. Methodol. 2019 , 19 , 25. [ Google Scholar ] [ CrossRef ]
• Batten, J.; Brackett, A. Ensuring rigor in systematic reviews: Part 6, reporting guidelines. Heart Lung 2022 , 52 , 22–25. [ Google Scholar ] [ CrossRef ]
• Cronin, P.; Rawson, J.V. Review of research reporting guidelines for radiology researchers. Acad. Radiol. 2016 , 23 , 537–558. [ Google Scholar ] [ CrossRef ]
• Florczak, K.L. Reflexivity: Should it be mandated for qualitative reporting? Nurs. Sci. Q. 2021 , 34 , 352–355. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Blignault, I.; Ritchie, J. Revealing the wood and the trees: Reporting qualitative research. Health Promot. J. Aust. 2009 , 20 , 140–145. [ Google Scholar ] [ CrossRef ]
• Coast, J.; Al-Janabi, H.; Sutton, E.J.; Horrocks, S.A.; Vosper, A.J.; Swancutt, D.R.; Flynn, T.N. Using qualitative methods for attribute development for discrete choice experiments: Issues and recommendations. Health Econ. 2012 , 21 , 730–741. [ Google Scholar ] [ CrossRef ]
• Levitt, H.M.; Bamberg, M.; Creswell, J.W.; Frost, D.M.; Josselson, R.; Suárez-Orozco, C. Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. Am. Psychol. 2018 , 73 , 26. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Misiak, M.; Kurpas, D. Checklists for reporting research in Advances in Clinical and Experimental Medicine: How to choose a proper one for your manuscript? Adv. Clin. Exp. Med. Off. Organ Wroc. Med. Univ. 2022 , 31 , 1065–1072. [ Google Scholar ] [ CrossRef ]
• King, O.A.; Pinson, J.A.; Dennett, A.; Williams, C.; Davis, A.; Snowdon, D.A. Allied health assistants’ perspectives of their role in healthcare settings: A qualitative study. Health Soc. Care Community 2022 , 30 , e4684–e4693. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Pearson, A.; Jordan, Z.; Lockwood, C.; Aromataris, E. Notions of quality and standards for qualitative research reporting. Int. J. Nurs. Pract. 2015 , 21 , 670–676. [ Google Scholar ] [ CrossRef ]
• Clark, J. How to peer review a qualitative manuscript. In Peer Review in Health Sciences ; Godlee, F., Jefferson, T., Eds.; BMJ Books: London, UK, 2003; pp. 219–235. [ Google Scholar ]
• Salzmann-Erikson, M. IMPAD-22: A checklist for authors of qualitative nursing research manuscripts. Nurse Educ. Today 2013 , 33 , 1295–1300. [ Google Scholar ] [ CrossRef ]
• Hollin, I.L.; Craig, B.M.; Coast, J.; Beusterien, K.; Vass, C.; DiSantostefano, R.; Peay, H. Reporting formative qualitative research to support the development of quantitative preference study protocols and corresponding survey instruments: Guidelines for authors and reviewers. Patient-Patient-Centered Outcomes Res. 2020 , 13 , 121–136. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zachariah, R.; Abrahamyan, A.; Rust, S.; Thekkur, P.; Khogali, M.; Kumar, A.M.; Davtyan, H.; Satyanarayana, S.; Shewade, H.D.; Delamou, A. Quality, Equity and Partnerships in Mixed Methods and Qualitative Research during Seven Years of Implementing the Structured Operational Research and Training Initiative in 18 Countries. Trop. Med. Infect. Dis. 2022 , 7 , 305. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tong, A.; Flemming, K.; McInnes, E.; Oliver, S.; Craig, J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med. Res. Methodol. 2012 , 12 , 181. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Schulz, K.F.; Altman, D.G.; Moher, D. CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. J. Pharmacol. Pharmacother. 2010 , 1 , 100–107. [ Google Scholar ] [ CrossRef ]
• Kline, R.B. Becoming a Behavioral Science Researcher: A Guide to Producing Research that Matters ; Guilford Press: New York, NY, USA, 2008. [ Google Scholar ]
• Maxwell, J.A. Qualitative Research Design: An Interactive Approach ; Sage Publications: Thousand Oaks, CA, USA, 2012. [ Google Scholar ]
• Nguyen, T.N.M.; Whitehead, L.; Dermody, G.; Saunders, R. The use of theory in qualitative research: Challenges, development of a framework and exemplar. J. Adv. Nurs. 2022 , 78 , e21–e28. [ Google Scholar ] [ CrossRef ]
• Patton, M.Q. Qualitative Research & Evaluation Methods: Integrating Theory and Practice ; Sage Publications: Thousand Oaks, CA, USA, 2015. [ Google Scholar ]
• Turcotte-Tremblay, A.-M.; Mc Sween-Cadieux, E. A reflection on the challenge of protecting confidentiality of participants while disseminating research results locally. BMC Med. Ethics 2018 , 19 , 5–11. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Morse, J.M. Critical analysis of strategies for determining rigor in qualitative inquiry. Qual. Health Res. 2015 , 25 , 1212–1222. [ Google Scholar ] [ CrossRef ]
• Morse, J.M.; Barrett, M.; Mayan, M.; Olson, K.; Spiers, J. Verification strategies for establishing reliability and validity in qualitative research. Int. J. Qual. Methods 2002 , 1 , 13–22. [ Google Scholar ] [ CrossRef ]
• Miles Matthew, B.; Huberman, A.M.; Saldana, J. Qualitative Data Analysis: A Methods Sourcebook ; Sage Publications: Thousand Oaks, CA, USA, 2014. [ Google Scholar ]
• Charmaz, K. Constructing Grounded Theory ; Sage: Thousand Oaks, CA, USA, 2014. [ Google Scholar ]
• Altman, D.G.; Simera, I. Using reporting guidelines effectively to ensure good reporting of health research. In Guidelines for Reporting Health Research: A User’s Manual ; Wiley: New York, NY, USA, 2014; pp. 32–40. [ Google Scholar ]
• Wu, S.; Wyant, D.C.; Fraser, M.W. Author guidelines for manuscripts reporting on qualitative research. J. Soc. Soc. Work Res. 2016 , 7 , 405–425. [ Google Scholar ] [ CrossRef ]
• Tate, R.L.; Douglas, J. Use of reporting guidelines in scientific writing: PRISMA, CONSORT, STROBE, STARD and other resources. Brain Impair. 2011 , 12 , 1–21. [ Google Scholar ] [ CrossRef ]
• Rowan, M.; Huston, P. Qualitative research articles: Information for authors and peer reviewers. CMAJ 1997 , 157 , 1442–1446. [ Google Scholar ] [ PubMed ]
• Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Int. J. Surg. 2021 , 88 , 105906. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Anderson, C. Presenting and evaluating qualitative research. Am. J. Pharm. Educ. 2010 , 74 , 7408141. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Van Tulder, M.; Furlan, A.; Bombardier, C.; Bouter, L.; Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine 2003 , 28 , 1290–1299. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Duran, R.P.; Eisenhart, M.A.; Erickson, F.D.; Grant, C.A.; Green, J.L.; Hedges, L.V.; Schneider, B. Standards for reporting on empirical social science research in AERA publications: American Educational Research Association. Educ. Res. 2006 , 35 , 33–40. [ Google Scholar ]
• Bavdekar, S.B. Enhance the value of a research paper: Choosing the right references and writing them accurately. J. Assoc. Physicians India 2016 , 64 , 66. [ Google Scholar ]
• Malterud, K. Qualitative research: Standards, challenges, and guidelines. Lancet 2001 , 358 , 483–488. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Sinha, P.; Paudel, B.; Mosimann, T.; Ahmed, H.; Kovane, G.P.; Moagi, M.; Phuti, A. Comprehensive Criteria for Reporting Qualitative Research (CCQR): Reporting Guideline for Global Health Qualitative Research Methods. Int. J. Environ. Res. Public Health 2024 , 21 , 1005. https://doi.org/10.3390/ijerph21081005

Sinha P, Paudel B, Mosimann T, Ahmed H, Kovane GP, Moagi M, Phuti A. Comprehensive Criteria for Reporting Qualitative Research (CCQR): Reporting Guideline for Global Health Qualitative Research Methods. International Journal of Environmental Research and Public Health . 2024; 21(8):1005. https://doi.org/10.3390/ijerph21081005

Sinha, Priyanka, Binita Paudel, Tamara Mosimann, Hanan Ahmed, Gaotswake Patience Kovane, Miriam Moagi, and Angel Phuti. 2024. "Comprehensive Criteria for Reporting Qualitative Research (CCQR): Reporting Guideline for Global Health Qualitative Research Methods" International Journal of Environmental Research and Public Health 21, no. 8: 1005. https://doi.org/10.3390/ijerph21081005

Article Metrics

Article access statistics, supplementary material.

ZIP-Document (ZIP, 93 KiB)

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 31 July 2024

First-line managers’ experience of guideline implementation in orthopaedic nursing and rehabilitation: a qualitative study

• Erika FJORDKVIST 1 , 2 ,
• Maria HÄLLEBERG NYMAN 1 , 2 , 3 ,
• Madeleine WINBERG 4 ,
• Eva JOELSSON-ALM 5 &
• Ann Catrine ELDH 4 , 6  

BMC Health Services Research volume  24 , Article number:  871 ( 2024 ) Cite this article

Metrics details

First-line managers have a unique role and potential in encouraging the use of evidence-based clinical practice guidelines (CPGs) and thus serve the provision of safe patient care. In acute and planned hospital care, effective yet safeguarded nursing procedures are a necessity. Little is currently known about how first-line managers engage in supporting the adoption of evidence-based nursing care and about what barriers and enablers there are for implementation of CPGs in the orthopaedic care context.

To investigate first-line managers’ experience of clinical practice guideline implementation in orthopaedic care.

This qualitative interview study included 30 first-line nursing and rehabilitation managers in 17 orthopaedic units in Sweden. A deductive content analysis, with the Ottawa Model of Implementation Leadership as a guide, was employed.

To the first-line managers, any guideline implementation required them to balance contexts, including their outer context (signified by the upper-level management and decision-makers) and their inner context, including staff and patients in their unit(s). Acting in response to these contexts, the managers described navigating the organization and its terms and conditions; using relations-, change-, and task-oriented leadership, such as involving the staff; motivating the change by emphasizing the patient benefits; and procuring resources, such as time and training. Even though they knew from past experience what worked when implementing CPGs, the first-line managers often encountered barriers within the contexts that hampered successful implementation.

Conclusions

Although first-line managers know how to effectively implement CPGs, an organization’s terms and conditions can limit their opportunities to fully do so. Organizational awareness of what supports and hinders first-line managers to offer implementation leadership can enhance opportunities to alter behaviours and conditions for the benefit of CPG implementation.

Trial registration

The study was registered as NCT04700969 with the U.S. National Institutes of Health Clinical Trials Registry on 8 January 2021.

Peer Review reports

Healthcare leaders are identified as key in facilitating evidence-based practice (EBP) [ 1 , 2 ]. A vital source of safe, evidence-based nursing procedures is evidence-based clinical practice guidelines (CPGs) [ 3 ], although the implementation of such CPGs varies greatly. It is estimated that as much as 30–40% of care delivered is not sufficiently evidence-based [ 4 , 5 ]. With no exact details on the know-do gap in nursing, knowledge implementation is known to vary [ 6 ]. The growing field of implementation science addresses barriers and enablers for EBP, shedding light on strategies to facilitate and sustain, for example, CPG adoption and adherence [ 6 , 7 ], although there is a need for further investigation of leadership components in knowledge implementation [ 2 , 8 ].

First-line managers (i.e., those who directly supervise staff providing care and who are, in turn, supervised by a superior manager) [ 9 ] are essential to both identifying and addressing the barriers and enablers for safer and better care [ 10 , 11 ]. In nursing care, they can enable nurses to use their competencies to enhance EBP [ 12 ] and to de-implement outdated healthcare procedures [ 13 ]. Although first-line managers should enact implementation leadership, what this encompasses is not fully known [ 14 ], particularly in acute care settings, with their high turnover of patients.

Orthopaedic surgical care is associated with both fundamental and advanced nursing and the rehabilitation needs of patients. In hospitals around the world, approximately 1.7 billion people are cared for every year due to musculoskeletal conditions [ 15 ]. Several orthopaedic surgical treatments have more recently been associated with shorter length of stay [ 16 , 17 ], calling for effective yet safeguarded care, including nursing care based on and reflecting EBP. Consequently, the orthopaedic context, that is, the milieu in which patients with musculoskeletal problems are cared for, is associated with numerous challenges and opportunities.

With respect to knowledge implementation, context is defined as “the environment or setting in which the proposed change is to be implemented” ([ 18 ] p.150). Context refers to a variety of dimensions, such as organizational support, organizational culture and climate, leadership, and financial resources, as well as social relations and support, physical environment, and patient preferences. Context is divided into different system levels: macro-, meso-, and micro-level. These are often grouped into descriptions as an outer or an inner context, even though the line between the two is not always clear. Context includes dimensions representing essential conditions for implementation as well as functions that drive the implementation towards its goal [ 19 ]. Little is currently known regarding what barriers and enablers in the orthopaedic context hinder and facilitate optimal nursing and rehabilitation. However, a previous study indicated a lack of leadership support, including management actions and attitudes, when it comes to CPG implementation [ 20 ]. Thus, there is a need to further address what options first-line managers have to support knowledge implementation, and how they proceed.

A descriptive, qualitative interview study [ 21 ], reported with respect to the Consolidated Criteria for Reporting Qualitative Research (COREQ) [ 22 ].

This study was performed within the Onset PrevenTIon of Orthopaedic Nursing and rehabilitation project, OPTION [ 23 ], enacted across 17 Swedish orthopaedic units. At the onset of the trial, we performed individual interviews with all of the units’ first-line managers (February–August 2021), including university, regional, and local hospitals.

Recruitment and participants

At each site, all nursing and rehabilitation managers were identified and invited to partake in an OPTION interview. The interviews were conducted by EF or MW; neither of the interviewers were involved in the subsequent OPTION intervention, and both were blind to the randomization outcome at the point of the interviews. Altogether, 35 eligible managers were contacted via email or telephone; four rehabilitation managers declined due to a heavy workload, resulting in 31 individuals consenting to participate in the study. As one interview was lost due to technical problems with the recording, the study includes interviews with 30 managers (at least one and often two managers from each site). Demographics are presented in Table  1 . The nursing managers were registered nurses (except one, who was a physiotherapist), while the rehabilitation managers were physiotherapists or occupational therapists. Participants’ experience as managers varied: some had been managers in several workplaces prior to their current position and others were in their first management role. Most of the first-line managers had full-time management positions, but two had part-time clinical practice scheduled. Yet all the first-line managers described helping out with clinical practice when they had the opportunity or if there was a staff shortage.

Data collection

Data was collected through semi-structured interviews based on a previously validated interview guide [ 24 ]. The guide indicated a narrative approach with the following areas of interest: experiences of nursing/rehabilitation care; management experience(s); development and in-house training opportunities; the organization and its structures and processes; change management and practice; improvement initiatives and performance; and outcomes and feedback. Probing questions were used sparingly to encourage further discussion. Prior to the interviews, the interview guide was discussed thoroughly between all authors to ensure a shared understanding and equivalent approach in all interviews. EF or MW held the telephone interviews, at a time chosen by each first-line manager.

The interviews lasted between 25 and 68 min (median 47 min), and all exhausted every aspect of the interview guide; they were digitally recorded in full and transcribed verbatim by an authorized secretarial service, resulting in 348 single-spaced pages of transcribed text.

Data analysis

Data consisted of the transcribed interviews which were analysed with qualitative content analysis [ 25 ], inspired by phenomenological hermeneutics [ 26 ] for a broadened understanding of managers’ lived experience vis-à-vis implementation of CPGs as a phenomenon.

First, all interviews were read in their entirety, and all authors individually compiled a short text representing their naïve understanding. The fundamentals were discussed until agreement was reached on the common content of the data set, providing a backdrop for the further analysis [ 26 ].

Second, the structured analysis applied a deductive approach [ 25 ], using categories from the Ottawa Model of Implementation Leadership (O-MILe) [ 27 ] as a matrix: core knowledge and skills; change-oriented leadership; relations-oriented leadership; and task-oriented leadership . In this phase, all interviews were reread several times; meaning units were identified and organized in accord with the matrix, using NVivo software (version 1.3) [ 28 ].

The deductive analysis progressed by means of a critical inspection of the leadership components and structures described in O-MILe. In this phase, the naïve understanding was used to anchor the analysis, investigating nuances of the process description in O-MILe and beyond. Data that did not match the O-MILe categories but corresponded to the study aim was analysed inductively, forming additional perspectives. These were either additional aspects of managing change or characteristics of the context influencing the implementation process.

Altogether the initial findings were subjected to an abductive analysis, guided by queries such as: what is shared (i.e., what is the managers’ experience of guideline implementation), by whom, in what context, and why? [ 25 ].

To conclude, a comprehensive understanding, demonstrating the final outcomes of the analysis, was formed [ 26 ].

While the analysis was primarily performed by the first author (EF) guided by the last author (ACE), all steps were repeatedly discussed between all authors and critically surveyed to compare maturing understandings of the data set at each of the phases. In alignment with criteria for qualitative research [ 22 ], trustworthiness (that is: credibility; dependability; conformability; transferability, and; authenticity) was considered in the preparation for and completion of the analysis, as well as in the organization and reporting of the results [ 29 ]. A selection of quotations illustrating the route to categories in the analysis is provided in Table  2 [ 30 ].

The results are presented in the following order: the outcomes of the structured analysis followed by the comprehensive understanding. For each section, the categories from O-MILe are  bolded, and supplementary categories are italicized.

The structured analysis’ outcomes

Directives to implement guidelines most often came from the outer context (described as the upper-level management, authorities, politicians, and sometimes individual physicians), and they were issued with no room for questioning as to when and how implementation should take place. The first-line managers described their mission as to implement the directed guidelines in the inner context , which was their unit, with the staff they supervise. The managers sometimes worried that a proposed implementation might not fit well in the inner context when they sensed that no proper risk assessment had been done to identify potential consequences for their units. The first-line managers did not always know exactly who had made the decision about a guideline prior to its implementation; they simply described the directives as coming “from above”. With no or limited opportunities to discuss the guidelines to be implemented, they still initiated the implementation by means of communication. The first-line managers described prioritizing marketing guidelines to the inner context (i.e., to their staff and unit(s)) while also figuring out how they could be translated to fit the inner context . By protecting their inner context, the first-line managers shielded the staff from petitions coming from the outer context, which could otherwise create stress and anxiety in their units. As the first-line managers indicated, the healthcare organizations were not functioning optimally at all times, making it hard to implement changes. However, this was often due to a lack of communication between units or other parts of health care. The first-line managers described adjusting actions in the process, considering whom they interacted with. They described using acceptance and adjustment and emphasizing their trust in decisions from top management in order to move forward with the implementation of guidelines in the inner context.

Adjusting actions in an implementation process included using a variety of leadership behaviours . The first-line managers emphasized the necessity of creating a plan, acting from learned experiences of what works , and they described the importance of informing and involving the staff early in the process, corresponding to relations-oriented leadership . The first-line managers described how they emphasized the patient perspective when communicating the reasons for adopting a new guideline, suggesting that this was a motivator for any change. Even with a vision of better patient care , the first-line managers indicated that they did not make enough effort to involve patients in the change process. They raised this as a limitation as, in their view, patient involvement in health care should be a higher priority. Being well rehearsed when it came to knowing the staff and the clinical routines, the first-line managers described relying on their insights into the culture and attitudes when addressing a change such as guideline implementation. To some extent, the first-line managers relied on staff members who were positive towards change in general to be facilitators when new routines or specific changes were introduced. However, there was a risk that the new routines may be dropped should those positive co-workers leave. If the professionals displayed a negative attitude towards a particular change or to change in general, the first-line manager would provide information repeatedly and sometimes focus more on the benefits for the staff over the benefits for the patients. The first-line managers suggested that they would give room to negative voices regarding a change or a new routine as a part of their strategy. Resistance had to be aired in order to be addressed, and they hoped that co-workers who were negative towards knowledge implementation would eventually accept the change.

The first-line managers also addressed  task-oriented leadership components of implementation, such as procuring resources and providing education and information to their staff. The first-line managers’ priorities were described as proactive and deliberate, yet they had to both act and react to support change . Some actions necessarily were more of reactions, initiated by the context. For example, high staff turnover would create the need to focus on introducing new staff members to basic care procedures rather than on purposeful actions to implement new routines. The managers expressed it as starting over repeatedly and never getting a chance to evolve.

The managers described the changes associated with the expected CPG implementation in positive terms to their staff, hoping that the staff would then be more positive towards implementing the new or altered guideline. Yet they also described having to be certain and consistent regarding proposed changes, making it clear to the staff that these were not negotiable. Both their positive attitudes towards and firmness regarding the changes signified  change-oriented leadership . The first-line managers made efforts to reach inter-professional consensus regarding new routines, but due to context restraints such as organizational structures or lack of direct authority over some professionals working outside the unit, this often meant they had to reach out to other managers in the organization in order to reach all professions. Co-workers that showed a willingness and ability to work with change processes were encouraged, even if their good intentions often fizzled out.

The managers stressed the importance of involving all staff when implementing a change, which was sometimes onerous. The first-line managers could find it challenging to manage implementation, particularly when they were short-staffed, and everyday patient care is always a priority. Even though they knew how to effectively facilitate a guideline implementation, they lacked sufficient time and staff. The first-line managers were concerned that directives to implement a CPG were sometimes issued without giving them adequate time to prepare. They also described high staff turnover as one of the context factors that presented the greatest challenge to implementation leadership, as competencies were lost when co-workers left. In their view, high staff turnover hampered their opportunities to implement improvements in care.

The comprehensive understanding

First-line managers’ role in guideline implementation is signified by balancing contexts . Their work situation incorporates both an outer and an inner context , and they act in response to both contexts, adapting to and/or making adaptations, by navigating the organization and its terms and conditions . This represented a workplace that lacked an effective command or organizational structure, with unclear directives, lack of communication, and one-way communication from the top down. The first-line managers viewed navigating the organization and its terms and conditions as part of their role, albeit a frustrating part. The organizational structure was often unfit for nursing, limiting opportunities for multi-professional communication and cooperation. The managers felt that the organization was not prioritizing nursing issues and that they therefore had less power to influence health care because they were stuck in their units and were having a hard time making their voices heard.

The first-line managers’ outer context was represented by higher-level decision-makers, both within and outside of the organization. These were somewhat inflexible and commanding, and less susceptible to dialogue; they simply distributed guidelines and directives to be implemented. In this sphere, the first-line managers had little impact and found it hard to voice any concerns about what they were expected to accomplish. Yet the first-line managers trusted that the directed guidelines were correct , which helped them justify the directives when addressing their staff. In the inner context, the managers experienced more of a relational setting, with a stronger mandate and greater influence. Here, the first-line managers acted and reacted to support change by using different leadership behaviours based on local factors and on their experiences of what had worked in the past.

The first-line managers envisioned implementing new guidelines with a view to improving patient care, although this was not transparent in the directives they got. However, trusting that the upper-level management or external authorities had the same purpose with the guideline, the first-line managers invoked this vision to promote adoption and change. Within the inner context, the first-line managers took command of the implementation agenda and used their leadership role to interact with the staff so that communication about CPGs or proposed changes was no longer one-way.

The comprehensive understanding is illustrated in Fig.  1 , with correspondence to the structured analysis (OMILe and additional elements).

Visualisation of first-line managers experience of implementing guidelines

The first-line managers’ experience of CPG implementation in an orthopaedic care context illustrates a balancing act between contexts. Nursing and rehabilitation first-line managers in orthopaedic care navigate organization and its terms and conditions and describe a variety of leadership behaviours to facilitate guideline adoption and adherence. Yet contextual factors compel them to react rather than to employ purposeful actions. The first-line managers portrayed this as a conflict between what they considered optimal routes for ideal nursing and rehabilitation care of orthopaedic patients and what is conceivable in a busy everyday management role and context. The challenges of first-line management in EBP and guideline implementation will be addressed, along with what leadership support and development is needed for implementation leadership to sustain evidence-based nursing and rehabilitation.

First-line managers are positioned between upper management and clinical practice [ 9 , 10 ]. As such, they have the potential to facilitate EBP. Through communication and by finding ways to bridge organizational barriers in order to motivate staff to see the patient outcomes of the new or altered guidelines, managers can create a shared vision of change. This requires navigating context and organizational conditions to explore the optimal routes to implementing new guidelines. Organizations often rely on the first-line managers’ ability to translate communication and visions mandated from the outer organization for staff’s acceptance of change [ 9 ]. This places the first-line managers in a central role: the progress of organizations depends heavily on their ability to make an ill-fitting guideline fit and to succeed with its implementation. Yet previous research suggests that lack of success in implementing EBPs could be explained by factors other than leadership [ 31 , 32 ], which should take some of the pressure off the first-line managers’ shoulders. At the same time, there is support for the central role of the first-line managers regarding implementation in different settings [ 2 , 8 ], leading to a perception of the importance of improving conditions for first-line managers to enable CPG implementation. In EBP implementation, leadership behaviours should be seen in the light of both the culture and the organization where the leadership is enacted [ 33 ], while at the same time leadership itself is often considered to be a part of the context [ 34 ].

The first-line managers in our study described numerous behaviours to support change originating from their previous experiences of successful practice. They also described the barriers they had identified when working with change processes. The behaviours illustrated have been suggested to be effective components in implementation leadership [ 27 ], but there is still a need for a better understanding of the timing of such behaviours as well as of how context interplays with them. In contrast to what has hitherto been suggested as managers’ proactive and targeted conduct, our findings indicate that first-line managers’ reactions in favour of guideline implementation is unconscious and intuitive. Rather than planning and defining their actions by means of an explicit understanding of implementation processes, the managers described acting in accordance with what they deemed most favourable for implementation, based on know-how and experience, as well as what they considered appropriate and possible given the context. First-line managers’ struggle to operate as facilitators for EBP implementation in an incompatible context with opposing demands [ 31 ]. Consequently, we suggest that training and supporting first-line managers in knowledge implementation recognizes their prior experiences, reinforcing what has served well, in what context, and for whom, yet re-establishing leadership strategies that might be more effective for promoting evidence-based practice.

Even with such skills, the managers described barriers that prevented them from acting in ways they thought would be right. Rather, actions more often became reactions because of the organization and its terms and conditions. Adapting to staff shortages or to high staff turnover, for example, led the first-line managers to put more efforts into certain leadership behaviours, such as procuring resources, rather than into what they actually considered more important in order to reach implementation success. This indicates that first-line managers might have unused competencies regarding EBP implementation and thus are underused as resources. Context factors have been identified as having a great impact when it comes to implementation in health care [ 34 , 35 , 36 ], and recent research highlights how first-line managers have applied situational work in order to handle the complexity of implementing changes in the public sector [ 37 ]. Knowing the importance of developing EBP while lacking sufficient resources to do so can be stressful for the first-line managers [ 38 ]. However, even though first-line managers understand the importance of their behaviours when it comes to facilitating the implementation of CPGs, they cannot always use them [ 39 ]. This indicates great potential for further support and training for first-line managers as well as a continued need for organizational support and leadership educational programmes with a focus on implementation leadership skills. With better conditions for first-line managers to enact implementation leadership, further opportunities for more evidence-based care can advance.

Methodological considerations/Study limitations

This study represents first-line managers engaged in the OPTION trial, and thus a large share of the orthopaedic care first-line managers across Sweden. Yet it still represents only a minor part of all nursing and rehabilitation managers across Sweden and globally. While the experiences shared are similar to those described in previous studies (such as [ 31 ] and [ 32 ]), this study has identified novel perspectives contributing to the growing understanding of first-line managers’ situation, conditions, and needs. These contemporary findings could be used to inform the development of validated leadership surveys in order to reach a broader population of first-line nursing and/or rehabilitation managers. To date, there are few such instruments for measuring implementation leadership available in Swedish and/or for the Swedish healthcare context. The Implementation Leadership Scale (ILS) [ 40 ] incorporates many, if not all, aspects identified in this study, but further development would be helpful.

This study illustrates first-line managers’ experience of implementing CPGs in orthopaedic nursing and rehabilitation care – a process in which they balance the outer and inner contexts. Their behaviours are intended to move forward with CPG implementation, and their position provides opportunities to enhance EBP, even if their full use of leadership is impaired by the organization and its limited resources.

The first-line managers described leadership behaviours that had previously proved effective for implementation leadership, here employed as a result of their navigation of their contexts and along with the everyday conditions of acute care. While the first-line managers meant to proceed with the CPG implementation, they balanced the often sub-optimal conditions for the benefit of adoption of and adherence to clinical practice guidelines. Further opportunities to plan and perform deliberate behaviours are needed, with strategies aiding first-line managers to facilitate implementation.

Availability of data and materials

The data that support the findings of this study are available on request from the corresponding author (EF), upon reasonable request. The data are not publicly available as it contains information that could compromise the privacy of research participants.

Abbreviations

Clinical Practice Guideline

Evidence Based Practice

The Ottawa Model of Implementation Leadership

Onset PrevenTIon of urinary retention in Orthopaedic Nursing and rehabilitation

Masic I, Miokovic M, Muhamedagic B. Evidence based medicine - new approaches and challenges. Acta informatica medica : AIM : journal of the Society for Medical Informatics of Bosnia & Herzegovina : casopis Drustva za medicinsku informatiku BiH. 2008;16:219–25. https://doi.org/10.5455/aim.2008.16.219-225 .

Article   Google Scholar  

Birken S, Clary A, Tabriz AA, Turner K, Meza R, Zizzi A, et al. Middle managers’ role in implementing evidence-based practices in healthcare: a systematic review. Implement Sci. 2018;13:149. https://doi.org/10.1186/s13012-018-0843-5 .

Article   PubMed   PubMed Central   Google Scholar  

Wallin L. Clinical practice guidelines in nursing: a straightforward route to evidence-based practice? AWHONN Lifelines. 2005;9:248–51. https://doi.org/10.1177/1091592305279125 .

Article   PubMed   Google Scholar  

Grol R. Successes and failures in the implementation of evidence-based guidelines for clinical practice. Med Care. 2001;39:Ii46–54. https://doi.org/10.1097/00005650-200108002-00003

Braithwaite J, Glasziou P, Westbrook J. The three numbers you need to know about healthcare: the 60–30-10 Challenge. BMC Med. 2020;18:102. https://doi.org/10.1186/s12916-020-01563-4 .

Forsman H, Rudman A, Gustavsson P, Ehrenberg A, Wallin L. Nurses’ research utilization two years after graduation—a national survey of associated individual, organizational, and educational factors. Implement Sci. 2012;7:46. https://doi.org/10.1186/1748-5908-7-46 .

Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3:32. https://doi.org/10.1186/s40359-015-0089-9 .

Gifford WA, Squires JE, Angus DE, Ashley LA, Brosseau L, Craik JM, et al. Managerial leadership for research use in nursing and allied health care professions: a systematic review. Implement Sci. 2018;13:127. https://doi.org/10.1186/s13012-018-0817-7 .

Gutberg J, Berta W. Understanding middle managers’ influence in implementing patient safety culture. BMC Health Serv Res. 2017;17:582. https://doi.org/10.1186/s12913-017-2533-4 .

Clavijo-Chamorro MZ, Romero-Zarallo G, Gómez-Luque A, López-Espuela F, Sanz-Martos S, López-Medina IM. Leadership as a Facilitator of Evidence Implementation by Nurse Managers: A Metasynthesis. West J Nurs Res. 2021;44:567–81. https://doi.org/10.1177/01939459211004905 .

Birken SA, Lee S-YD, Weiner BJ. Uncovering middle managers' role in healthcare innovation implementation. Implement Sci. 2012;7:28. https://doi.org/10.1186/1748-5908-7-28

Orton ML, Andersson Å, Wallin L, Forsman H, Eldh AC. Nursing management matters for registered nurses with a PhD working in clinical practice. J Nurs Manag. 2019;27:955–62. https://doi.org/10.1111/jonm.12750 .

Upvall MJ, Bourgault AM. De-implementation: A concept analysis. Nurs Forum. 2018. https://doi.org/10.1111/nuf.12256 .

Castiglione SA. Implementation leadership: A concept analysis. J Nurs Manag. 2020;28:94–101. https://doi.org/10.1111/jonm.12899 .

Cieza A, Causey K, Kamenov K, Hanson SW, Chatterji S, Vos T. Global estimates of the need for rehabilitation based on the Global Burden of Disease study 2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet. 2021;396:2006–17. https://doi.org/10.1016/S0140-6736(20)32340-0 .

Gholson JJ, Noiseux N, Otero J, Gao Y, Shah A. Patient Factors Systematically Influence Hospital Length of Stay in Common Orthopaedic Procedures. Iowa Orthop J. 2017;37:233–7.

CAS   PubMed   PubMed Central   Google Scholar  

Nordström P, Bergman J, Ballin M, Nordström A. Trends in Hip Fracture Incidence, Length of Hospital Stay, and 30-Day Mortality in Sweden from 1998–2017: A Nationwide Cohort Study. Calcif Tissue Int. 2022;111:21–8. https://doi.org/10.1007/s00223-022-00954-4 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care. 1998;7:149. https://doi.org/10.1136/qshc.7.3.149 .

Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. 2019;19:189. https://doi.org/10.1186/s12913-019-4015-3 .

Hälleberg Nyman M, Forsman H, Wallin L, Ostaszkiewicz J, Hommel A, Eldh AC. Promoting evidence-based urinary incontinence management in acute nursing and rehabilitation care—A process evaluation of an implementation intervention in the orthopaedic context. J Eval Clin Pract. 2019;25:282–9. https://doi.org/10.1111/jep.12879 .

Patton MQ. Qualitative research & evaluation methods. 3 ed. London: London : SAGE; 2002.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19:349–57. https://doi.org/10.1093/intqhc/mzm042 .

Eldh AC, Joelsson-Alm E, Wretenberg P, Hälleberg-Nyman M. Onset PrevenTIon of urinary retention in Orthopaedic Nursing and rehabilitation, OPTION—a study protocol for a randomised trial by a multi-professional facilitator team and their first-line managers’ implementation strategy. Implement Sci. 2021;16:65. https://doi.org/10.1186/s13012-021-01135-x .

Rycroft-Malone J, Seers K, Eldh AC, Cox K, Crichton N, Harvey G, et al. A realist process evaluation within the Facilitating Implementation of Research Evidence (FIRE) cluster randomised controlled international trial: an exemplar. Implement Sci. 2018;13:138. https://doi.org/10.1186/s13012-018-0811-0 .

Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62:107–15. https://doi.org/10.1111/j.1365-2648.2007.04569.x .

Lindseth A, Norberg A. A phenomenological hermeneutical method for researching lived experience. Scand J Caring Sci. 2004;18:145–53. https://doi.org/10.1111/j.1471-6712.2004.00258.x .

Gifford W, Graham ID, Ehrhart MG, Davies BL, Aarons GA. Ottawa Model of Implementation Leadership and Implementation Leadership Scale: mapping concepts for developing and evaluating theory-based leadership interventions. J Healthc Leadersh. 2017;9:15–23. https://doi.org/10.2147/jhl.S125558 .

QSR International Pty Ltd. NVivo (Released in March 2020) Denver, CO: Lumivero; 2020

Elo S, Kääriäinen M, Kanste O, Pölkki T, Utriainen K, Kyngäs H. Qualitative Content Analysis: A Focus on Trustworthiness. SAGE Open. 2014;4:1–10. https://doi.org/10.1177/2158244014522633 .

Eldh AC, Årestedt L, Berterö C. Quotations in qualitative studies: reflections on constituents, custom and purpose. Int J Qual Methods. 2020;19:1–6. https://doi.org/10.1177/1609406920969268 .

Wilkinson JE, Nutley SM, Davies HTO. An Exploration of the Roles of Nurse Managers in Evidence-Based Practice Implementation. Worldviews Evid Based Nurs. 2011;8:236–46. https://doi.org/10.1111/j.1741-6787.2011.00225.x .

Mosson R, Hasson H, Wallin L, von Thiele SU. Exploring the Role of Line Managers in Implementing Evidence-Based Practice in Social Services and Older People Care. Br J Soc Work. 2017;47:542–60. https://doi.org/10.1093/bjsw/bcw004 .

Sandström B, Borglin G, Nilsson R, Willman A. Promoting the Implementation of Evidence-Based Practice: A Literature Review Focusing on the Role of Nursing Leadership. Worldviews Evid Based Nurs. 2011;8:212–23. https://doi.org/10.1111/j.1741-6787.2011.00216.x .

Nilsen P, Birken SA. Handbook on implementation science. Cheltenham, UK: Edward Elgar Publishing; 2020.

Book   Google Scholar  

Granberg A, Matérne M, Lundqvist L-O, Duberg A. Navigating change – managers’ experience of implementation processes in disability health care: a qualitative study. BMC Health Serv Res. 2021;21:571. https://doi.org/10.1186/s12913-021-06570-6 .

Correa VC, Lugo-Agudelo LH, Aguirre-Acevedo DC, Contreras JAP, Borrero AMP, Patiño-Lugo DF, et al. Individual, health system, and contextual barriers and facilitators for the implementation of clinical practice guidelines: a systematic metareview. Health Res Policy Syst. 2020;18:74. https://doi.org/10.1186/s12961-020-00588-8 .

Klemsdal L, Andreassen TA, Breit E. Resisting or Facilitating Change? How Street-Level Managers’ Situational Work Contributes to the Implementation of Public Reforms. Journal of Public Administration Research and Theory. 2022;32:736–49. https://doi.org/10.1093/jopart/muac004 .

Abelsson T, Karlsson AK, Morténius H. A Feeling of Ambiguity: A Qualitative Content Analysis of Managers’ Experiences of Evidence-Based Practice in Swedish Primary Care. Journal of healthcare leadership. 2022;14:143–53. https://doi.org/10.2147/jhl.S371643 .

Eldh AC, Olai L, Jönsson B, Wallin L, Denti L, Elf M. Supporting first-line managers in implementing oral care guidelines in nursing homes. Nord J Nurs Res. 2018;38:87–95. https://doi.org/10.1177/2057158517713379 .

Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9:45. https://doi.org/10.1186/1748-5908-9-45 .

Download references

Acknowledgements

The authors would like to thank all first line managers who participated in the study for sharing their experiences.

Open access funding provided by Örebro University. This study was as a part of the OPTION-trial [ 23 ] funded by FORTE (ID STYA-2020/0002) and Region Örebro county; the funding bodies have neither been involved in the framing of the study nor the analysis/reporting of findings.

Author information

Authors and affiliations.

Faculty of Medicine and Health, School of Health Sciences, Örebro University, 701 82, Örebro, Sweden

Erika FJORDKVIST & Maria HÄLLEBERG NYMAN

Department of Orthopaedics, Faculty of Medicine and Health, Örebro University, 701 82, Örebro, Sweden

Faculty of Medicine and Health, University Health Care Research Center, Örebro University, 701 82, Örebro, Sweden

Maria HÄLLEBERG NYMAN

Faculty of Medicine and Health Sciences, Department of Health, Medicine and Caring Sciences, Linköping University, 581 83, Linköping, Sweden

Madeleine WINBERG & Ann Catrine ELDH

Department of Clinical Science and Education, 118 83Stockholm, , Södersjukhuset, Karolinska Institutet, Sweden

Eva JOELSSON-ALM

Department of Public Health and Caring Sciences, Uppsala University, Box 564, 751 22, Uppsala, Sweden

Ann Catrine ELDH

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to designing the study. EF and MW performed the interviews. EF performed the analysis with support of ACE, while MHN, MW, and EJA repeatedly reflected on the analysis process and the emerging findings. EF authored the manuscript in co-operation with ACE and the other authors. All authors have contributed to the manuscript and given approval of the final version prior to submission.

Corresponding author

Correspondence to Erika FJORDKVIST .

Ethics declarations

Ethics approval and consent to participate.

The OPTION trial [ 23 ] was approved by the Swedish ethical review authority (ID 2020–06140), including the interviews employed here. Due to covid-19 restrictions (with reduced opportunities to meet face-to-face), amendments for this data collection were required and approved (ID 2021–02434 and ID 2021 03755). All participants provided written informed consent prior to the interviews.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

FJORDKVIST, E., HÄLLEBERG NYMAN, M., WINBERG, M. et al. First-line managers’ experience of guideline implementation in orthopaedic nursing and rehabilitation: a qualitative study. BMC Health Serv Res 24 , 871 (2024). https://doi.org/10.1186/s12913-024-11353-w

Download citation

Received : 09 March 2024

Accepted : 24 July 2024

Published : 31 July 2024

DOI : https://doi.org/10.1186/s12913-024-11353-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Clinical practice guideline
• Evidence-based health care
• Evidence-based nursing
• Evidence-based practice
• First-line manager
• Implementation
• Implementation leadership

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

This paper is in the following e-collection/theme issue:

Published on 30.7.2024 in Vol 26 (2024)

This is a member publication of University of Oxford (Jisc)

The Acceptability, Engagement, and Feasibility of Mental Health Apps for Marginalized and Underserved Young People: Systematic Review and Qualitative Study

Authors of this article:

• Holly Alice Bear 1 , DPhil   ; 
• Lara Ayala Nunes 1 , DPhil   ; 
• Giovanni Ramos 2 , PhD   ; 
• Tanya Manchanda 1 , MEd   ; 
• Blossom Fernandes 1 , PhD   ; 
• Sophia Chabursky 3 , MSc   ; 
• Sabine Walper 3 , DPhil   ; 
• Edward Watkins 4 , DPhil   ; 
• Mina Fazel 1 , DM  

1 Department of Psychiatry, University of Oxford, Oxford, United Kingdom

2 Department of Psychological Science, University of California, Irvine, CA, United States

3 German Youth Institute, Munich, Germany

4 School of Psychology, University of Exeter, Exeter, United Kingdom

Corresponding Author:

Holly Alice Bear, DPhil

Department of Psychiatry

University of Oxford

Warneford Hospital

Warneford Lane

Oxford, OX3 7JX

United Kingdom

Phone: 44 01865 6182

Email: [email protected]

Background: Smartphone apps may provide an opportunity to deliver mental health resources and interventions in a scalable and cost-effective manner. However, young people from marginalized and underserved groups face numerous and unique challenges to accessing, engaging with, and benefiting from these apps.

Objective: This study aims to better understand the acceptability (ie, perceived usefulness and satisfaction with an app) and feasibility (ie, the extent to which an app was successfully used) of mental health apps for underserved young people. A secondary aim was to establish whether adaptations can be made to increase the accessibility and inclusivity of apps for these groups.

Methods: We conducted 2 sequential studies, consisting of a systematic literature review of mental health apps for underserved populations followed by a qualitative study with underserved young male participants (n=20; age: mean 19). Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, an electronic search of 5 databases was conducted in 2021. The search yielded 18,687 results, of which 14 articles met the eligibility criteria.

Results: The included studies comprised a range of groups, including those affected by homelessness, having physical health conditions, living in low- and middle-income countries, and those with sexual and gender minority identities. Establishing and maintaining user engagement was a pervasive challenge across mental health apps and populations, and dropout was a reported problem among nearly all the included studies. Positive subjective reports of usability, satisfaction, and acceptability were insufficient to determine users’ objective engagement.

Conclusions: Despite the significant amount of funding directed to the development of mental health apps, juxtaposed with only limited empirical evidence to support their effectiveness, few apps have been deliberately developed or adapted to meet the heterogeneous needs of marginalized and underserved young people. Before mental health apps are scaled up, a greater understanding is needed of the types of services that more at-risk young people and those in limited-resource settings prefer (eg, standard vs digital) followed by more rigorous and consistent demonstrations of acceptability, effectiveness, and cost-effectiveness. Adopting an iterative participatory approach by involving young people in the development and evaluation process is an essential step in enhancing the adoption of any intervention, including apps, in “real-world” settings and will support future implementation and sustainability efforts to ensure that marginalized and underserved groups are reached.

Trial Registration: PROSPERO CRD42021254241; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=254241

Introduction

Addressing health inequities is a key challenge for the mental health field, especially when trying to ensure that interventions and services are accessible and acceptable for all populations. Nearly 50% of lifelong mental health disorders begin by the age of 14 years, and by the age of 24 years, 75% of mental health disorders have begun [ 1 ]. Given the frequent onset of mental health problems during youth, here defined as the period between 15 and 24 years, special attention must be paid to older adolescents, including those from underserved and marginalized minority groups and socioeconomically deprived backgrounds [ 2 ]. In these groups, common barriers to accessing mental health services can be exacerbated (eg, poor mental health literacy, lack of knowledge about where to seek help, negative attitudes toward professional help seeking, embarrassment, preference for self-reliance, fear of stigma, and confidentiality concerns), and additional barriers exist (eg, reliance on informal supports, shame, lack of housing or money, and therapist factors, such as different race and level of experience), creating increased risk of untreated mental health problems and thus poorer mental health outcomes [ 3 - 7 ]. In this study, marginalized and underserved populations are defined as those with higher prevalence of mental health problems and lower rates of help seeking, such as racially and ethnically minoritized individuals, rural and remote communities, financially deprived groups, individuals experiencing homelessness, refugee and migrant populations, and sexual and gender minority groups [ 8 - 10 ] and those with lower inclusion in mental health intervention research than one would expect from population estimates [ 11 ], respectively. These groups are exposed to risk factors for poor mental health and experience disparities in mental health care, including lower access to care, poorer treatment quality, and limited engagement in treatment [ 5 , 10 , 12 ].

Smartphone apps could offer an opportunity to deliver mental health and well-being resources and interventions in a scalable, cost-effective, and potentially personalized manner, particularly for those who experience the greatest barriers to accessing health care [ 13 , 14 ]. Given that smartphone ownership is nearly ubiquitous among young people in high-income nations and increasingly across lower-resource settings, apps have the potential to address some of the accessibility issues in service provision for young people’s mental health [ 15 ]. Young people are more digitally connected (ie, they are more likely to own smartphones and spend more time on the web) and more likely to seek health information on the web than older generations, meaning that app-based interventions may be particularly well-suited for this population [ 16 , 17 ].

Not surprisingly, the number of mental health apps being developed, both commercially and in academic research programs, has expanded rapidly, outpacing scientific evaluations of their effectiveness [ 18 , 19 ]. Emerging evidence suggests that some apps may produce significant symptom improvement across multiple outcomes, compared with waitlist or control conditions [ 20 - 22 ]. Despite promise, empirical research often fails to translate into meaningful and sustained implementation in “real-world” settings [ 23 , 24 ]. Research has focused primarily on efficacy under ideal “laboratory” conditions rather than effectiveness in real-world settings [ 25 ]. Therefore, at present, most apps, especially those available to the public, lack strong empirical support [ 19 ]. The acceptability (ie, perception that a given technology is useful, agreeable, palatable, or satisfactory); accessibility (ie, the technology being easy to obtain or use); engagement (ie, initial adoption and sustained interactions with the technology, including the level of app use, intervention adherence, and premature dropout); and feasibility (ie, the actual fit, utility, or suitability and the extent to which the technology can be successfully used or conducted within a given context) of apps for marginalized and underserved groups remain poorly understood [ 25 - 28 ]. Although mental health apps may provide a possible solution, marginalized and underserved groups of young people face unique challenges to engage with and benefit from these interventions (eg, intervention cost, content that is not culturally attuned and lack of reliable access to the internet) and are typically underrepresented in intervention research [ 25 , 27 ]. Although increased access is often seen as a major benefit of digital mental health interventions, issues related to the “digital divide” describe the phenomena that technology is not equally available to all social groups due to economic, social, or cultural inequalities and is a potential ethical concern [ 29 ]. Furthermore, underrepresentation in the intervention development process potentially reinforces structural inequalities by limiting the availability of products that are culturally accessible, inclusive, and effective or by skewing the product features to attract young people from more advantaged backgrounds [ 9 , 14 ]. Therefore, considering diversity, equity, and inclusion issues at the outset of health care research as well as within app evaluation is essential to prevent the perpetuation of existing inequities [ 27 ].

To date, little has been published on the attempts to create new or adapt existing app interventions to meet the heterogeneous needs of diverse groups of young people [ 9 , 14 ]. Moving forward, careful consideration is needed to ensure optimal leveraging of all mental health intervention research, including that of mental health apps, to increase health equity while also ensuring that innovations do not inadvertently widen the digital divide and exacerbate health inequalities [ 14 ]. Although the efficacy of many mental health apps remains unclear, future attempts to translate findings for underserved populations will need to ensure that all apps are developed with enough flexibility to fit a wider range of user needs and preferences. To achieve this goal, research is needed to assess the acceptability and feasibility of mental health apps for underserved young people to ensure that they are not further excluded from research and to advance toward mental health provision that meets their needs.

This Research

We conducted two sequential studies: (1) a systematic literature review and (2) a qualitative study with a targeted sample of young people who often are underrepresented in research, with limited access to health care and socioeconomic deprivation. The overarching aim of these combined studies was to better understand whether mental health apps are feasible and acceptable to underserved young people. A secondary aim was to determine which adaptations might enable accessibility of and effective engagement with mental health apps for these groups.

The research questions of interest were as follows: (1) On the basis of the existing literature, are mental health apps acceptable, feasible, and engaging for marginalized and underserved young people and how have these constructs been measured? (2) On the basis of the qualitative study, what are young people’s experiences of using a mental well-being app, including its acceptability, feasibility, and level of engagement? (3) On the basis of both studies, are apps an acceptable, feasible, and engaging intervention approach to meet the specific needs of underserved young people? What adaptations can be made to ensure that mental health apps are accessible and inclusive for these groups?

To fully address our research questions, we adopted a 2-pronged approach. First, we conducted a systematic review of the literature to better understand the acceptability and feasibility of mental health apps for underserved young people. To explore the findings of the systematic review in greater depth and to provide further insights from multiple perspectives, we next conducted a qualitative study with young men not in education, employment, or training (NEET) in the United Kingdom and Spain and asylum seekers and refugees in Germany. The interviews were conducted between August 2021 and February 2022.

Systematic Review

Literature search and search strategy.

An electronic literature search was performed in English on the following databases from January 2009 to May 2021: Cochrane Library, Embase, MEDLINE, and PsycINFO. We used key search terms relating to (1) underserved young people; (2) mental health mobile apps; and (3) acceptability, feasibility, and engagement. The search strategy was guided by similar reviews exploring digital mental health interventions for young people [ 25 , 30 ], and the terms for apps were derived from Cochrane reviews [ 31 , 32 ]. An updated search was conducted in September 2023. The full search strategies are available in Multimedia Appendix 1 .

Inclusion and Exclusion Criteria

Screened articles were included if (1) the study targeted marginalized and underserved young people with a mean age of 15 to 25 years, including individuals who were NEET, apprentices, teenage parents, members of minoritized racial or ethnic groups, members of sexual and gender minoritized groups, residents of low- and middle-income countries (LMICs), experiencing homelessness, socioeconomic deprivation, refugees or asylum seekers, and migrants; individuals with substance use disorders; those under state or statutory care; people with physical disabilities; and individuals involved in the criminal justice system or incarcerated; (2) the intervention was a “native” mobile app (ie, not on a web browser), whose primary aim was to promote well-being, prevent mental health problems, or treat existing mental health problems, delivered as a stand-alone intervention or as an adjunct to therapist-assisted interventions; (3) the primary outcome was a measure of mental health or well-being; and (4) the study reported a measure of user acceptability or feasibility.

Articles were excluded if (1) the mean age of participants was outside of the 15 to 25 years range; (2) the intervention was not a mobile app, that is, other digital interventions, including teletherapy (eg, therapy delivered by phone, SMS text messages, video platforms, or PCs); and (3) there was no measure of acceptability or feasibility. Gray literature was not included in the search.

Study Selection

In accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [ 33 ], the flowchart presented in Figure 1 provides step-by-step details of the study selection procedure. The PRISMA checklist is provided in Multimedia Appendix 1 . The search strategy identified 11,539 citations after deduplication. After an initial screening of the titles, which resulted in the exclusion of 10,061 (87.19%) irrelevant entries, the abstracts of 1478 (12.81%) studies were screened by 4 members of the review team (LAN, HAB, BF, and TM). The identified 176 (11.91%) full texts were then screened by LAN. In this final stage, 11 (6.2%) studies, corresponding to 9 interventions, were identified for inclusion in the review, with 8 (73%) found through the electronic search and 3 (27%) through manual searches of the reference lists of relevant articles. The updated search identified a further 7148 citations after deduplication, of which 3 met inclusion criteria and were included in the review.

Data Extraction

Data were extracted by 1 reviewer (LAN, HAB, or BF) and reviewed for accuracy and completeness by a second reviewer. After verifying all the extracted data, discrepancies were resolved by discussion or adjudication by another author (MF). Extracted data included information on study characteristics (ie, authors, publication year, country, study design, and study population); intervention characteristics (ie, characteristics of the technology, app name, therapeutic modality, and intervention outcomes); and feasibility and acceptability.

Quality Assessment

We used the Mixed Methods Appraisal Tool (MMAT; version 2018) to assess the methodological quality of the included studies [ 34 ]. MMAT was developed by combining the core relevant methodological criteria found in different well-known and widely used qualitative and quantitative critical appraisal tools. The MMAT consists of 2 screening questions applicable to all types of study design and a further 5 questions applicable to specific study designs. Responses were rated on a categorical scale as “no,” “unclear,” or “yes” to any of the methodological quality criteria. Quality assessments were made by 1 reviewer (TM). We did not exclude any studies based on quality assessment scores.

Data Synthesis and Analysis

The extracted data were collated and summarized to produce a narrative summary of the study; sample characteristics; and acceptability, feasibility, and engagement outcomes. A codebook approach was used to code and synthesize implementation data from all available sources according to the outcome categories [ 35 ].

Qualitative Study

Study context.

The Emotional Competence for Well-Being (ECoWeB) cohort multiple randomized controlled trial involved a longitudinal prospective cohort to examine the well-being, mental health, and emotional competence in individuals aged 16 to 22 years across 12 months. The experimental arm was an emotional competence self-help app (ie, MyMoodCoach).

Intervention

MyMoodCoach was designed to test if an app could improve different processes affecting mental well-being, including, but not limited to, improving emotion regulation by reducing maladaptive strategies such as worry and rumination and replacing them with constructive alternatives and problem-solving and enhancing emotional knowledge and perception through psychoeducation and learning tasks. The app was designed for young people broadly and was not targeted at a specific population. Full details of the ECoWeB trial are reported in the study protocol [ 36 ].

Participant Recruitment

In parallel to the trial, we additionally recruited young people to understand the views and experiences of those underrepresented in the study sample (and most other app-based studies). As there was an overrecruitment of White, university-educated female participants in the study sample, we decided to only recruit male participants and to focus on 2 specific groups: NEET men and migrant populations (including both voluntary and forced migrants). In the United Kingdom and Spain, recruitment was conducted through a variety of channels, including Twitter, targeted adverts on social media (eg, Instagram and Facebook), newsletters sent to youth and practitioner networks, outreach to third sector organizations and mental health support groups, and advertisements placed on university and charity websites. In Germany, participants were recruited directly from refugee homes, integration courses, migration services, and youth centers.

Inclusion criteria required the participants to be (1) aged between 16 and 22 years; (2) able to speak and read English, Spanish, or German; (3) male; (4) NEET (United Kingdom), NEET or migrant (Spain), or asylum seeker or refugee (Germany); (5) having access to a smartphone with the minimal technological specifications necessary for the app (ie, iOS 9 or later or Android 8.0 or later); and (6) having access to the internet via mobile data or Wi-Fi. The exclusion criteria were having current suicidal ideation, psychosis, or bipolar disorder.

The research team was led by the principal investigator, MF, professor of Adolescent Psychiatry and Consultant in Children’s Psychological Medicine, with particular expertise in the mental health needs of refugee populations. EW, professor of Experimental and Applied Clinical Psychology, Chartered Clinical Psychologist, provided additional supervisory input and guidance as a leading expert in the field of child and adolescent mental health research. SW, professor of Education with a focus on youth research and director at German Youth Institute, provided supervisory oversight of the interviews in Germany. HAB and LAN were postdoctoral researchers at the time the research was conducted, both of whom have several years of experience conducting qualitative research with young people and expertise in analyzing qualitative data. SC, researcher at the German Youth Institute, conducted and analyzed the interviews in Germany and has experience conducting qualitative research with refugee populations. The authors had no relationship with any of the participants.

In the United Kingdom and Spain, participation involved (1) short questionnaires about mood and feelings and demographic questions such as age, gender, race, ethnicity, educational attainment, and country of origin; (2) downloading MyMoodCoach and using it for 4 weeks; and (3) completing a follow-up interview. Interviews were conducted by HAB and LAN via MS Teams (Microsoft, Corp) and lasted approximately 45 minutes. A similar procedure was followed in conducting the interviews with refugees in Germany, but given the likely language barriers in navigating the app, an additional, earlier interview was included 2 to 3 weeks after the initial instructions had been sent to explore if features of the app were understood and to clarify questions. After 2 further weeks, a second interview was conducted. One interview was conducted in person, while all the others were conducted over the telephone.

Interviews followed a semistructured schedule based on a taxonomy of implementation constructs [ 26 ]. Topics included self-reported app use, satisfaction and feedback about content, usability, and acceptability ( Multimedia Appendix 2 ). In Germany, the interview topic guide was translated and adapted to the target group. As not to stigmatize the young men, the wording “refugee” was avoided, and instead, when referring to the target group, the wording “young man such as yourself” was used.

Ethical Considerations

In the United Kingdom, ethical approval was granted by the University of Exeter Research Ethics Committee (eCLESPsy000048 v10.0); in Spain, by the Jaime I University Research Ethics Committee (CD/93/2021); and in Germany, by the Ethics Committee of the Faculty of Medicine at Ludwig-Maximilians University Munich (PNr 19-0468/19-0315). Cognizant of the ethical and practical implications of conducting research with underserved populations, we conducted the interviews in private, quiet spaces and in a friendly and reassuring manner to ensure that participants felt comfortable and safe.

With prior consent, all interviews were audio recorded. Participants were reimbursed with a shopping voucher of up to €50 or £50 (US $64) for their time and app use.

Analytic Strategy

Interviews were transcribed verbatim, and the transcripts were assigned a unique pseudonym to anonymize participants. The interviews were analyzed using a combination of theory- and data-driven analysis techniques, consisting primarily of deductive, theory-driven thematic analysis [ 37 ]. Analysis of the transcripts in the United Kingdom and Spain was conducted by HAB, using NVivo (version 11; Lumivero). Following a similar procedure, the interviews in Germany were transcribed and analyzed using the coding software MAXQDA (VERBI GmbH) by SC.

Initial familiarization with the data was achieved through the transcription process and iterative rereading of the interviews. Analysis was carried out through a recursive process of open coding, when concepts were named and their properties and dimensions identified, followed by axial coding, when links and associations were drawn between codes. Codes were based on language used by the young people and were applied to each new unit of meaning. Data extracts were multiply coded when appropriate, as were contradictory and minority features of the data. The data set was iteratively reviewed, and codes were systematically applied to the whole data set until a finalized coding manual was established. Codes were organized into potential themes using thematic maps and tables. The development of the coding manual was iteratively reviewed and refined through discussion with all authors throughout the analysis process to ensure the reliability and rigor of the process and results.

Approach to Inquiry

Analysis was conducted from a critical realist perspective to provide a more nuanced understanding and explanation of participants’ experiences [ 38 ]. This position assumes that although participants’ accounts provide important insights about the real world, these accounts are not objective and represent an interpretation of reality [ 39 ]. These data require interpretation and explanation by the researcher, who also has their own perspectives on the world, to better understand the underlying mechanisms and processes and, in turn, make recommendations for practice [ 40 ].

Study Characteristics

Characteristics of the 14 included studies, examining 12 interventions, are presented in Table 1 . The included studies were published in the United States (n=3), Australia (n=3), Switzerland (n=2), India (n=2), South Korea (n=1) and Germany (n=1). As for the type of intervention, 7 apps were stand-alone, and the rest (n=5) were delivered in combination with other forms of professional support. The interventions (n=12) were targeted at apprentices and the unemployed (n=3), homeless populations (n=2), those with physical health conditions (n=2), sexual and gender minoritized individuals (n=3), those residing in LMICs (n=1) and those with co-occurring autism spectrum disorders (n=1). Most of the interventions had been co-designed with young people (11/12, 92%). The study sample size ranged from 9 to 877, with 4 of the 14 studies having a sample size of >200 participants.

a CBT: cognitive behavioral therapy.

b CF: cystic fibrosis.

c JIA: juvenile idiopathic arthritis.

d T1D: type 1 diabetes.

e RCT: randomized controlled trial.

f iCBT: internet-based cognitive behavioral therapy.

g Mobile phones were preloaded with several apps designed to promote mental health wellness and provide real-time resources. Pocket Helper was 1 app specifically designed for this study.

h Studies identified in the updated search.

Study Quality

The included studies varied in their methodological quality ( Multimedia Appendix 3 ) [ 41 - 54 ]. Most (13/14, 93%) were judged to contain possible limitations in at least 1 criterion. All studies but 1 (13/14, 93%) were clear in their description of study participants or the process of recruiting a sample representative of the population of interest. All studies addressed the research question using collected data and reported in some way on feasibility and acceptability. Most studies (12/14, 86%) effectively used appropriate qualitative, quantitative, or mixed methods to answer their research question. However, many studies (6/14, 43%) did not have a sufficiently large sample to warrant definitive conclusions about the feasibility and acceptability of the intervention studied.

Acceptability and Feasibility

User acceptability was found to be high across all included studies, with participants rating the apps positively and reporting high satisfaction with the content of the interventions ( Table 2 ). In studies where participants were asked to indicate if they would recommend the study to someone else, the vast majority reported they would [ 42 , 44 - 46 , 51 ]. It is notable that despite many users across the studies reporting high satisfaction levels and being willing to recommend the apps, they themselves did not intend to continue using the apps (ie, low predicted engagement), as they did not find them useful or relevant for their own circumstances [ 41 , 42 , 51 ]. It is also important to note that many studies incentivized participation with payments, prize draws, and vouchers, and this may have influenced acceptability ratings and engagement [ 41 - 43 , 46 , 49 - 51 , 53 , 54 ]. Furthermore, in the only study that asked if participants would pay to use apps, most were unwilling to do so [ 42 ].

a Studies identified in the updated search.

Regarding co-design strategies used in these studies, early stakeholder consultation and service provider focus groups were conducted in the early development phase of POD Adventures, a gamified intervention for people with or at risk of anxiety, depression, and conduct difficulties in India; the results highlighted that “self-help” was not a culturally congruent concept for most Indian adolescents [ 47 ]. This early feedback was important as it revealed the norms around seeking or receiving direct instruction from parents, teachers, and other elders and that support from a counselor might be necessary to ensure acceptability, feasibility, and engagement [ 47 ]. The app was therefore designed to incorporate a combination of teaching methods, including direct instruction, modeling, and practice to accommodate different learning styles and to emphasize self-efficacy [ 47 ]. Furthermore, user testing also highlighted the need for more direct language, particularly around problem-solving concepts [ 47 ]. The iterative study methodology used in this study enabled the participants to guide the development and provide their inputs at each stage to increase acceptability and feasibility.

Although, overall, the apps were well received by young people, poor engagement (eg, not engaging at the recommended frequency or complete the full course of the intervention), measured through both self-report, intervention adherence, and data capture was a commonly reported issue. Many studies failed to achieve continued participation, with high rates of attrition [ 41 , 42 , 44 ]. In addition, app use often decreased markedly after the initial few weeks [ 41 , 42 , 44 ]. The results of some studies suggested that engagement (in the form of intervention completion) was related to the effectiveness the intervention [ 41 , 42 ]. Although engagement was problematic in many of the stand-alone interventions, engagement and study participation in a school setting seemed more promising [ 49 ]. For example, a proactive invitation for study participation in a school enabled 4 out of 5 eligible adolescents to participate in the “ready4life” life skills program [ 49 ]. This strategy consisted of individuals who were trained in the program to be delivered, giving arranged sessions lasting 30 minutes in participating vocational schools during regular school lessons reserved for health education. Within this session, the students were informed about and invited to participate in the study, including being informed about the study’s aims and assessments, reimbursement, and data protection.

Barriers to Engagement

Qualitative interviews and user feedback provided important insights about relevant barriers to engagement. The most frequently mentioned reasons for not using the app were that participants could not see the obvious benefits of using the app [ 41 ], lack of time or forgetting [ 42 ], and technical difficulties [ 41 , 50 , 51 ]. In a life skills training app for vocational students, participation was lower in male adolescents and among those reporting an immigrant background [ 49 ], although the reasons behind this poor engagement remained unclear.

What Do Young People Want From Apps?

There was some heterogeneity between studies in terms of the features and content that participants found acceptable and appropriate. For example, findings suggested that young people who experience homelessness tended to prefer both automated and self-help features compared with ones involving more direct human interaction [ 46 ]. However, participants in other studies valued both human interaction with professionals either via the app interface or through face-to-face contact and self-help features [ 47 ]. Human support was suggested as being helpful in offering both instruction and guidance as well as personalized support when needed. Numerous participants wanted opportunities to interact with peers [ 43 , 44 , 52 ] and even suggested connecting apps to social media [ 41 ]. Others also wanted the design of the apps to be more attractive (eg, improve the layout and create a more intuitive structure) and made suggestions about how gamifying apps could make them more interesting [ 41 , 43 , 47 ].

We interviewed 13 young men in the United Kingdom (age: mean 18.7, SD 2.5 y), 2 in Spain (age: mean 17, SD 0 y), and 5 in Germany (age: mean 20.2, SD 1.6 y). In the United Kingdom, 62% (8/13) of the participants self-reported as ethnically White, compared with 50% (1/2) in Spain and 20% (1/5) in Germany ( Table 3 ).

In terms of participants’ mental health and well-being ( Table 4 ), the mean Patient Health Questionnaire-9 score in the United Kingdom was 9.7 (SD 7.3) compared with 5 (SD 1.4) in Spain.

a WEBWBS: Warwick-Edinburgh Mental Well-Being Scale.

b Not available.

c PHQ-9: Patient Health Questionnaire-9.

d GAD-7: Generalized Anxiety Disorder Assessment.

A key finding was that despite best efforts and financial incentives, recruiting underserved young male participants, especially in Spain and Germany, was challenging. This might suggest that these young people may not deem such an emotional competence app as relevant or useful to them, making recruitment and engagement problematic. We also assessed if the app was deemed acceptable (ie, useful, agreeable, palatable, or satisfactory) and appropriate (ie, relevant, suitable, or compatible). Overall, the app was viewed by participants in the United Kingdom, Spain, and Germany as being appropriate and relevant for young people of different ages and walks of life, as they thought that all young people had a smartphone and were adept at using technology:

So, I was able to learn about my feelings, I was able to evaluate how I actually felt today, concerning my feelings, if I was angry or I was sad. I was actually able to write them down in detail. [Participant in Germany]

Several participants commented that the content of the app was best suited to university and school students. Another common view was that the app was better suited to those struggling with their mental health and that it was less relevant for those for whom things were going well. Many participants perceived the app to be aimed at improving mental health problems, as opposed to being a universal intervention intended to improve well-being, which represented a barrier to engagement. Of those who reported that the app was not relevant to them, they did see it as being of potential use to friends and family members who were stressed, anxious, or going through a difficult time:

There will be folks who maybe aren’t going through a good time in their lives, and they will need the app to feel... to understand themselves, mostly. And I think it’s relevant at any age, because I am lucky that I don’t think I need it as much as someone else who feels like that. [Participant in Spain]

Partly it was important, partly it was not. I’ll give an example again, for example if a refugee came to Germany from a war zone, it’s going to be difficult, very difficult to find a topic that would fit him, for the future I mean, so the version now is already okay if you want all persons to use this app. Partly it’s already relevant and partly it’s not. If someone has mental problems or bad experiences, you cannot find such a topic in the app. [Participant in Germany]

Although some participants reported using the app regularly during the 4-week study period, a consistent finding was that participants tended to use the app most when they first downloaded it, with a marked reduction in use over time:

Uh, I probably used it about three times in the first week. And then not really that much at all I’m afraid. [Participant in the United Kingdom]

I don’t know, I just dropped off using a little bit after a couple of weeks, but I’ve been trying to keep on top of doing that like the daily rating things and everything.... I kind of lost my motivation to use it. [Participant in the United Kingdom]

We identified several barriers that hindered participants’ engagement and use of the app. These included the following: (1) repetitive and time-consuming app contents, (2) a paucity of new content and personalized or interactive tools (eg, matching mood to tools), (3) unclear instructions, (4) a lack of rationale for the app, (5) perceiving the app as not being relevant, (6) a lack of motivation, and (7) privacy concerns:

Yes, for example, I would not like to write in this diary, because I do not know if it would be one hundred percent anonymous and if others might read it. And maybe I have more privacy if I do not write it. [Participant in Germany]

I think by now I would slowly stop using the app. It was nice up to this point, but I think for me I might need a step further now. To really deal with my personal problems and I don’t know how much an app like this can help and that rather an expert and therapy is needed. [Participant in Germany]

For the asylum seekers and refugees in Germany, the language and content of the app was not suited to their needs. The participants would have preferred the app in their native language as some had to use translation programs to help access the content. Furthermore, specific topics of relevance to refugees were missing, such as dealing with asylum uncertainty, whereabouts of family members, and their living situation.

Finally, underserved young people, including asylum seekers and refugees, migrants, and those NEET, are more likely to experience financial deprivations and therefore less likely to pay directly for apps, especially for those that do not address their primary difficulties:

If it came to the point that I had to pay for it, I would look for free options. [Participant in Germany]

Summary of Findings

The use of mobile apps in mental health care continues to attract interest and investment; however, research geared toward understanding the needs of marginalized and underserved populations is still nascent. This study, focusing on the implementation of mental health apps in underserved young people, highlighted that little research exists to support the widespread adoption of these apps as a mental health intervention for marginalized and underserved groups. Findings from both our systematic review and qualitative study were largely consistent: markers of acceptability and usability were positive; however, engagement for underserved young people was low, which is notable given the widespread ownership of smartphones [ 55 , 56 ]. To date, research has focused primarily on efficacy studies rather than effectiveness and implementation in “real-world” settings and may have overestimated users’ “natural tendency” to adopt smartphone apps for their mental health and well-being [ 57 ]. Our findings suggest that despite the rapid proliferation of mobile mental health technology, the uptake and engagement of mental health apps among marginalized young people are low and remain a key implementation challenge.

Our data suggest that establishing and maintaining user commitment and engagement in the content of the intervention as intended is a pervasive challenge across mental health apps and marginalized populations, and premature dropout was prominent in nearly all the included studies. This is consistent with the literature that suggests that the majority of those offered these app-based interventions do not engage at the recommended frequency or complete the full course of treatment [ 58 , 59 ]. In this study, various app components were associated with engagement level, with the most engaging interventions providing young people with some form of associated real-human interaction and those having a more interactive interface. This aligned with other findings that the feedback of personalized information to participants is an especially important aspect of creating engaging and impactful digital tools [ 60 ]. Young people tend to quickly disengage if there are technical difficulties or if the app does not specifically target their perceived needs [ 41 , 50 , 51 ]. Furthermore, recruitment of marginalized groups to app-based studies is difficult. For instance, in this study, the use of advertisements, financial incentives, vouchers, and prize draw incentives seemed to be insufficient to recruit a significant number of participants in Spain and Germany.

Measuring engagement is a challenge that has likely contributed to our lack of knowledge on app components that effectively increase user engagement. Reporting engagement with mental health apps in intervention trials is highly variable, and a number of basic metrics of intervention engagement, such as rate of intervention uptake, weekly use patterns, and number of intervention completers, are available, yet not routinely reported [ 58 , 59 ]. The results of this study highlight the importance of objective engagement measures and that relying on positive subjective self-reports of usability, satisfaction, acceptability, and feasibility is insufficient to determine actual engagement. Furthermore, the findings suggest that apps involving human interactions with a professional (eg, therapist or counselor) or that are completed in a supervised setting tend to be more acceptable and effective and have higher engagement rates [ 47 , 48 ]. Our research suggests that similar to traditional face-to-face mental health services, app-based programs still face numerous barriers to reach marginalized youth, especially since the mental health apps available to the public do not seem to consider the unique developmental needs of these groups, participants do not seem to perceive an obvious benefit from using them, and some potential users prefer to interact with a professional face to face. Thus, it is also possible that the digital mental health field might be inadvertently contributing to mental health inequities among this population by not engaging marginalized groups sufficiently at the outset of research to ensure that the designed app meets their needs. However, for the studies included in this study that did engage these groups in the co-design of the apps, there was no notable improvement in engagement. Thus, we hope these findings encourage researchers and clinicians to think more critically of the role that mental health apps can truly have in addressing mental health equities among underserved groups.

As in other areas of mental health research, young people from LMICs were underrepresented in these studies, which typically originated from high-income settings, including the United States, Australia, and Canada. There are relatively few app-based interventions that were designed or adapted for young people in LMICs that have been rigorously evaluated or are even available in local languages [ 47 , 48 ]. Many living in LMIC regions, for example, adults in Asian countries, are often faced with apps that are not culturally relevant or in the right language [ 61 ]. These inequities are surprising given the high rates of smartphone use in Asia, even in rural regions [ 62 ]. Yet, it is still likely that youth in this region faced barriers related to data availability and more limited phone access, which will likely inhibit the broad implementation of apps beyond research studies [ 16 ]. Considerable work is required to ensure the availability of mental health apps that fit a wide range of user needs and preferences. It is important to ensure that the acceptability and feasibility of mental health apps for young people residing in LMICs are prioritized so that they are not further excluded from relevant mental health research.

Finally, a significant challenge is the lack of diversity in mental health app research participation, which limits our understanding of “real-world” efficacy and implementation for underserved and marginalized groups. While undoubtedly invaluable, and indeed deemed gold standard when evaluating efficacy of interventions, randomized controlled trial of mental health apps are not without flaws [ 63 , 64 ]. Trial recruitment is often highly selective due to stringent inclusion and exclusion criteria resulting in lower inclusion in research than one would expect from population estimates [ 65 ]. In the United Kingdom, the National Institute for Health and Care Research data have revealed that geographies with the highest burden of disease also have the lowest number of patients taking part in research [ 66 ]. The postcodes in which research recruitment is low also aligns closely to areas where earnings are the lowest and indexes of deprivation are the highest [ 66 ]. There are many reasons why some groups are underrepresented in research: language barriers, culturally inappropriate explanations, poor health literacy and the use of jargon, communication not being suitable for people with special learning needs, requirement to complete many administrative forms, negative financial impact in participating, lack of effective incentives for participation, or lack of clarity around incentives, and specific cultural and religious beliefs [ 66 ]. Failing to include a broad range of participants is problematic in that results may not be generalizable to a broad population.

Limitations

Although this research was carefully executed and used a robust methodological approach with an exhaustive search strategy, it is not without limitations. Foremost, although the systematic review attempted to identify and include as many articles as possible, some papers may have been missed because of the inconsistencies in how feasibility and acceptability outcomes are recorded and reported. It was also difficult to ensure that all apps for this age group were identified because those aged between 15 and 25 years are harder to differentiate in adolescent and adult studies, meaning we might have missed some relevant studies where data could not be disaggregated by age. The exclusion of gray literature (eg, institutional reports and websites) may have also made us overlook potentially relevant apps, albeit lacking the quality assurance of peer-reviewed research. It is also likely that commercial organizations, including app companies, collect rich user demographic and engagement data but do not share it publicly, thus limiting our ability to conduct empirical analyses about the “real-world” acceptability, engagement, and implementation for specific populations. We did not analyze the extent to which publication bias may have influenced the results of our search, and, therefore, there may be a much higher number of mental health apps that have been developed with an underserved sample of young people, but due to their lack of efficacy or acceptability, these studies have not been submitted or accepted for publication. The sample sizes of many of the included studies were relatively low, which potentially limits their generalizability. However, we included all study designs so as to ensure that our learning from existing research was maximized. Furthermore, many of the studies included in the systematic review, as well as our qualitative study, had some form of language competency as an inclusion criterion (eg, English speaking), which likely excludes important perspectives from the results. For the qualitative study, we were only able to gather data from those who had used the app at least once and who were therefore somewhat engaged in the app. Despite our best efforts, we were unable to recruit participants who, following consent, had never then downloaded or used the app and so could not explore barriers to engagement for the least engaged young people or understand why the app was not appealing to those who chose not to proceed or take part. Those who did participate in this research were financially incentivized to do so and often highlighted the importance of this incentive in keeping them engaged. Therefore, we were unable to draw conclusions about the naturalistic engagement, feasibility, and acceptability of the app, if it were to be made available without payment in schools, universities, and health services or to be made commercially available on the app marketplace. It is also possible that social desirability bias (ie, a tendency to present reality to align with what is perceived to be socially acceptable) occurred during the interviews, whereby participants responded to the interview questions in a manner that they believed would be more acceptable to the study team, concealing their true opinions or experiences [ 67 , 68 ]. As previously noted by others, results may be subject to further bias in that findings could be led by more articulate young people, while it is more difficult to hear the voices of those who are less articulate or digitally literate [ 69 ]. Finally, it is also possible that the positionality of the research team, including our own experiences, backgrounds, and biases, impacted what information participants disclosed to the research team as well as the interpretation of the qualitative data in this study.

Recommendations of Adaptations to Increase Acceptability, Feasibility, and Engagement

To overcome this complex engagement and implementation challenge, we have taken together our findings with relevant previous literature to generate 3 key suggestions about how to improve the feasibility and potential utility of apps for young people from marginalized and underserved populations.

Increasing Participant Diversity in Mental Health Intervention Research

Studies should aim to prioritize the inclusion of marginalized groups in trials testing the effectiveness of digital interventions by intentionally planning recruitment efforts aimed to reach these communities [ 70 ]. First, steps can be taken to build trust, connections, and credibility between the research team and these communities. NHS England [ 66 ] suggests involving representatives from those groups during the inception and implementation of recruitment efforts. This approach ensures that the intervention is relevant to the target group by meeting their preferences and needs, incorporating culturally salient factors relevant for recruitment efforts, addressing concerns about community mistrust and participant resource constraints, and establishing partnerships with key community stakeholders that can be gatekeepers in the community [ 14 , 71 ]. These strategies are likely to improve research accessibility, recruitment, and retention. Research teams need to ensure that the findings and any actionable takeaways from the research conducted with the participants are shared with them by asking participants how they would like to receive this information (eg, verbal, written, or via a trusted advocate). Equally important is to explain that the research process can be slow. These steps help create a positive legacy for the research project and build trust between individuals and public institutions, helping future health researchers to further address underrepresentation of marginalized groups in digital research.

Identifying and Addressing Needs and Preferences of Underserved and Marginalized Groups by Using Human-Centered Design Principles

A comprehensive understanding of the needs, challenges, and life circumstances of the target population is a key implementation driver for designing relevant, engaging, and effective mental health apps. This knowledge is particularly important when the app is a stand-alone intervention received during daily life outside of traditional psychotherapy or human support [ 50 ]. This goal can be best achieved through a participatory approach, which reflects a growing recognition among intervention researchers and developers that end users need to be involved in the creation of interventions and their future iterations [ 47 , 72 ]. This process may involve a series of stages, including (1) person-centered co-design to ensure that tools are developed to be acceptable to the underserved or marginalized populations as well as meet their specific needs, life circumstances, and cultural norms [ 47 ]; (2) iterative testing that incorporates users’ feedback on a rolling basis to ensure the relevance of the intervention [ 43 , 47 , 72 ]; and (3) changes and adaptations needed to meet users’ needs in “real-world” settings including consideration of economic viability and implementation [ 27 ].

Especially relevant for the underserved and marginalized groups is the need (or lack thereof) to culturally adapt app interventions for specific racial, ethnic, or cultural groups through this person-centered design. In traditional face-to-face interventions, some have argued that all treatments need to be culturally adapted to ensure their validity, relevance, and effectiveness since these interventions are often developed with individuals who can be substantially different from some marginalized populations [ 73 ]. Similar to culturally adapted face-to-face interventions [ 74 - 76 ], culturally adapted digital mental health interventions seem to be effective [ 77 , 78 ]. However, there is no evidence that these culturally adapted interventions outperform the original programs [ 79 , 80 ]. Given that culturally adapting digital interventions is a time-consuming and resource-intensive process, this approach may not be sustainable and limit the dissemination and implementation impact of app programs [ 28 ]. In lieu of culturally adapting digital interventions without careful consideration, Ramos and Chavira [ 28 ] recommend using information gathered through person-centered approaches to integrate culture into the use of already available digital interventions (including apps), using an idiographic, flexible, and personalized approach. This strategy may have a broader implementation and dissemination potential, given that few researchers and clinicians are in a position to develop new apps.

Embedding Apps Within Existing Care Structures

Several systematic reviews and meta-analyses have demonstrated that app-based mental health interventions with a human-support component are more effective and more acceptable than stand-alone, fully automatized, or self-administered apps [ 13 , 25 , 81 ]. Young people seem to want practical skills and usable tools to apply to their current daily life stressors to improve their well-being and functioning. Intervention engagement is enhanced if the intervention serves an obvious purpose, is relevant, and has a clear rationale and instructions, and embedding these interventions within the systems and structures that are already working with users (eg, clinical services, schools, universities, and community agencies) will likely improve implementation. Considering the broad and highly varied nature of intervention formats and modalities, it may be useful for future research to focus on identifying core components of app-based interventions (ie, active ingredients of interventions associated with uptake, adherence, and clinical outcomes) that will allow such integration of app interventions into the varied context of care for marginalized youth.

Conclusions

Despite the enthusiasm that has surrounded the potential of digital technologies to revolutionize mental health and health care service delivery, little evidence yet supports the use of mental health apps for marginalized and underserved young people. Despite the substantial financial and human investment directed to the development of mental health apps over several years, only a small proportion have empirical evidence to support their effectiveness, and there have been few attempts to develop or adapt interventions to meet some of the more unique and heterogeneous needs of diverse groups of young people. Although acceptability seems to be good, engagement is poor and attrition is high, particularly if not supported by in-person elements. Given that most interventions are implemented in high-income countries, very little is known about the generalizability of the findings to LMICs and to a range of adolescents and young people with different socioeconomic, cultural, and racial backgrounds. In this paper, we have drawn several insights about the feasibility and acceptability of mental health apps for underserved young people that may be useful to future app-based mental health promotion and treatment projects. However, before the widespread adoption and scaling-up of digital mental health interventions progresses further, especially for more vulnerable and underserved populations and in settings with limited resources, a greater understanding is needed on the unique barriers faced by these groups in accessing treatment and the types of services young people themselves prefer (eg, standard vs digital) followed by more rigorous and consistent demonstrations of feasibility, effectiveness, and cost-effectiveness.

Acknowledgments

This project received funding from the European Union’s Horizon 2020 research and innovation program (grant agreement number 754657).

The authors are grateful to the young people who took the time to participate in this research and who shared their insights with us. The authors would also like to thank those who supported this research including professional youth advisor Emily Bampton, research assistant Catherine Reeve, and researchers Dr Alexandra Langmeyer and Simon Weiser. Finally, the authors would like to thank the ECoWeB (Emotional Competence for Well-Being) Consortium for their support and feedback throughout the duration of this research, including, but not limited to, Dr Lexy Newbold, Dr Azucena Garcia Palacios, and Dr Guadalupe Molinari.

Data Availability

The data extracted to support the findings of the systematic review are available from the corresponding author upon reasonable request. Due to the confidential and sensitive nature of the interview transcripts, qualitative data will not be made available.

Authors' Contributions

HAB, LAN, and MF designed the systematic review including the research questions and methods. LAN carried out the database search. HAB, LAN, TM, and BF conducted the study screening and data extraction. TM did the study quality assessments, and HAB did the data synthesis and analysis. MF, SW, EW, and HAB were involved in the conception of the qualitative study. HAB, LAN, and SC conducted the quality study including conducting the qualitative interviews and analysis. HAB wrote the first draft and HAB, LAN, MF, and GR contributed substantially to manuscript drafting. All authors contributed to the manuscript and approved the submitted version.

Conflicts of Interest

None declared.

Search strategy.

Topic guide.

Study quality assessment.

• Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. Jun 2005;62(6):593-602. [ CrossRef ] [ Medline ]
• Solmi M, Radua J, Olivola M, Croce E, Soardo L, Salazar de Pablo G, et al. Age at onset of mental disorders worldwide: large-scale meta-analysis of 192 epidemiological studies. Mol Psychiatry. Jan 2022;27(1):281-295. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brown A, Rice SM, Rickwood DJ, Parker AG. Systematic review of barriers and facilitators to accessing and engaging with mental health care among at-risk young people. Asia Pac Psychiatry. Mar 03, 2016;8(1):3-22. [ CrossRef ] [ Medline ]
• Marrast L, Himmelstein DU, Woolhandler S. Racial and ethnic disparities in mental health care for children and young adults: a national study. Int J Health Serv. Oct 20, 2016;46(4):810-824. [ CrossRef ] [ Medline ]
• Cook BL, Trinh NH, Li Z, Hou SS, Progovac AM. Trends in racial-ethnic disparities in access to mental health care, 2004-2012. Psychiatr Serv. Jan 01, 2017;68(1):9-16. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Rimes KA, Shivakumar S, Ussher G, Baker D, Rahman Q, West E. Psychosocial factors associated with suicide attempts, ideation, and future risk in lesbian, gay, and bisexual youth. Crisis. Mar 2019;40(2):83-92. [ CrossRef ] [ Medline ]
• Reiss F. Socioeconomic inequalities and mental health problems in children and adolescents: a systematic review. Soc Sci Med. Aug 2013;90:24-31. [ CrossRef ] [ Medline ]
• Warnecke RB, Oh A, Breen N, Gehlert S, Paskett E, Tucker KL, et al. Approaching health disparities from a population perspective: the National Institutes of Health Centers for Population Health and Health Disparities. Am J Public Health. Sep 2008;98(9):1608-1615. [ CrossRef ]
• Schueller SM, Hunter JF, Figueroa C, Aguilera A. Use of digital mental health for marginalized and underserved populations. Curr Treat Options Psych. Jul 5, 2019;6(3):243-255. [ CrossRef ]
• Introduction: NIH minority health and health disparities strategic plan 2021-2025. National Institutes of Health National Institute on Minority Health and Health Disparities. URL: https://www.nimhd.nih.gov/about/strategic-plan/nih-strategic-plan-directors-foreword.html [accessed 2024-06-29]
• Improving inclusion of under-served groups in clinical research: Guidance from INCLUDE project. National Institute for Health and Care Excellence. Aug 7, 2020. URL: https:/​/www.​nihr.ac.uk/​documents/​improving-inclusion-of-under-served-groups-in-clinical-research-guidance-from-include-project/​25435 [accessed 2024-06-29]
• Evans-Lacko S, Aguilar-Gaxiola S, Al-Hamzawi A, Alonso J, Benjet C, Bruffaerts R, et al. Socio-economic variations in the mental health treatment gap for people with anxiety, mood, and substance use disorders: results from the WHO World Mental Health (WMH) surveys. Psychol Med. Nov 27, 2017;48(9):1560-1571. [ CrossRef ]
• Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual research review: digital health interventions for children and young people with mental health problems - a systematic and meta-review. J Child Psychol Psychiatry. Apr 10, 2017;58(4):474-503. [ CrossRef ] [ Medline ]
• Friis-Healy EA, Nagy GA, Kollins SH. It is time to REACT: opportunities for digital mental health apps to reduce mental health disparities in racially and ethnically minoritized groups. JMIR Ment Health. Jan 26, 2021;8(1):e25456. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mobile fact sheet. Pew Research Center. Jan 31, 2024. URL: https://www.pewresearch.org/internet/fact-sheet/mobile/ [accessed 2024-07-12]
• Silver L. 2. In emerging economies, smartphone adoption has grown more quickly among younger generations. Pew Research Center. Feb 5, 2019. URL: https:/​/www.​pewresearch.org/​global/​2019/​02/​05/​in-emerging-economies-smartphone-adoption-has-grown-more-quickly-among-younger-generations/​ [accessed 2024-07-12]
• Pretorius C, Chambers D, Coyle D. Young people's online help-seeking and mental health difficulties: systematic narrative review. J Med Internet Res. Nov 19, 2019;21(11):e13873. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grist R, Porter J, Stallard P. Mental health mobile apps for preadolescents and adolescents: a systematic review. J Med Internet Res. May 25, 2017;19(5):e176. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lehtimaki S, Martic J, Wahl B, Foster KT, Schwalbe N. Evidence on digital mental health interventions for adolescents and young people: systematic overview. JMIR Ment Health. Apr 29, 2021;8(4):e25847. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Leech T, Dorstyn D, Taylor A, Li W. Mental health apps for adolescents and young adults: a systematic review of randomised controlled trials. Child Youth Serv Rev. Aug 2021;127:106073. [ CrossRef ]
• Buttazzoni A, Brar K, Minaker L. Smartphone-based interventions and internalizing disorders in youth: systematic review and meta-analysis. J Med Internet Res. Jan 11, 2021;23(1):e16490. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eisenstadt M, Liverpool S, Infanti E, Ciuvat RM, Carlsson C. Mobile apps that promote emotion regulation, positive mental health, and well-being in the general population: systematic review and meta-analysis. JMIR Ment Health. Nov 08, 2021;8(11):e31170. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lipschitz J, Hogan TP, Bauer MS, Mohr DC. Closing the research-to-practice gap in digital psychiatry: the need to integrate implementation science. J Clin Psychiatry. May 14, 2019;80(3):18com12659. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hollis C. Youth mental health: risks and opportunities in the digital world. World Psychiatry. Feb 11, 2022;21(1):81-82. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bear HA, Ayala Nunes L, DeJesus J, Liverpool S, Moltrecht B, Neelakantan L, et al. Determination of markers of successful implementation of mental health apps for young people: systematic review. J Med Internet Res. Nov 09, 2022;24(11):e40347. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. Mar 19, 2011;38(2):65-76. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ramos G, Ponting C, Labao JP, Sobowale K. Considerations of diversity, equity, and inclusion in mental health apps: a scoping review of evaluation frameworks. Behav Res Ther. Dec 2021;147:103990. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ramos G, Chavira DA. Use of technology to provide mental health care for racial and ethnic minorities: evidence, promise, and challenges. Cognit Behav Pract. Feb 2022;29(1):15-40. [ CrossRef ]
• Wies B, Landers C, Ienca M. Digital mental health for young people: a scoping review of ethical promises and challenges. Front Digit Health. Sep 6, 2021;3:697072. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liverpool S, Mota CP, Sales CM, Čuš A, Carletto S, Hancheva C, et al. Engaging children and young people in digital mental health interventions: systematic review of modes of delivery, facilitators, and barriers. J Med Internet Res. Jun 23, 2020;22(6):e16317. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Marcano Belisario JS, Huckvale K, Greenfield G, Car J, Gunn LH. Smartphone and tablet self management apps for asthma. Cochrane Database Syst Rev. Nov 27, 2013;2013(11):CD010013. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Marcano Belisario JS, Jamsek J, Huckvale K, O'Donoghue J, Morrison CP, Car J. Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev. Jul 27, 2015;2015(7):MR000042. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. Mar 29, 2021;372:n71. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hong QN, Fàbregues S, Bartlett G, Boardman F, Cargo MP, Dagenais P, et al. The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers. Educ Inf. Dec 18, 2018;34(4):285-291. [ CrossRef ]
• Crabtree BF, Miller WL. A template approach to text analysis: developing and using codebooks. In: Crabtree BF, Miller WL, editors. Doing Qualitative Research. Thousand Oaks, CA. SAGE Publications, Inc; 1992:93-109.
• Newbold A, Warren FC, Taylor RS, Hulme C, Burnett S, Aas B, et al. Promotion of mental health in young adults via mobile phone app: study protocol of the ECoWeB (emotional competence for well-being in young adults) cohort multiple randomised trials. BMC Psychiatry. Sep 22, 2020;20(1):458. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. Jan 2006;3(2):77-101. [ CrossRef ]
• Ellaway RH, Kehoe A, Illing J. Critical realism and realist inquiry in medical education. Acad Med. Jul 2020;95(7):984-988. [ CrossRef ] [ Medline ]
• Willig C. Perspectives on the epistemological bases for qualitative research. In: Cooper H, Camic PM, Long DL, Panter AT, Rindskopf D, Sher KJ, editors. APA Handbook of Research Methods in Psychology, Vol. 1. Foundations, Planning, Measures, and Psychometrics. Washington, DC. American Psychological Association; 2012.
• Fletcher AJ. Applying critical realism in qualitative research: methodology meets method. Int J Soc Res Methodol. Feb 29, 2016;20(2):181-194. [ CrossRef ]
• Bohleber L, Crameri A, Eich-Stierli B, Telesko R, von Wyl A. Can we foster a culture of peer support and promote mental health in adolescence using a web-based app? A control group study. JMIR Ment Health. Sep 23, 2016;3(3):e45. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Deady M, Glozier N, Collins D, Einboden R, Lavender I, Wray A, et al. The utility of a mental health app in apprentice workers: a pilot study. Front Public Health. Sep 4, 2020;8:389. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fleming JB, Hill YN, Burns MN. Usability of a culturally informed mHealth intervention for symptoms of anxiety and depression: feedback from young sexual minority men. JMIR Hum Factors. Aug 25, 2017;4(3):e22. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Francis J, Cross D, Schultz A, Armstrong D, Nguyen R, Branch-Smith C. Developing a smartphone application to support social connectedness and wellbeing in young people with cystic fibrosis. J Cyst Fibros. Mar 2020;19(2):277-283. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Geirhos A, Domhardt M, Lunkenheimer F, Temming S, Holl RW, Minden K, et al. Feasibility and potential efficacy of a guided internet- and mobile-based CBT for adolescents and young adults with chronic medical conditions and comorbid depression or anxiety symptoms (youthCOACH): a randomized controlled pilot trial. BMC Pediatr. Jan 29, 2022;22(1):69. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Glover AC, Schueller SM, Winiarski DA, Smith DL, Karnik NS, Zalta AK. Automated mobile phone-based mental health resource for homeless youth: pilot study assessing feasibility and acceptability. JMIR Ment Health. Oct 11, 2019;6(10):e15144. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gonsalves PP, Hodgson ES, Kumar A, Aurora T, Chandak Y, Sharma R, et al. Design and development of the "POD adventures" smartphone game: a blended problem-solving intervention for adolescent mental health in India. Front Public Health. Aug 23, 2019;7:238. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gonsalves PP, Hodgson ES, Bhat B, Sharma R, Jambhale A, Michelson D, et al. App-based guided problem-solving intervention for adolescent mental health: a pilot cohort study in Indian schools. Evid Based Ment Health. Feb 18, 2021;24(1):11-18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Haug S, Paz Castro R, Meyer C, Filler A, Kowatsch T, Schaub MP. A mobile phone-based life skills training program for substance use prevention among adolescents: pre-post study on the acceptance and potential effectiveness of the program, Ready4life. JMIR Mhealth Uhealth. Oct 04, 2017;5(10):e143. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Leonard NR, Casarjian B, Fletcher RR, Praia C, Sherpa D, Kelemen A, et al. Theoretically-based emotion regulation strategies using a mobile app and wearable sensor among homeless adolescent mothers: acceptability and feasibility study. JMIR Pediatr Parent. Mar 01, 2018;1(1):e1. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Schueller SM, Glover AC, Rufa AK, Dowdle CL, Gross GD, Karnik NS, et al. A mobile phone-based intervention to improve mental health among homeless young adults: pilot feasibility trial. JMIR Mhealth Uhealth. Jul 02, 2019;7(7):e12347. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Escobar-Viera CG, Porta G, Coulter RW, Martina J, Goldbach J, Rollman BL. A chatbot-delivered intervention for optimizing social media use and reducing perceived isolation among rural-living LGBTQ+ youth: development, acceptability, usability, satisfaction, and utility. Internet Interv. Dec 2023;34:100668. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Torok M, Han J, McGillivray L, Wong Q, Werner-Seidler A, O'Dea B, et al. The effect of a therapeutic smartphone application on suicidal ideation in young adults: findings from a randomized controlled trial in Australia. PLoS Med. May 2022;19(5):e1003978. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yang YJ, Chung KM. Pilot randomized control trial of an app-based CBT program for reducing anxiety in individuals with ASD without intellectual disability. J Autism Dev Disord. Apr 2023;53(4):1331-1346. [ CrossRef ] [ Medline ]
• Children and parents: media use and attitudes report 2018. Ofcom. Mar 5, 2019. URL: https:/​/www.​ofcom.org.uk/​media-use-and-attitudes/​media-habits-children/​children-and-parents-media-use-and-attitudes-report-2018/​ [accessed 2024-07-12]
• Silver L. Smartphone ownership is growing rapidly around the world, but not always equally. Pew Research Center. Feb 5, 2019. URL: https:/​/www.​pewresearch.org/​global/​2019/​02/​05/​smartphone-ownership-is-growing-rapidly-around-the-world-but-not-always-equally/​ [accessed 2024-07-12]
• Chan A, Kow R, Cheng JK. Adolescents’ perceptions on smartphone applications (apps) for health management. JournalMTM. Aug 2017;6(2):47-55. [ CrossRef ]
• Lipschitz JM, Van Boxtel R, Torous J, Firth J, Lebovitz JG, Burdick KE, et al. Digital mental health interventions for depression: scoping review of user engagement. J Med Internet Res. Oct 14, 2022;24(10):e39204. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ng MM, Firth J, Minen M, Torous J. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv. Jul 01, 2019;70(7):538-544. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Oakley-Girvan I, Yunis R, Longmire M, Ouillon JS. What works best to engage participants in mobile app interventions and e-health: a scoping review. Telemed J E Health. Jun 2022;28(6):768-780. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brian RM, Ben-Zeev D. Mobile health (mHealth) for mental health in Asia: objectives, strategies, and limitations. Asian J Psychiatr. Aug 2014;10:96-100. [ CrossRef ] [ Medline ]
• Li H, Lewis C, Chi H, Singleton G, Williams N. Mobile health applications for mental illnesses: an Asian context. Asian J Psychiatr. Dec 2020;54:102209. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Spielmans GI, Gatlin ET, McFall JP. The efficacy of evidence-based psychotherapies versus usual care for youths: controlling confounds in a meta-reanalysis. Psychother Res. Mar 23, 2010;20(2):234-246. [ CrossRef ] [ Medline ]
• Rapport F, Smith J, Hutchinson K, Clay-Williams R, Churruca K, Bierbaum M, et al. Too much theory and not enough practice? The challenge of implementation science application in healthcare practice. J Eval Clin Pract. Dec 15, 2022;28(6):991-1002. [ CrossRef ] [ Medline ]
• Jensen SA, Corralejo SM. Measurement issues: large effect sizes do not mean most people get better - clinical significance and the importance of individual results. Child Adolesc Ment Health. Sep 2017;22(3):163-166. [ CrossRef ] [ Medline ]
• Increasing diversity in research participation: a good practice guide for engaging with underrepresented groups. National Health Service England. Feb 14, 2023. URL: https://www.england.nhs.uk/aac/publication/increasing-diversity-in-research-participation/ [accessed 2023-03-16]
• Bergen N, Labonté R. "Everything is perfect, and we have no problems": detecting and limiting social desirability bias in qualitative research. Qual Health Res. Apr 2020;30(5):783-792. [ CrossRef ] [ Medline ]
• Vogt WP, Johnson RB. Dictionary of Statistics & Methodology: A Nontechnical Guide for the Social Sciences. Thousand Oaks, CA. SAGE Publications; 2011.
• Midgley N, Parkinson S, Holmes J, Stapley E, Eatough V, Target M. "Did I bring it on myself?" An exploratory study of the beliefs that adolescents referred to mental health services have about the causes of their depression. Eur Child Adolesc Psychiatry. Jan 20, 2017;26(1):25-34. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Willis A, Isaacs T, Khunti K. Improving diversity in research and trial participation: the challenges of language. Lancet Public Health. Jul 2021;6(7):e445-e446. [ CrossRef ]
• Ellard-Gray A, Jeffrey NK, Choubak M, Crann SE. Finding the hidden participant: solutions for recruiting hidden, hard-to-reach, and vulnerable populations. Int J Qual Method. Dec 17, 2015;14(5). [ CrossRef ]
• Moltrecht B, Patalay P, Bear HA, Deighton J, Edbrooke-Childs J. A transdiagnostic, emotion regulation app (Eda) for children: design, development, and lessons learned. JMIR Form Res. Jan 19, 2022;6(1):e28300. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bernal G, Jiménez-Chafey MI, Domenech Rodríguez MM. Cultural adaptation of treatments: a resource for considering culture in evidence-based practice. Prof Psychol Res Pract. Aug 2009;40(4):361-368. [ CrossRef ]
• Benish SG, Quintana S, Wampold BE. Culturally adapted psychotherapy and the legitimacy of myth: a direct-comparison meta-analysis. J Couns Psychol. Jul 2011;58(3):279-289. [ CrossRef ] [ Medline ]
• Hall GC, Ibaraki AY, Huang ER, Marti CN, Stice E. A meta-analysis of cultural adaptations of psychological interventions. Behav Ther. Nov 2016;47(6):993-1014. [ CrossRef ] [ Medline ]
• Huey SJJ, Polo AJ. Evidence-based psychosocial treatments for ethnic minority youth. J Clin Child Adolesc Psychol. Jan 03, 2008;37(1):262-301. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ellis DM, Draheim AA, Anderson PL. Culturally adapted digital mental health interventions for ethnic/racial minorities: a systematic review and meta-analysis. J Consult Clin Psychol. Oct 2022;90(10):717-733. [ CrossRef ] [ Medline ]
• Harper Shehadeh M, Heim E, Chowdhary N, Maercker A, Albanese E. Cultural adaptation of minimally guided interventions for common mental disorders: a systematic review and meta-analysis. JMIR Ment Health. Sep 26, 2016;3(3):e44. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Balci S, Spanhel K, Sander LB, Baumeister H. Culturally adapting internet- and mobile-based health promotion interventions might not be worth the effort: a systematic review and meta-analysis. NPJ Digit Med. Mar 23, 2022;5(1):34. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Spanhel K, Balci S, Feldhahn F, Bengel J, Baumeister H, Sander LB. Cultural adaptation of internet- and mobile-based interventions for mental disorders: a systematic review. NPJ Digit Med. Aug 25, 2021;4(1):128. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grist R, Croker A, Denne M, Stallard P. Technology delivered interventions for depression and anxiety in children and adolescents: a systematic review and meta-analysis. Clin Child Fam Psychol Rev. Jun 18, 2019;22(2):147-171. [ FREE Full text ] [ CrossRef ] [ Medline ]

Abbreviations

Edited by T de Azevedo Cardoso, S Ma; submitted 13.05.23; peer-reviewed by P Whelan, I Vainieri, H Bao; comments to author 13.09.23; revised version received 26.09.23; accepted 10.06.24; published 30.07.24.

©Holly Alice Bear, Lara Ayala Nunes, Giovanni Ramos, Tanya Manchanda, Blossom Fernandes, Sophia Chabursky, Sabine Walper, Edward Watkins, Mina Fazel. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 30.07.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Iran J Nurs Midwifery Res
• v.20(6); Nov-Dec 2015

Challenges in conducting qualitative research in health: A conceptual paper

Hamidreza khankeh.

1 Department of Health in Disaster and Emergencies and Nursing, University of Social Welfare and Rehabilitation, Tehran, Iran and Department of Clinical Sciences and Education, Karolinska Institute, Stockholm, Sweden

Maryam Ranjbar

2 Department of Psychology in Institute of Humanities and Social Studies, and Social Determinants of Health Research Center in University of Social Welfare and Rehabilitation, Tehran, Iran

Davoud Khorasani-Zavareh

3 Social Determinants of Health Research Center, Uremia University of Medical Sciences, Uremia, Iran and Department of Clinical Science and Education, Södersjukhuset, Karolinska Institutet, Stockholm, Sweden

Ali Zargham-Boroujeni

4 Nursing and Midwifery Care Research Center, Faculty of Nursing and Midwifery, Isfahan University of Medical Sciences, Isfahan, Iran

Eva Johansson

5 Department of Public Health Sciences, Karolinska Institutet, Stockholm, Sweden

Background:

Qualitative research focuses on social world and provides the tools to study health phenomena from the perspective of those experiencing them. Identifying the problem, forming the question, and selecting an appropriate methodology and design are some of the initial challenges that researchers encounter in the early stages of any research project. These problems are particularly common for novices.

Materials and Methods:

This article describes the practical challenges of using qualitative inquiry in the field of health and the challenges of performing an interpretive research based on professional experience as a qualitative researcher and on available literature.

One of the main topics discussed is the nature of qualitative research, its inherent challenges, and how to overcome them. Some of those highlighted here include: identification of the research problem, formation of the research question/aim, and selecting an appropriate methodology and research design, which are the main concerns of qualitative researchers and need to be handled properly. Insights from real-life experiences in conducting qualitative research in health reveal these issues.

Conclusions:

The paper provides personal comments on the experiences of a researcher in conducting pure qualitative research in the field of health. It offers insights into the practical difficulties encountered when performing qualitative studies and offers solutions and alternatives applied by these authors, which may be of use to others.

I NTRODUCTION

Health services and health policy research can be based on qualitative research methods, especially when they deal with a rapid change and develop a more fully integrated theory base and research agenda. However, the field must be with the best traditions and techniques of qualitative methods and should distinguish the essentiality of special training and experience in applying these methods.[ 1 ]

Qualitative research methodologies could help improve our understanding of health-related phenomena. Health knowledge must also include interpretive action to maintain scientific quality when research methods are applied. Qualitative and quantitative strategies should be seen as complementary rather than being thought of as incompatible. Although the procedures of interpreting texts are different from those of statistical analysis, due to their different type of data and questions to be answered, the underlying scientific principles are very much the same.[ 2 ]

While working for more than a decade as qualitative designer, Khankeh faced a lot of challenges in conducting qualitative research in the field of health which occupied the mind of other health researchers. Therefore, this article contributes to the discussion of challenges related to qualitative research in healthcare in the light of personal experiences of a researcher conducting purely qualitative health research.

A M AIN I SSUE FOR THE Q UALITATIVE R ESEARCHER

Qualitative research methods involve systematic collection, organizing, and interpretation of material in textual form derived from talk or observations. They are useful to explore the meanings of social phenomena as experienced by individuals in their natural context. The health community still looks at qualitative research with skepticism and accuses it for the subjective nature and absence of facts. Scientific standards, criteria and checklists do exist and the adequacy of guidelines has been vigorously debated within this cross-disciplinary field.[ 2 ]

Clinical knowledge consists of interpretive action and interaction – factors that involve communication, shared opinions, and experiences. The current quantitative research methods indicate a confined access to clinical knowledge, since they insert only the questions and phenomena that can be controlled, measured, and are countable where it is necessary to investigate, share and contest the tacit knowledge of an experienced practitioner. Qualitative research focuses on the people's social world, and not their disease. It is concerned with increased understanding of the meaning of certain conditions for health professionals and patients, and how their relationships are built in a particular social context.[ 3 ] These kinds of research allow exploration of the social events as experienced by individuals in their natural context. Qualitative inquiry could contribute to a broader understanding of health science [ 4 ] considering the substantial congruence between the core elements of health practice and the principles underpinning qualitative research. The globalization progress augments the necessity of qualitative research.[ 5 ]

Corbin (2008) reported that in the past 10 years, the interest in qualitative methods in general and grounded theory in particular has burgeoned according to a review of the literature and dissertation abstracts.[ 6 ]

A researcher engaged in qualitative research will be confronted with a number of challenges. Identifying the research problem and forming the research question are some of the initial challenges that researchers encounter in the early stages of a qualitative research project. Researchers and students sometimes fail to understand that adopting a qualitative approach is only the first stage in the process of selecting an appropriate research methodology.[ 7 ]

Once the initial research question has been identified, the crucial decision to be made is on the selection of an appropriate method, such as content analysis, ethnography, or grounded theory, and selecting the research design as well. Subsequent arrangements would be on the proper methods of data collection, participants, and the research setting, according to the methodology and the research question.[ 8 ] Qualitative researchers should also handle other important concerns such as data analysis, ethical issues, and rigor methods of results.

In this paper, we are going to discuss important practical challenges of qualitative inquiry in health and the challenges faced by researchers using interpretive research methodologies.

U NDERSTANDING THE R EAL N ATURE OF Q UALITATIVE R ESEARCH AND ITS C HALLENGES

It is important to provide an honest and concise appreciation of the essential characteristics of the qualitative research before discussing the challenges of the interpretive research approach to studies in health.

Virtues of qualitative research

Qualitative research does not promise a clear or direct and orderly method of tackling research problems in health studies. It does not provide researchers with a set of rules to be followed or give them a comforting sense of security and safety backup against possible mistakes on the road to knowledge. This research method depends on the “power of words and images,” but does not offer the assimilated meanings such as numbers and equations; it is rather “an attentive search of meaning and understanding” and an attempt for profound comprehension and awareness of the problems and phenomena. The essentially “diagnostic and exploratory nature” of qualitative research is invaluable in developing conceptualizations in health as an evolving discipline. It tenders the possible tap into the sea of complex interactions in health that can be as follows.

Researchers launch the quest for new theories in health which should acknowledge that “qualitative research is an approach rather than a particular set of techniques, and its appropriateness derives from the nature of the social phenomena to be explored.”[ 9 ] In qualitative research, knowledge derives from the context-specific perspective on the experienced phenomena, interpretations, and explanation of social experiences.

Why qualitative research in the health professions?

Researcher should justify the reason for which he or she selected qualitative research. Qualitative researchers pursue a holistic and exclusive perspective. The approach is helpful in understanding human experiences, which is important for health professionals who focus on caring, communication, and interaction.[ 10 ] Many potential researchers intend to find the answer to the questions about a problem or a major issue in clinical practice or quantitative research can not verify them.

In fact, they choose qualitative research for some significant reasons:

• The emotions, perceptions, and actions of people who suffer from a medical condition can be understood by qualitative research
• The meanings of health professions will only be uncovered through observing the interactions of professionals with clients and interviewing about their experience. This is also applicable to the students destined for the healthcare field
• Qualitative research is individualized; hence, researchers consider the participants as whole human beings, not as a bunch of physical compartments
• Observation and asking people are the only ways to understand the causes of particular behaviors. Therefore, this type of research can develop health or education policies; policies for altering health behavior can only be effective if the behavior's basis is clearly understood.[ 10 , 11 ]

Before adhering to a distinct research methodology, researchers have to exactly understand the nature and character of their inquiries and the knowledge they choose to create. The majority of health researchers face many loopholes in justification. However, all defects and challenges of qualitative research should be realized rather than discarded as a compelling way to knowledge structure. New endeavors in excellent academic achievement and building new tradition of qualitative research in health can be facilitated through acknowledging traps and clarifying the real practical challenges.[ 9 ]

Finally, qualitative research provides investigators with the tools to study the health phenomena from the perspective of those experiencing them. This approach is especially applied in situ ations that have not been previously studied, where major gaps exists in research field, and when there is a need for a new perspective to be identified for the arena of health care intervention.[ 6 ]

Based on corbin and strauss (2008), “ Committed qualitative researchers lean toward qualitative work because they are drawn to the fluid, evolving, and dynamic nature of this approach in contrast to the more rigid and structured format of quantitative methods. Qualitative researchers enjoy serendipity and discovery. It is the endless possibilities to learn more about people that qualitative researchers resonate to. It is not distance that qualitative researchers want between themselves and their participants, but the opportunity to connect with them at a human level (Epistemology). Qualitative researchers have a natural curiosity that leads them to study worlds that interest them and that they otherwise might not have access to. Furthermore, qualitative researchers enjoy playing with words, making order out of seeming disorder, and thinking in terms of complex relationships. For them, doing qualitative research is a challenge that brings the whole self into the process .”

Choosing an approach for health research

Researchers select approaches and methodology based on some scientific logics, not on being easy or interesting. The nature and type of the research question or problem; the researcher's epistemological stance, capabilities, knowledge, skills, and training; and the resources available for the research project are the criteria upon which adopting methodology and procedures depend.[ 6 , 10 ]

Inconsistency between research question and methodology, insufficient methodological knowledge, and lack of attention on philosophical underpinning of qualitative methodology can be mentioned as some important challenges here.

There are several different ways of qualitative research and researchers will have to select between various approaches. The qualitative research is based on the theoretical and philosophical assumptions that researchers try to understand. Then, the research methodology and process should be chosen to be consistent with these basic assumptions and the research question as well.[ 10 ]

Some researchers believe that there is no need to study the methodology and methods before beginning the research. Many researchers neglect to gain this knowledge because they are not aware of the qualitative inquiry complexities which make them go wrong. For instance, lack of information about interview, qualitative data analysis, or sampling is very common.[ 10 ]

My experience shows that lack of knowledge, experience, and skills in a research team to do qualitative research can hinder the formation of original knowledge and improvement in understanding the phenomenon under study. The result of such a study will not be new and interesting, and even the study process will be very mechanical without good interpretation or enough exploration. Sometimes there is an inconsistency between research question, research methodology, and basic philosophical assumptions, and the researchers fail to justify their methods of choice in line with the research question and the ontological and epidemiological assumptions.

Finally, the researcher's intentions, the aims of the research question/inquiry, and the chosen approach are regarded as the most important reasons to select a qualitative research method consistent with them and their underpinning philosophical assumptions as well.[ 6 , 10 ]

Research question and aim

Qualitative research is exciting because it asks questions about people's everyday lives and experiences. A qualitative researcher will have the chance of discovering the “significant truths” in the lives of people. That is a wonderful privilege, but you need to get those questions right if you dig into people's lives and ask about their real experiences. An adequate and explicit research question, or a set of interrelated questions, builds the basis for a good research. But excellent research questions are not easy to write at all. A good research requires a good research question as well because it allows us to identify what we really want to know. However, at the beginning of a project, researchers may be uncertain about what exactly they intend to know, so vague questions can lead to an unfocused project.

Common problems coming up with a research question include:

• Deciding about the research area among a range of issues that are heeded in your field of interest
• Not capable of pointing toward any interesting area or topic sufficient to focus a major piece of work on
• Knowing about the area you want to concentrate on (e.g. emergency), but not a certain topic
• Knowing what area and topic is specifically difficult to articulate a clear question.

Just make sure that you give serious consideration to the chosen area as the basis of your research and that a qualitative project is relevant and possible

Having identified a research area, your next step will be to identify a topic within that interesting area. Research questions should be derived from the literature. The research question can come from the list of “suggestions for future work” at the end of a paper you have found interesting. Moreover, you can search for some verifiable gaps through literature review, or based on your personal or professional experience and expert opinion , which should be studied. Therefore, all the previous studies that have already been conducted in the area are considered as important. In this way, you do not run the risk of asking a research question that has already been addressed and/or answered. Based on my experience, novice researchers have some problems finding the right topics in their field of interest because they do not perform a broad literature review to find the gaps and problems suitable to be investigated. Sometimes their field of interest is different from that of their supervisors or there are no experts to help them in this regard.

Although the topic may retain your interest and you may be committed to undertake such a study, it is important to recognize that some topics of personal relevance may also be deeply significant and difficult to research. Finally you need to make sure that your topic of interest is the one that you can actually study within the project constraints such as time and fund.[ 12 ]

Once you have identified your interesting topic for research (according to a broad literature review, personal and professional experience, and/or expert opinion), you can begin to create a research question.

Forming the research question is one of the initial challenges that researchers encounter in the early stages of a research project. Therefore, it acquires significance by the very fact that it provides brief, but nevertheless, important information on the research topic that allows the reader to decide if the topic is relevant, researchable, and a remarkable issue. Furthermore, the research question in qualitative studies has an additional significance as it determines the manner of conducting the study.

The qualitative research question delineates the procedures that are executed in the study and provides a map to the readers by which they can trail the researcher's intentions and actions in the study. Therefore, special attention is needed on how a qualitative research question will specifically be structured, organized, and formed in the way to quote the necessary information and elements that allow the readers to assess and evaluate the study.

The formation of a qualitative research question acquires a basic conducting role for the study and a fundamental function to develop an audit trail that can empower the readers to judge the value, rigor, and validity of the whole research project. Hence, researchers should not only pay special attention toward developing a significant and relevant question, but also formulate it properly. The qualitative research question must be provided in such a way as to impart, reflect, and conjoin the theoretical and abstract assumptions with the practical and pragmatic means of attaining them.

In plain words, a good qualitative research question implicates particular phrasing, whereas the order of words should make the topic of interest amenable to the qualitative quest.

The researcher has to concentrate on how the content of the research topic is understood when phrasing the qualitative research questions, adhering to the topic with the philosophical/theoretical suggestions and to the structure of the study which requires compounding specific principal elements.

The content of a good qualitative research question takes the form of a declarative rather than an interrogative statement

Also, the content provides a brief focus on the issue to be investigated, but does not define the exact relationship of the variables to make these relationships flexible in emanating from the study according to the qualitative research theory. The qualitative research question incepts necessarily with an active verb like understanding, exploring, interpreting, constructing, explaining, describing, etc., to reflect the paradigm/philosophy underpinning the qualitative study. Consequently, specific nouns that represent the aims of qualitative studies, such as experiences, feelings, views, perspectives, knowledge, etc., should be applied. Finally, the methodology or method should appear in the qualitative research question coherent with them. Meanwhile, the structure of a good qualitative research question will address five of the following six: who, when, where, what, how, and why, and the entire research question should devise the sixth element.[ 13 ]

For instance, “Exploring the experiences of self-immolated women regarding their motives for attempting suicide: A qualitative content analysis study in Kermanshah Iran”

Make sure that your research question is consistent with the approach you are adopting. It is like an easy trap if you decide about the research question before considering the proper way by which you are intending to make assumptions and analyze your data.

My experiences show that novice researchers formulate their research question without considering the approach of their study in a proper way and usually their research questions are very broad, unclear, and vague. Since the intention of their studies is not completely clear at the beginning, they cannot decide about the research approach; also, they have to change their research question and take different directions in the course of study or they will end up without adequate results that can help readers or consumers improve their understanding or solve the problem.

Although a researcher initiates a study with a general question and topic, the interesting aspect of qualitative research is that the questions, which are more specific and can help in further data collection and analysis, arise during the course of the study. Thus, a qualitative research question can be broadly, rather than narrowly, focused in the beginning. Researcher can try to refine and make it more focused later. This is why qualitative research is usually cyclic rather than linear. Qualitative research is cyclic, which means that the research question in this approach immerses gradually into the topic. It means that when you come to know more and more about your topic, your ideas develop about what to focus, either through reading, thinking about what you have read, or in early stages of data analysis. Finally, it is literature review, general reading, and discussion with an expert supervisor that can help you find the right topic. If the background knowledge is poor at the beginning of the study, broad but clear research question can be reasonable. Research question may become more focused or develop in a different direction according to more reading and/or preliminary data analysis. A clear and focused research question is articulated and used to conduct further analysis and any future literature reviews necessary for the final write-up.

However, it is very important to take time to choose a research question, because it can be a very challenging exercise. Actually, the ultimate success of the project depends on selecting a clear and convenient question. The question should be appropriate for the qualitative research and for the specific approach you choose which must be grounded in research. It must ask precisely what you want to find out and be articulated and clear. Knowing this will help you plan your project.[ 12 ]

Choosing the right methodology and research design

Crucial decisions need to be made about an appropriate methodology, such as ethnography or grounded theory, after identifying the initial research question. The main concern of novice researchers is to find the reason and appropriate design to do the research, and proper methodology to answer the question. Researchers ought to figure out about the planning of qualitative research and how to choose the methodology.

Researchers sometimes fail to understand that in the process of selecting an adequate research methodology, adopting a qualitative approach is only the first stage. Students, and sometimes researchers, choose qualitative research because they think it is easier to use than the other methodologies. But this reasoning is fumble since qualitative research is a complex methodology where data collection and analysis can be mostly challenging. Sometimes lack of planning and inadequate attention paid to the properness of the selected approach considering the purpose of research will be problematic.

For new qualitative researchers, it often seems that the researcher should totally concentrate on the dual process of data collection and data analysis. It is very important to consider thorough planning in all stages of the research process, from developing the question to the final write-up of the findings for publication.[ 6 ]

The research design and methodology must be adequate to address the selected topics and the research question. Researchers have to identify, describe, and justify the methodology they chose, besides the strategies and procedures involved. So, it is pivotal to find the proper method for the research question. It should be noticed that some of the details of a qualitative research project cannot be ascertained in advance and may be specified as they arise during the research process.[ 10 ] An important problem for novice researchers is the little acknowledgement of different approaches that address different kinds and levels of questions and take a different stance on the kind of phenomena which is focused upon. More discussion and debates are necessary before selecting and justifying an approach.

The need for consistency and coherence becomes more obvious when we consider the risk of something called “method-slurring.” This is the problem of blurring distinctions between qualitative approaches. Each approach has to demonstrate its consistency to its foundations and will reflect them in data collection, analysis, and knowledge claim.

It may be important to acknowledge the distinctive features by specific approaches such as phenomenology or grounded theory at some levels such as the type of question they are suited to answer, data collection methods they are consistent with, and also the kinds of analysis and presentation of the results that fit within the approach – such as “goodness of fit” or logical staged linking – and can be referred to as “consistency.”

If such consistency occurs, then the whole thing “hangs together” as coherent; that is, the kind of knowledge generated in the results or presentation section doing what is said it would do following the aims of the project. In order to consider these criteria of consistency and coherence in greater detail, we need to look at the distinctive differences between qualitative approaches in the following: the aims of the research approach, its roots in different disciplines and ideologies, the knowledge claims linked to it, and to a lesser extent, the data collection and analysis specific to each approach.[ 11 ]

My experience shows that novice researchers have some problems to justify their methodology of choice and sometimes they experience some degree of methodological slurring. They do not have any clear understanding of the research process in terms of data gathering strategies, data analysis method, and even appropriate sampling plan, which should be indentified based on philosophical and methodological principles.

Finally, besides the above-mentioned problems, regarding research design, there are two common problems encountered especially by students who want to do qualitative study; sometimes researchers and research team try to identify everything, even the sample size, in advance when they design their study because they have a strong background of quantitative research, and this is completely in contrast with the flexible nature and explorative approach of qualitative research. The other problem is the examination committee and the format of proposal of grant sites and funding agencies, which are based on the principles of quantitative study. This rigid format pushes the researchers to try to clarify everything in advance. So, flexibility is regarded as the most important credibility criterion in all kinds of qualitative research and it should be considered when designing the study and following its process.[ 1 ]

C ONCLUSIONS

Qualitative research focuses on social world and provides investigators with the tools to study health phenomena from the perspective of those experiencing them.

Identifying the research problem, forming the research question, and selecting an appropriate methodology and research design are some of the initial challenges that researchers encounter in the early stages of a qualitative research project.

Once the research problem and the initial research question are identified, the crucial decision has to be made in selecting the appropriate methodology. Subsequent arrangements would be on the proper methods of data collection, and choosing the participants and the research setting according to the methodology and the research question. It is highly recommended that the researchers exactly understand the nature and character of their inquiries and the knowledge they choose to create before adhering to a distinct research methodology based on scientific knowledge.

The essence and type of the research question or problem, the researcher's epistemological stance, capabilities, knowledge, skills and training, and the resources available for the research project are the criteria upon which the adopting methodology and procedures depend.

Inconsistency between research question and methodology, insufficient methodological knowledge, and lack of attention to the philosophical underpinning of qualitative methodology are some important challenges.

Lack of knowledge, experience, and skills to do qualitative research can hinder the formation of original knowledge and improvement in understanding the phenomenon under study. The result of such a study will not be new and interesting, and even the study process will be very mechanical without good interpretation or enough exploration. A good research requires a good research question as well because it allows us to identify what we really want to know. However, at the beginning of a project, researchers may be wavering about what they exactly intend to know; so, vague questions can lead to an unfocused project.

Broad literature review, personal and professional experience, and/or expert opinion can be regarded as the main sources to identify interesting research topics and research questions as well. Forming the research question is one of the initial challenges that researchers encounter in the early stages of a research project. Therefore, it acquires significance by the very fact that it provides brief, but nevertheless, important information on the research topic that allows the reader to decide if the topic is relevant, researchable, and a remarkable issue that can help the researcher to determine the manner of conducting the study.

Then crucial decisions need to be made about an appropriate methodology. The main concern of novice researchers is to find the reason and appropriate design to do the research and the proper methodology to answer the question. Researchers first ought to figure out the planning of qualitative research and how to choose the methodology.

It is very important to consider thorough planning in all stages of the research process, from developing the question to final write-up of the findings for publication. It is worth knowing that some of the details of a qualitative research project cannot be ascertained in advance and may be specified as they arise during the research process. For a novice researcher, more discussions and debates are necessary before selecting and justifying an approach.

Method-slurring is another common problem, which means the act of blurring distinctions between qualitative approaches. Each approach has to demonstrate its consistency to its foundations and will reflect them in data collection, analysis, and knowledge claim.

It is not rare to find that researchers and research team try to identify everything, even sample size, in advance when they design their qualitative study because of the strong background they have about the quantitative research. This is completely in contrast with the flexible nature and explorative approach of qualitative research; as these kinds of researches are completely explorative, the mentioned issues – such as sample size – should be clarified in the course of the study.

The other problem is the examination committee and the format of proposal in the grant sites and funding agencies, which is based on the principles of quantitative study. Therefore, flexibility is actually the most important credibility criterion in all qualitative researches that should be considered when a study is designed and the study process is followed.

As the final word, the researcher should make sure that he/she gives serious consideration to the chosen area as the basis of research and that a qualitative project is relevant and possible. Thus, forming the research question in a proper way and selecting appropriate methodology can guarantee original, interesting, and applied knowledge, which at least can increase our understanding about the meaning of certain conditions for professionals and patients and how their relationships are built in a particular social context.

Source of Support: Nil

Conflict of Interest: None declared.

R EFERENCES

ORIGINAL RESEARCH article

Pregnant individuals perspectives towards receiving covid-19 vaccination during their pregnancy: an in-depth qualitative study.

• 1 VU Amsterdam, Amsterdam, Netherlands
• 2 Academic Medical Center, Amsterdam, Noord-Holland, Netherlands
• 3 Onze Lieve Vrouwe Gasthuis (OLVG), Amsterdam, Netherlands

The final, formatted version of the article will be published soon.

Select one of your emails

You have multiple emails registered with Frontiers:

Notify me on publication

Please enter your email address:

If you already have an account, please login

You don't have a Frontiers account ? You can register here

Pregnant individuals have an increased risk of severe illness from coronavirus disease 2019 (COVID-19) infection. Vaccination is an effective strategy to prevent severe illness and complications for pregnant individuals. Pregnant individuals are often excluded from research and remain hesitant to receive vaccination against COVID-19. It is pivotal to study factors related to vaccine uptake and hesitancy among pregnant individuals. We studied barriers and facilitators for pregnant individuals choice and motivation regarding vaccination against COVID-19 during pregnancy to aid future pregnant individuals in their decision to vaccinate against various infectious agents. In this qualitative study, pregnant individuals were interviewed, between October 2021 and January 2022, using a semi-structured approach. A topic list was used to explore their feelings, perceptions and ideas regarding vaccination against COVID-19 during pregnancy. Interviews were transcribed verbatim and thematic analyses was performed using MAX-QDA. After nine interviews, saturation was reached. Three main themes were identified that influenced pregnant individuals choice and motivation regarding vaccination: health consequences, ambiguity of information and societal motivation. Health consequences mainly concerned the effect for their offspring, and the unknown long-term effects of COVID-19 vaccination. The advice from the Dutch institute for Public Health and Environment changed from not vaccinating pregnant individuals after release of the developed vaccine, to routinely vaccinating all pregnant individuals after research data were available from the USA. This change of policy fueledfuelled doubt and confusion for vaccination. Arguments in favor of vaccination from the social perspective were specific behaviorbehaviour rules and restrictions due to the pandemic. E.g. without vaccination people were unable to travel abroad and having to take a COVID-19 test every time entering a public place.In conclusion, pregnant individuals need clear, unambiguous information concerning health consequences, short-and long-term, particularly for their offspring, in the decision-making process regarding COVID-19 vaccination. Additionally, the societal perspective needs to be addressed. Besides the aforementioned themes, general counselingcounselling should focus on misperceptions of vaccine safety and the role of misinformation which are also important in the non-pregnant population. This study underlines the importance of including pregnant individuals in research programs to obtain specific information targeted to their needs.

Keywords: Pregnancy, Pregnant individuals, COVID-19, Vaccine hesitancy, SARS-CoV-2

Received: 10 Apr 2024; Accepted: 29 Jul 2024.

Copyright: © 2024 Zilver, Rietveld, Schonewille, Bakker, Broekman, Van Leeuwen and de Groot. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Sanne J. Zilver, VU Amsterdam, Amsterdam, Netherlands

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.