Category:Text Analysis

of

Related Videos

RLogo.png

The initial stage of most text analyses is to do some pre-processing of the text, which includes breaking the text up into separate words and doing some basic cleaning. These steps are performed by Create > Text Analysis > Setup Text Analysis . This creates an output in the Report which can then be used as an input to the other analyses from the Techniques section of the menu. The text processing is designed to be used with English-language text.

Word Cloud - Create a Word Cloud from an existing Text question . No setup is required, although you can make use of your processing by first using the Save Tidied Text option listed below.

Setup Text Analysis - Cleans text data for further analysis. Apply spell-checking, stemming, and manual replacement and removal of words.

The Term Document Matrix is particularly useful as an input to other statistical algorithms, by writing your own R code .

Subcategories

This category has only the following subcategory.

  • ► Coding ‎ (9 P)

Pages in category 'Text Analysis'

The following 18 pages are in this category, out of 18 total.

  • Coding in Older Versions of Q
  • Manipulating Text with JavaScript
  • R Packages for Text Analysis
  • Text Analysis - Advanced - Map
  • Text Analysis - Advanced - Predictive Tree
  • Text Analysis - Advanced - Principal Components Analysis (Text)
  • Text Analysis - Advanced - Save Variable(s) - Tidied Text
  • Text Analysis - Advanced - Search
  • Text Analysis - Advanced - Setup Text Analysis
  • Text Analysis - Advanced - Term Document Matrix
  • Text Analysis - Automatic Categorization - Entity Extraction
  • Text Analysis - Automatic Categorization - List of Items
  • Text Analysis - Automatic Categorization - Unstructured Text
  • Text Analysis - Save Variable(s) - Categories
  • Text Analysis - Save Variable(s) - First Category
  • Text Analysis - Sentiment
  • Text Analysis Case Study - Trump's Tweets
  • Visualization - Word Cloud - Word Cloud

Navigation menu

Insight Platforms Logo - Wordmark

Q Research Software

Q Research Software is analysis software designed to help market researchers quickly find the story in their survey data.

Q Research Software Overview

Q Research Software is analysis and reporting software designed to help market researchers quickly find the story in their survey data.

Automates the grunt work – Everything that can be automated is automated, including cleaning & formatting data, statistical testing, generating tables, updating analyses and reproducing reports.

DIY advanced analysis and visualization – With the latest machine learning & statistical techniques, Q uses smart systems and automation for widely used methods including correspondence analysis, latent class analysis, regression, MaxDiff, Shapley, and TURF. It also fully supports the R language.

Share insight-filled stories – Visualize information to show patterns that matter and design reports that make storytelling effortless with seamless Microsoft Office and Displayr integration.

———–

“It’s so user-friendly that you can design a MaxDiff in Q even if you have no concept of analysis whatsoever. With the warnings that pop up, Q holds your hand as you’re designing it.” Eddie Sheppard, Vice President, Calgary Office, Leger

Q Research Software Categories

expirable direct uploads 2F1a877944 9060 4fbb bbae dca4f8952fca 2Fscreen2 768x432

Q Research Software Products & Services

Q research software featured content.

q research software text analysis

Change Location

Find awesome listings near you.

Analyzing Text Data

  • Overview of Text Analysis and Text Mining

Choosing a Method

How much data do you need, word frequency analysis, machine learning/natural language processing, sentiment analysis.

  • Library Databases
  • Social Media
  • Open Source
  • Language Corpora
  • Web Scraping
  • Software for Text Analysis
  • Text Data Citation

Library Data Services

Choosing the right text mining method is crucial because it significantly impacts the quality of insights and information you can extract from your text data. Each method provides different insights and requires different amounts of data, training, and iteration. Before you search for data, it is essential that you:

  • identify the goals of your analysis
  • determine the method you will use to meet those goals
  • identify how much data you need for that method
  • develop a sampling plan to build a data set that accurately represents your object of study.

Starting with this information in mind will make your project go more quickly and smoothly, and help you overcome a lot of hurdles such as incomplete data, too much or too little data, or problems with access to data.

More Resources:

  • Content Analysis Method and Examples, from Mailman School of Public Health, Columbia University
  • Qualitative Research Methods Overview, from Northeastern University

Before you start collecting data, think about how much data you really need. New researchers in text analysis often want to collect every source mentioning their topic, but this is usually not the best approach. Collecting so much data takes a lot of time, uses many computational resources, often goes against platform terms of service, and doesn't necessarily improve analysis.

In text analysis, an essential idea is saturation , where adding more data doesn't significantly improve performance. Saturation is when the model has learned as much as it can from the available data, and no new patterns are themes are emerging with additional data. Researchers often use experimentation and learning curves to determine when saturation occurs; you can start by analyzing a small or mid-sized dataset and see what happens if you add more data.

Once you know your research question, the next step is to create a sampling plan . In text analysis, sampling means selecting a representative subset of data from a larger dataset for analysis. This subset, called the sample, aims to capture the diversity of sentiments in the overall dataset. The goal is to analyze this smaller portion to draw conclusions about the information in the entire dataset.

For example, in a large collection of customer reviews, sampling may involve randomly selecting a subset for sentiment analysis instead of analyzing every single review. This approach saves computational resources and time while still providing insights into the overall sentiment distribution of the entire dataset. It's crucial to ensure that the sample accurately reflects the diversity of sentiments in the complete dataset for valid and reliable generalizations.

Example Sampling Plans

Sampling plans for text analysis involve selecting a subset of text data for analysis rather than analyzing the entire dataset. Here are two common sampling plans for text analysis:

Random Sampling:

  • Description: Randomly select a subset of text documents from the entire dataset.
  • Process: Assign each document a unique identifier and use a random number generator to choose documents for inclusion in the sample.

Stratified Sampling:

  • Description: Divide the dataset into distinct strata or categories based on certain characteristics (e.g., product types, genres, age groups, race or ethnicity). Then, randomly sample from each stratum.
  • Process: Divide the dataset into strata, and within each stratum, use random sampling to select a representative subset.

Remember, the choice of sampling plan depends on the specific goals of the analysis and the characteristics of the dataset. Random sampling is straightforward and commonly used when there's no need to account for specific characteristics in the dataset. Stratified sampling is useful when the dataset has distinct groups, and you want to ensure representation from each group in the sample.

Exactly How Many Sources do I need?

Determining the amount of data needed for text analysis involves a balance between having enough data to train a reliable model and avoiding unnecessary computational costs. The ideal dataset size depends on several factors, including the complexity of the task, the diversity of the data, and the specific algorithms or models being used.

  • Task Complexity:  If you are doing a simple task, like sentiment analysis or basic text classification, a few dozen articles might be enough. More complex tasks, like language translation or summarization, often require datasets on the scale of tens of thousands to millions.
  • Model Complexity:  Simple models like Naive Bayes often perform well with smaller datasets, whereas complex models, such as deep learning models with many parameters, will require larger datasets for effective training.
  • Data Diversity:  Ensure that the dataset is diverse and representative of the various scenarios the model will encounter. A more diverse dataset can lead to a more robust and generalizable model. A large dataset that is not diverse will yield worse results than a smaller, more diverse dataset.
  • Domain-Specific Considerations:  Sometimes there is not a lot of data available, and it is okay to make do with what you have!

Start by taking a look at articles in your field that have done a similar analysis. What approaches did they take? You can also schedule an appointment with a Data Services Librarian to get you started.

More Readings on Sampling Plans for Text Analysis:

  • How to Choose a Sample Size in Qualitative Research, from LinkedIn Learning  (members of the GW community have free access to LinkedIn Learning using their GW email account)
  • Sampling in Qualitative Research , from Saylor Academy
  • Lowe, A., Norris, A. C., Farris, A. J., & Babbage, D. R. (2018). Quantifying Thematic Saturation in Qualitative Data Analysis. Field Methods, 30(3), 191-207. https://doi.org/10.1177/1525822X17749386

Word Frequency Analysis of Coffee and Tea from the HathiTrust Database. Coffee is more common than tea after 1907.

Software for Word Frequency Analysis

  • NVivo via GW's Virtual Computer Lab NVivo is a software package used for qualitative data analysis. It includes tools to support the analysis of textual data in a wide array of formats, as well as and audio, video, and image data. NVivo is available through the Virtual Computer Lab. Faculty and staff may find NVivo available for download from GW's Software Center.
  • Analyzing Word and Document Frequency in R This chapter explains how to use tidy to analyze word and document frequency using Tidy Data in R.
  • word clouds in R R programming functionality to create pretty word clouds, visualize differences and similarities between documents, and avoid over-plotting in scatter plots with text.
  • ATLAS.ti Trial version of qualitative analysis workbench for processing text, image, audio, and video data. (Note: Health science students may have access to full version through Himmelfarb Library)

Related Tools Available Online

  • Google ngram Viewer When you enter phrases into the Google Books Ngram Viewer, it displays a graph showing how those phrases have occurred in a corpus of books (e.g., "British English", "English Fiction", "French") over the selected years.
  • HathiTrust This link opens in a new window HathiTrust is a partnership of academic and research institutions, offering a collection of millions of titles digitized from libraries around the world. To log in, select The George Washington University as your institution, then log in with your UserID and regular GW password.
  • Voyant Voyant is an online point-and-click tool for text analysis. While the default graphics are impressive, it allows limited customizing of analysis and graphs and may be most useful for exploratory visualization.

Related Library Resources

  • HathiTrust and Text Mining at GWU HathiTrust is an international community of research libraries committed to the long-term curation and availability of the cultural record. Through their common efforts and deep commitment to the public good, the libraries support the teaching and learning activities of the faculty, students or researchers at their home institutions, and the scholarly needs of the broader public as well.
  • HathiTrust+Bookworn From the University of Illinois Library: HathiTrust+Bookworm is an online tool for visualizing trends in language over time. Developed by the HathiTrust Research Center using textual data from the HathiTrust Digital Library, it allows you to track changes in word use based on publication country, genre of works, and more.
  • Python for Natural Language Processing A workshop offered through GW Libraries on natural language processing using Python.
  • Text Mining Tutorials in R A collection of text mining course materials and tutorials developed for humanists and social scientists interested to learn R.
  • Oxford English Dictionary This link opens in a new window The Oxford English Dictionary database will provide a word frequency analysis over time, drawing both from Google ngrams and the OED's own databases.

Example Projects Using Word Frequency Analysis

Robinson, J. S. and D. (n.d.). 3 Analyzing word and document frequency: Tf-idf | Text Mining with R . Retrieved November 21, 2023, from https://www.tidytextmining.com/tfidf.html

  • Zhang, Z. (n.d.). Text Mining for Social and Behavioral Research Using R . Retrieved November 21, 2023, from https://books.psychstat.org/textmining/index.html
  • Exploring Fascinating Insights with Word Frequency Analysis In the realm of data analysis, words hold immense power. They convey meaning, express ideas, and shape our understanding of the world. In this article, we’ll explore the fascinating world of textual data analysis by examining word frequencies. By counting the occurrence of words in a text, we can uncover interesting insights and gain a deeper understanding of the underlying themes and patterns. Join us on this word-centric journey as we dive into the realm of word frequency analysis using Python.

Machine learning for text analysis is a technology that teaches computers to understand and interpret written language by exposing them to examples. There are two types of machine learning for text analysis: supervised learning, in which a human helps to train the computer to detect patterns, and unsupervised learning, which enables computers to automatically categorize, analyze, and extract information from text without needing explicit programming.

One type of machine learning for text analysis is  Natural Language Processing (NLP).  NLP for text analysis is a field of artificial intelligence that involves the development and application of algorithms to automatically process, understand, and extract meaningful information from human language in textual form. NLP techniques are used to analyze and derive insights from large volumes of text data, enabling tasks such as sentiment analysis, named entity recognition, text classification, and language translation. The aim is to equip computers with the capability to comprehend and interpret written language, making it possible to automate various aspects of text-based information processing.

Software for Natural Language Processing

  • NLTK for Python NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries, and an active discussion forum.
  • scikit Simple and efficient tools for predictive data analysis, using Python.

Related Resources Available Online

  • [Large Language Models] LLMs in Scientific Research
  • HathiTrust and Text Mining at GWU Information on text data mining using HathiTrust
  • Social Feed Manager Social Feed Manager software was developed to support campus research about social media including Twitter, Tumblr, Flickr, and Sina Weibo platforms. It can be used to track mentions of you or your articles and other research products for the previous seven days and on into the future.. Email [email protected] to get started with Social Feed Manager or to schedule a consultation

Example Projects using Natural Language Processing

  • Redd D, Workman TE, Shao Y, Cheng Y, Tekle S, Garvin JH, Brandt CA, Zeng-Treitler Q. Patient Dietary Supplements Use: Do Results from Natural Language Processing of Clinical Notes Agree with Survey Data?  Medical Sciences . 2023; 11(2):37. https://doi.org/10.3390/medsci11020037
  • Nguyen D, Liakata M, DeDeo S, Eisenstein J, Mimno D, Tromble R, Winters J. How We Do Things With Words: Analyzing Text as Social and Cultural Data. Front Artif Intell. 2020 Aug 25;3:62. doi: 10.3389/frai.2020.00062. 

Sentiment analysis is a method of analyzing text to determine whether the emotional tone or sentiment expressed in a piece of text is positive, negative, or neutral. Sentiment analysis is commonly used in businesses to gauge customer feedback, social media monitoring, and market research.

Software for Sentiment Analysis

  • Sentiment Analysis using NLTK for Python NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries, and an active discussion forum.
  • Sentiment Analysis with TidyData in R This chapter shows how to implement sentiment analysis using tidy data principles in R.
  • Tableau Tableau works with numeric and categorical data to produce advanced graphics. Browse the Tableau public gallery to see examples of visuals and dashboards. Tableau offers free one-year Tableau licenses to students at accredited academic institutions, including GW. Visit https://www.tableau.com/academic/students for more about the program or to request a license.
  • Qualtrics Text iQ Qualtrics is a powerful tool for collecting and analyzing survey data. Qualtrics Text iQ automatically performs sentiment analysis on collected data.

Related Resources Available Online

  • finnstats. (2021, May 16). Sentiment analysis in R | R-bloggers. https://www.r-bloggers.com/2021/05/sentiment-analysis-in-r-3/

Example Projects Using Sentiment Analysis

  • Duong, V., Luo, J., Pham, P., Yang, T., & Wang, Y. (2020). The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic. 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 126–130. https://doi.org/10.1109/ASONAM49781.2020.9381379
  • Ali, R. H., Pinto, G., Lawrie, E., & Linstead, E. J. (2022). A large-scale sentiment analysis of tweets pertaining to the 2020 US presidential election. Journal of Big Data, 9(1). https://doi.org/10.1186/s40537-022-00633-z
  • << Previous: Overview of Text Analysis and Text Mining
  • Next: Finding Text Data >>
  • Last Updated: Jul 10, 2024 4:08 PM
  • URL: https://libguides.gwu.edu/textanalysis

Root out friction in every digital experience, super-charge conversion rates, and optimize digital self-service

Uncover insights from any interaction, deliver AI-powered agent coaching, and reduce cost to serve

Increase revenue and loyalty with real-time insights and recommendations delivered to teams on the ground

Know how your people feel and empower managers to improve employee engagement, productivity, and retention

Take action in the moments that matter most along the employee journey and drive bottom line growth

Whatever they’re are saying, wherever they’re saying it, know exactly what’s going on with your people

Get faster, richer insights with qual and quant tools that make powerful market research available to everyone

Run concept tests, pricing studies, prototyping + more with fast, powerful studies designed by UX research experts

Track your brand performance 24/7 and act quickly to respond to opportunities and challenges in your market

Explore the platform powering Experience Management

  • Free Account
  • Product Demos
  • For Digital
  • For Customer Care
  • For Human Resources
  • For Researchers
  • Financial Services
  • All Industries

Popular Use Cases

Customer Experience

Employee experience.

  • Net Promoter Score
  • Voice of Customer
  • Customer Success Hub
  • Product Documentation
  • Training & Certification
  • XM Institute
  • Popular Resources
  • Customer Stories
  • Artificial Intelligence

Market Research

  • Partnerships
  • Marketplace

The annual gathering of the experience leaders at the world’s iconic brands building breakthrough business results, live in Salt Lake City.

  • English/AU & NZ
  • Español/Europa
  • Español/América Latina
  • Português Brasileiro
  • REQUEST DEMO
  • Experience Management
  • Text Analysis
  • What is text analysis?
  • Text analysis vs text mining

The importance of text analysis

  • Business applications of text analysis

Topic modeling in text analysis

  • How to perform topic modeling
  • Best practice for topic modeling

Accuracy in text analysis

  • Analyzing text in multiple languages
  • Text analysis tools

How Qualtrics can help

  • Text analysis in customer experience

Try Qualtrics for free

Text analysis: definition, benefits & examples.

45 min read Find out how text analysis software works, and how you can use it to find breakthrough insights in unstructured data to take your customer, employee, brand, and product experience programs to another level. Written by Rohan Sinha, Senior Principal CX at Qualtrics.

Text feedback is the closest we ever get to a 1:1 conversation with every customer, every citizen, and every employee. In free text, our customers get to tell us what they really care about and why, unconstrained by the questions we decided to ask them. It’s where the customers get to decide what’s most important.

However, internalizing ten thousand pieces of feedback is roughly equal to reading a novel and categorizing every sentence. It’s time-consuming, laborious, and hard to make text actionable. To effectively understand open-text feedback at scale, you need to either scale your team reading feedback or use a text analytics tool to surface the most important pieces and themes of feedback. Let’s go through the basics of text analysis together, and give you some helpful tools to consider using.

Learn about Text iQ – our text analysis software

What is text analysis / analytics?

Text analysis is the process by which information is a­utomatically extracted and classified from text data. Within the field of Experience Management text could take the form of survey responses , emails, support tickets, call center notes, product reviews, social media posts, and any other feedback given in free text, as opposed to a multiple-choice format. Text analytics enables businesses to discover insights from within this unstructured data format.

Text analytics can help you answer two core questions:

  • How are you performing on the topics you know about like wait time, service reliability, and cost?
  • What’s lurking out there that you didn’t even think to look for like bugs in software, confusing onboarding process, or flaws in your product?

A powerful text analytics program can answer both of these – at scale – while keeping you connected to the voice of your customer and the next actions to take.

The two most widely used techniques in text analysis are:

  • Sentiment analysis — this technique helps identify the underlying sentiment (say positive, neutral, and/or negative) of text responses
  • Topic detection/categorization — this technique is the grouping or bucketing of similar themes that can be relevant for the business & the industry (eg. ‘Food quality’, ‘Staff efficiency’ or ‘Product availability’)

Both techniques are often used concurrently, giving you a view not only of what topics people talk about but also whether they talk positively or negatively when they talk about such topics.

These are broad techniques that encompass all other different ways of identifying emotions , intent, etc. It’s worth mentioning that some software claims to do emotion analysis from text — these tend to use the combination of words used in the text to arrive at the emotion.

This can be rather misleading because one could say “The flight was delayed” with anger, despair, joy (if they did something exciting at the airport), etc. but the text would never show the tonality or the expression behind the sentence.

Hence, using a combination of topics and sentiment from the words is the only way to ascertain emotion, rather than a ‘catch all’ algorithm.

Top text topics by sentiment report

Text analysis, text mining, and natural language processing (NLP) explained

It’s common when talking about text analysis to see key terms like text mining and text analysis used interchangeably — and often there’s confusion between the two.

There is a lot of ambiguity in the differences between the two topics, so it’s perhaps easier to focus on the application of these rather than their specific definitions.

Text Mining is a technical concept that involves the use of statistical techniques to retrieve quantifiable data from unstructured text which can then be used for further applications, for example, MIS reporting, regulatory non-compliance, fraud detection, or job application screening. Quantitative text analysis is important, but it’s not able to pull sentiment from customer feedback.

Text Analysis on the other hand is a very business-focussed concept that involves the use of similar techniques as text mining but enhances them, identifying patterns, insights, sentiment, and trends for customer or employee experience programs . Text analysis focuses on insights discovery for action taking within specialized fields like experience management.

As part of text analysis, there’s also natural language processing (NLP) , also termed natural language understanding. It’s a form of sentiment analysis that helps technology to “read” or understand text from natural human language. Natural language processing algorithms can use machine learning to understand and evaluate valuable data, consistently and without any bias. It can be sophisticated enough to understand the context of text data, even with complicated concepts and ambiguities.

Hence, it is very important to use specialized text analytics platforms for Voice of the Customer or Employee data as opposed to general text mining tools available out there.

See how Qualtrics Text iQ uses sentiment analysis

Text analysis has become an important part of many business intelligence processes, particularly as part of experience management programs as they look for ways to improve their customer , product , brand , and employee experiences .

Before text analysis, most businesses would need to rely on quantitative survey data in order to find areas where they can improve the experience.

However, while still essential to any program, quantitative data has its limitations in that it’s restricted to a predetermined set of answers.

For example, a telecoms company may ask a typical customer satisfaction or CSAT question after a support call – ‘How satisfied were you with the service you received?’.

A follow-up question on customer surveys might look to find out the reasons behind the customer satisfaction score and might have options like:

  • Waiting time
  • Speed of resolution
  • Advisor Attitude

These options are limited and hence restrict the analysis that one can do for the scores. For example, if the customer’s reason is not listed in those options, then valuable insight will not be captured.

It would be almost impossible to list every possible reason in a customer survey, so including open text feedback helps to dig deeper into the experience.

This is where text analysis is crucial to identify the unknown unknowns — the themes the business does not know about but could be driving dissatisfaction with customers.

A better alternative is asking an open-ended question on the reasons for the score – ‘Why did you give us that score?’

Using survey text analysis techniques on that open-ended response then enables organizations to understand the topics customers mention when they are dissatisfied, but also helps in identifying extremely negative topics versus not so negative ones.

By being able to ask customers to say in their own words why they were or weren’t satisfied with the experience, you can better pinpoint customer insights. Text analytics helps you to be much more specific about the actions you need to take to improve their experience .

Being able to drive correlations between structured and unstructured data provides extremely powerful information on clear action taking.

It could be that there’s a strong correlation between people who talk about staff giving a clear explanation of the next steps and high CSAT, or between those who talk about the staff having a good knowledge of the product and high CSAT.

And with text analysis techniques, that data can be easily organized and fed into your experience management program in the same way as quantitative data in order to give you deeper insights into what drives the customer, employee, brand, or product experience.

By being able to see what people talk about when they talk in their own words about an experience and being able to perform sentiment analysis and topics in real-time, you can identify improvements that would otherwise have gone unnoticed using only qualitative data.

Business applications of text analytics

Text analysis is used in several different ways within experience management (XM) — if we break out XM into 4 pillars, we can see some of the most common use cases below:

  • Increasing Loyalty – Unearth key insights on the top issues promoters face so action can be taken to stop promoters from becoming detractors .
  • Preventing Churn – Smart identification of competitor mentions with highly dissatisfied customers. You can also use text analytics to close the loop on negative sentiment or key topics like Churn Potential that appear in your customer feedback .
  • Cross Sell/Up Sell – Combining Operational data like Customer Spend or Lifetime Value with upcoming renewal dates & analysis on their comments for topics like Loyalty, Reward, Incentives, etc. it’s possible to predict cross-sell potential using a combination of AI & text analysis.
  • Employee Attrition – Combine structured data like Engagement scores with Low sentiment around topics like Manager Support etc.
  • Employee Wellbeing – Using real-time alerts on topics like Depression & Anxiety, intervention can be done where required.
  • Work-Life Balance – Using text analysis to understand topics around Work-life Balance, identifying which segments of employees are most affected, and taking action accordingly.

Product Experience

  • New Product Launch – Using text analysis to get valuable feedback on what features to improve or drop in the next release .
  • Product Usage – Analysing warranty data can give key insights on what features to invest more in to increase usage, reduce service costs, etc.

Brand Experience

  • Campaign Effectiveness – Analyse the top drivers of satisfaction for your campaigns alongside Operational data like Campaign Spend, Reach, etc to ascertain ROI.
  • Brand Tracking – Understanding top themes that come to mind for the Brand & competitors.

In text analytics, one of the most common techniques of providing structure to this data is a process known as topic modeling (sometimes referred to as categorization or taxonomy structures.)

Here we’ll explore what it is, how it works, and how to use it when analyzing text responses in multiple languages.

What is a topic model?

‘Topics’ or ‘categories’ refer to a group of similar concepts or themes in your text responses.

Say for example a utility company customer says “ The dual tariff is expensive ” while another says “ The dual pricing package is expensive ”, while the words they’re using are different (‘tariff’ vs ‘pricing package’) they are both referring to the same topic.

As such, both comments can be grouped under the topic ‘Tariff type’.

Topic modeling is a process that looks to amalgamate different topics into a single, understandable structure. It is possible to have a single-layer topic model, where there are no groupings or hierarchical structures, but typically they tend to have multiple layers.

This type of Parent-Child topic grouping is usually referred to as the Taxonomy, which involves grouping topics into broader concepts that make sense for a particular business.

Common examples could be a parent topic such as ‘Staff attributes’ that contain various children topics (or subtopics) such as ‘staff attitude’, ‘staff efficiency’, and ‘staff knowledge’.

Sentiment by topic financial services report

The taxonomy is essential in Experience Management because it can be used for reporting to relevant stakeholders and routing feedback to the right teams and departments that can act on the insights.

For example in a Hotel business, the category ‘Staff Experience’ might be relevant for the Hotel Manager from a training perspective, while the Room Experience may be of specific interest to the Housekeeping Manager.

Having a taxonomy is essential in order to get the right insights, to the right people across the organization.

Key elements of topic modeling in text analysis

Number of layers.

A topic model could have many tiers or hierarchical levels. However, it is best practice in Experience Management to restrict the model to two layers. Anything over two layers becomes extremely complex to understand and navigate for a business user, but more importantly, it is very tedious to build and maintain over time.

Exclusive Topics

Another basic concept in topic modeling is the possibility of having multiple topics for the same sentence or response. This means topics need to be mutually non-exclusive. For example, “My baggage loss was a cause of extreme frustration.” could be categorized under two sentiment analysis topics at the same time – ‘Lost Baggage’ and ‘Emotion — Frustration’.

Multi language

The topic model must be able to apply to all languages your business operates in. This means the model should be able to capture multilingual verbatims under the respective topics in your model. For example, if, a customer in London says “long queue at the branch for withdrawing cash using a cheque” while a customer in Paris says “longue file d’attente à la succursale pour retirer de l’argent en utilisant un chèque”, the topic model should be able to capture both pieces of feedback under its topic for ‘Branch Experience – Waiting Time’. So from a reporting perspective, there is consistency in the single model being used.

Learn more about text analysis in multiple languages

How to model topics for text analysis

In our experience, there are two ways to do topic modeling in an Experience Management program:

  • Bottom up — the underlying dataset informs the topics being built
  • Top down — topics are prescribed independently from the dataset

Bottom-up topic modeling in text analysis

When talking about topic modeling, you’ll often see plenty of jargon (‘bag of words’, ‘ngrams’, and ‘vectors’ being some of our favorites!) but for the purposes of this article, we’ve kept things simple with three main ways to build your topics based on an existing dataset.

Machine learning algorithms — this is a common feature in good text analysis software, and it often uses a reference dataset to come up with topics. These reference datasets are usually created using publicly available text data like research articles, media content, or blogs. While this is great from a linguistic perspective, it may be not helpful when you are using it to formulate topics for a VOC program or an Employee Experience program. So while it’s a useful method, you should be cautious of using learning algorithms alone to develop your topic model.

Statistical techniques — advanced statistical analysis like clustering can be used to suggest top keywords or combinations used based on their occurrence or frequency. While this approach is rudimentary, it makes a lot of sense when looking at experience data using text analysis techniques — especially if you think about specific touchpoints in the Customer Experience that are both specific and tend to have a larger volume of data.

Manual query — the simplest, and also a very effective way of bottom-up topic building approach is to formulate topics manually based on the word count of different words used in the dataset. This may sometimes be discarded as labor-intensive, inefficient, and archaic. However, there are many simple techniques that can be used to expedite this process and make it very relevant for your dataset.

Top-down topic modeling in text analysis

This type of modeling is a much more prescriptive way to build your model and there are typically two main methods:

  • Industry models. You can apply pre-built taxonomy models and a lot of text analysis software offers both industry and horizontal models based on their experience with other clients with similar use cases). This is a good way to start the topic modeling exercise, however, it’s important you don’t rely solely on the pre-built model. Companies within the same industries have many different ways or nuances of doing business and their customers will use totally different terminology to refer to the products, services, or promotions which could be very unique in each case. It’s important too that you’re able to check the recall on the model, in case this is a standard approach for a text analysis vendor.
  • Manual queries based on domain experience. This is very similar to the manual approach suggested in bottom-up modeling, except that it’s more prescriptive in nature and is based on the experience and bias of the user building the model. Examples include duplicating topics from a historical taxonomy model, or an experienced business user dictating the topics they know their customers refer to.

Our best-practice approach to modeling topics for text analysis

We’ve looked at the pros and cons of each approach, and when it comes to your own modeling for text analytics purposes, we’d recommend a combination of them to be most effective.

Whether it’s customer experience or employee feedback data, the following steps could give you the best topic model in an efficient way.

Step 1: Top Down with a pre-built model

Say your team has 100k verbatims from a particular customer touchpoint and you need to provide an analysis on all the topics in the data. The fastest way to apply a model and get a head start is by using a pre-built model. This could be done in 2 ways:

  • Industry Models – The text analysis software should be able to give you options of using pre-built horizontal/vertical models to select from within your project area.

Topic import by industry / theme

  • Pre-configured models – Using a model that had been configured for a similar use-case somewhere else in the organization historically. This is extremely important that the text software provides the capability to use models from other projects. Which could even be a simple export of a pre-configured model from one project into an exportable file, and then importing the file in the new project where the analysis needs to be done. Don’t worry too much about the precision of the topics at this stage.

Import preview screenshot

Step 2: Bottom Up – Automatic Topic Detection

Most text analysis software should be able to detect themes on the dataset or automatically pick up topics from the dataset based on whatever learning or clustering ability it uses.

While you should never fully rely on the automatic topic recommendations, they are a useful second step to bolster the model you’ve used in step one. Once you’ve got your recommendations, it’s very important to go through the automatically generated topics and add the ones that seem interesting, to the existing model.

Topic recommendations screenshot

Step 3: Enhance Precision

The Pre-built model plus the auto-generated topics now need a precision tweak. Go through each topic to check if it’s capturing/tagging the right responses. For the first pass, we would advise checking at least 15 to 20 verbatim responses per topic to get a good level of precision.

It’s imperative here that whatever text analysis software you’re using provides an easy user interface that allows you to:

  • Easily select the topics and check the recall on each one
  • Check the rules for each topic
  • Check the verbatims each topic is part of speech tagging
  • Make changes to the rules and check the changes the edit has made to the count of verbatim tagged

Step 4: Increase the Recall.

The final, and arguably most important, step is to increase the recall on the model and make it more effective by manually tweaking it to increase the total percentage of comments that have at least one topic association.

There are two approaches here:

1. Improve existing topics — the existing topics in the model may need to include more similar words or synonyms to increase the frequency/count or verbatim for that topic. To do that, you need to include more words in your existing topic rules — this process could involve significant manual reading and be very time-consuming. Machine learning can help in this process by providing suggestions of word mentions similar to the ones already used in the topic, hugely expediting the process if the software you’re using has it available.

Staff attributes - staff professionalism related terms

2. Create more topics to capture verbatims from the Untagged/Unknown Bucket — a true bottom-up approach will start from the verbatims and use them to build the model. But who will read 10,000 individual pieces of feedback? Instead, use techniques like lemmatized word cloud reports. These clearly show the most frequently mentioned words in the dataset and, when the report is filtered for the ‘unknown’ bucket, you can see the most mentioned words in that section. This gives you an easy view of which of the words the model has left out, so you can identify which should be assigned to different topics, or indeed if a new topic needs creating.

Verbatim topics word cloud

In order to make decisions and take actions based on data, you need to have confidence in that structured or unstructured data. As such, many people obsess over the accuracy of their text analytics.

It does indeed matter, but there are many instances where accuracy can be a red herring, particularly in VOC and other XM programs where signals from text analysis are vital, regardless of their accuracy.

How is text analysis accuracy measured?

When talking about accuracy, it’s important to remember that it will depend on a wide variety of factors, including:

  • the source of text data (eg. Tweets, Product reviews, chat transcripts, etc.)
  • the complexity of the language in the industry you’re in
  • regional and cultural influences, for example, introducing concepts like sarcasm
  • the length & complexity of the sentences used by respondents

Accuracy in text analysis is usually measured using two concepts – recall and precision.

  • Recall is the number of correct results divided by the number of results that should have been returned. 80% recall means that 20% of your data has not been captured by the analysis at all, and has not been tagged in any category or topic.
  • Precision is the number of correct results divided by the number of all returned results. 80% precision means that 20% of your data has been incorrectly included in the model.

In Customer Experience and Voice of the Customer programs, recall and coverage are usually measured as the percentage of records that are actually tagged under at least 1 topic in the taxonomy model.

For example, in a customer feedback data set of 100 verbatims for a Telecom provider, we know 70 verbatims refer to the various Tariff Plans available for the customers.

The text analysis model pulls 50 verbatims as relevant for ‘Tariff Plans’.

And of those 50, only 45 correctly contain mentions of ‘Tariff Plans’.

Recall and precision - false and true negatives and positives

In this example: True Positives: 45 False Positives: 50 – 45 = 5 False Negatives: 70 – 45 = 25 True Negatives: 30 – 5 = 25

Recall = TP/ (TP+FN) = 45/70 or 64% Precision = TP / (TP+FP) = 45/50 or 90%

To combine the 2 under 1 score, the statisticians use the F Score. The F1 score is the harmonic mean of precision and recall taking both metrics into account.

F1=2 * Precision*Recall/ Precision + Recall

How accurate does your text analysis need to be?

Now that we understand the concept of accuracy, it’s also useful to understand the dangers of being pedantic about accuracy in text analysis, particularly when it comes to experience management programs like voice of the customer.

There are three main challenges with accuracy calculations:

1. Large datasets present a challenge

Accuracy is a statistical concept and can be very difficult to ascertain in big datasets, say for example where you are applying text analysis techniques to millions of customer feedback records.

2. It takes a lot of legwork

Understanding accuracy relies on sophisticated methods and calculations, and some even use probabilistic calculations to get there. In order to use True Positives and False Negatives to understand your accuracy score, you need up-to-date information about what’s correct, and what’s not. This can only be done by manually tagging the data, and can become a very cumbersome process, even when the analysis itself is done via machine learning.

3. It’s impractical with multiple topics

To understand accuracy, most people look at the recall of the taxonomy or the topic model. For example, if you have 10,000 pieces of verbatim feedback, and your multi-tier (taxonomical/hierarchical) topic model covers tags 8,500 of those as containing at least one of the topics in the model, then we would consider the recall is 85%.

However, the recall calculation in our example above (Tariff Plan) is actually done for just one topic. But what if our telco had 30 topics? The true recall model would be to see the recall of each & every topic or category node within the model – and this is where it runs into difficulty.

Say a piece of text feedback that says “Pay as You Go plan is great but the staff was unhelpful”, was actually tagged under the topic ‘Staff Helpfulness’ topic but not under ‘Tariff Plan’, by the Topic Model level recall calculation, the recall will be 100%. However, if you do the same analysis at the level of Tariff Plan, the Recall is 0.

Topic and sentiment analysis report telco

Accuracy of sentiment analysis in text analytics

We’ve looked at some of the challenges of accuracy in topic analysis, but there are challenges in sentiment analysis too:

Irony & sarcasm

When people express negative emotions using positive words, it becomes challenging for sentiment models. There are different ways to spot these using rule-based or learning-based methods. Rule-based methods however are limited for this approach as they can only catch as many that there are rules for. Learning-based models which use massive reference datasets are more likely to return better accuracy.

The great news, however, is that in a multi-channel Customer Experience program , generally, such instances would be far less than even 0.5% of your overall VOC data.

This refers to the use of ‘flippers’ or negator words like ‘not’, or ‘never’. Explicit negations like “staff was not polite” are easily picked up by rules-based or lexical/dictionary-based systems. Implicit ones like “it cost me an arm and a leg” require custom rules or learning-based sentiment models to capture them accurately.

Does accuracy matter in text analysis?

The short answer is yes. Being able to take actions and make decisions based on people’s feedback of course requires confidence in the data itself and in your text analysis.

However, as we’ve seen, considering accuracy as a statistical project can be difficult, and potentially limit the value you get from it.

There are instances where high recall is vital because action needs to be taken on just a few instances of feedback. Like a credit card company – just a couple of mentions of the word ‘fraud’ should be enough to trigger an action.

Or a digital team, where any spikes in mentions of ‘Broken Links’ or ‘Page Errors’ should be enough to take action and improve the experience.

There are occasions too where precision doesn’t matter. For example, in brand analysis competitor name mentions should be analyzed regardless of the sentiment.

Or, if you have customer feedback assigned to topics related to Injury, Lawsuits, Legal Proceedings, etc. these don’t need sentiment precision in order to raise a flag and trigger a deeper investigation.

Text analysis in multiple languages

A big part of taking action to improve the experience of customers and employees is the task of listening to the vast universe of unstructured feedback that exists in the form or customer survey responses, call center conversations, emails, social media, and many more channels.

Big global companies have the added challenge of having to systematically listen , analyze, and report on feedback in multiple languages. Indeed, some of the biggest companies need to do this across millions of verbatim responses in 20 or more different types of human language.

There are typically 2 ways to do it:

  • Using native language analysis for each respective human language
  • Translate all responses into a single ‘base language’ and analyze all content in that language

While there are pros and cons to each approach, the main thing is to balance accuracy and cost.

Weighing up accuracy and cost in text analysis

It’s widely accepted that native language analysis tends to offer greater accuracy. This is true given that translation may lose the linguistic nuances and return grammatically incorrect results.

However, there are a few points to keep in mind:

  • translation engines are getting smarter every year with new technologies being added. For example, Google translate has become more accurate over the years with machine learning capabilities that account for linguistic nuances
  • translation works pretty well on nouns, adjectives, and adverbs — these parts of speech are typically used most in topic building. The overall sentence structure might lose accuracy, but largely these parts of speech are translated well. And that is what is used in topic building and lexical sentiment tools. If the technology is not lexical, and uses a learning mechanism, then the sentiment results on the translated text can vary in accuracy.

Native language analysis can be costly too.

In most text analysis tools, the taxonomy is built/customized to reflect a consistent structure to capture verbatim feedback that will be used to measure and report on employees’ or customers’ experiences.

So it’s the taxonomy where all the resources have to be invested upfront to build, and then periodically maintain, for consistent accuracy.

The cost of building the topic model goes up exponentially for native language analysis. For example, if it takes 2 weeks to build a fully customized automotive model for the after-sale/service touchpoints in English, it will potentially take 4 weeks to do that in German as well.

This also assumes that the CX team is able to find the local users in each market, train them to use the technology/software & then have them build the local language models.

The cost doesn’t end in the build phase — as you add more touchpoints or surveys, the text models need to be refreshed, in all languages. Every 3 months you would need to audit and add or edit topics to maintain consistent accuracy levels, and you’d need to do this in all languages.

Organizations need to assess whether the incremental value of increasing the accuracy by using native language analysis is worth the extra cost of resources.

There are some other things to bear in mind for native language analysis, too:

Availability of native language capabilities. Language analysis capabilities need to exist for each language in question.

And while it’s easy to find native language analysis capabilities for languages like German, French, Spanish, etc. it’s more difficult to find those capabilities when it comes to Nordic or Baltic languages for example. Some of the biggest text analysis engines in the world only analyze limited languages in their native form for this reason.

  • You can always report in all local languages. Text analysis has three different phases — Build, Analyse & Report. You can do model building in any language, but then for reporting to various different countries in role-based dashboards, the reports can always be presented in the local language. So the local users should still be able to read the reports & the analysis in their local language.
  • Consistency is key for measurement — irrespective of whether a technology supports 35 languages, the ultimate aim should always be to have consistency in modeling and reporting; and efficiency in building and maintaining a taxonomy model whether that’s through native language analysis, or using a ‘base language’ approach.
  • Sentiment analysis is impacted more by translation than topic analysis. Therefore it’s preferable to have the sentiment scoring done in the native language as opposed to the translated language. This should not mean spending any resources, as most of the text analysis solutions use pre-built sentiment analysis techniques which usually do not require any labor-intensive model building work in a CX scenario.

What is best practice for text analysis in multiple languages?

Ideally, model building should be done in no more than 2 base languages keeping in mind the team size, the geographical spread, the linguistic abilities of the insights teams, and the cost/effort to build and maintain multiple language models.

The most effective approach involves four key steps:

  • Choose a technology that can seamlessly and automatically translate multi-language verbatims into one or two base languages.
  • Build a topic model in the translated language using a combination of bottom-up & top-down approaches. The technology should make it easy to build this using a combination of automatic plus manual methods of categorization.
  • The technology must be able to provide sentiment scoring in the native language as that is more accurate
  • For reporting purposes, the text model or category labels contained in the base model should be easily translated to the native language at the reporting layer, so that native language users can easily see the reports in their own language along with the original native language verbatim.

Multilingual topic and sentiment analysis report with word cloud and topic trend chart

Essential tools for text analysis software

Throughout this guide, we’ve looked at the various methods behind text analysis and the complexities of building models and hierarchies and running text analysis in multiple languages.

Running all of this yourself is a big ask — and one very few organizations will be set up to do.

Thankfully, there are plenty of text analysis tools available to help you draw insights out of open text. Here’s what you need to look out for in a text analysis tool for your organization:

Multi-Channel – solicited and unsolicited collection of text data is absolutely essential for an enterprise CX program. If you’re only analyzing survey data, then you’re missing out on a lot of actionable insights in sources such as Social media, Call center interactions, Online Chat, etc.

The best text analysis tools can analyze data from multiple sources rather than being limited to just one or two. This helps you to see the complete picture of what customers or employees are saying, wherever they’re saying it, so you can build up a better picture of the experience and therefore take the right actions to improve it.

Data analysis

Statistical + Text Analysis – Must have the ability to run regression analysis on the Text Topics & Sentiment, to determine the actual impact on the CX KPI score. Whether the Staff Attitude has more impact on my NPS or the Product Quality, it’s very important to understand the correlation and regression of structured scores with text information.

Action taking

Text analysis may not just be used for aggregate root cause analysis & driving improvements from the back office. It must be able to enhance real-time close the loop for dissatisfied customers based on their open-ended comments. Close the loop must not be confined to conditions based on scores given by the customer but should be able to get triggered based on the topic or sentiment definitions from the comments.

Data visualization

Flexibility in Visualisations – Text analysis is more than just colored word clouds or topic bubbles. Giving endless flexibility in visualizing text analysis information with Structured Data (like Segments, Regions, NPS, Effort Score, etc) and Operational Data (like Call Volumes, Handling Time, Customer Lifetime Value, etc), enables ease & speed of insights discovery & action prioritization. The most useful ones are Hierarchical Topic & Sentiment Bar chart, Stacked Bars with Topic & Operational Data, Loyalty Group Bars with Sentiment line, etc.

Easy-to-understand — make sure the visualizations are easy to interpret for everyone in the organization. Typically you should be able to see at a glance trending topics, a breakdown of sentiment, plus changes over time

Drill-down into comments — knowing the trends in topics and sentiment are the start, but you’ll also want to be able to drill down into reports through to individual responses. Of course, you’re not going to read every comment in full, but it’s useful to dig deeper to see what people are actually saying in dipping trends or low NPS segments. And also to sense check that your topic model is working well.

Ecosystem + integrations

One platform — for any experience management program, it’s better that your quantitative and text data are collected and analyzed on the same platform. This saves hours of manual effort bringing different data sets and technologies together to get the complete picture

Integrations — if you’re running a closed-loop customer experience program, make sure your text analytics tool is integrated into your existing systems like your ticketing application. This means that, based on sentiment and topic, relevant customer comments can automatically trigger a follow-up in the systems your teams are using already, making it faster to follow up and ensuring people have the right information to close the loop effectively with a customer.

X + O data together — the ultimate goal of any experience management program is to drive value back to the business. So look for a platform that brings together experience data (X-data) like text, alongside operational data (O-data) like sales figures, or HR data. This helps you to make connections between what people are saying, and their behavior – for example, do people who talk about helpful staff in-store spend more than those who don’t. That way, the actions you take based on the insights you gather from text analysis will be geared towards delivering ROI and growing the business.

The Qualtrics XM Discover Platform offers best-in-class text analytics that’s powered by AI, machine learning, and deep-learning algorithms. But ours is a platform that goes a step further, bringing text, voice, and third-party sources together into one seamless solution via natural language processing.

Qualtrics XM™ is built on three key pillars that help brands turn customer insights into action:

1. Listen and understand

We help brands do this by empowering them to actively engage with their audience – via customer surveys, questionnaires, and research – while text analytics and natural language processing help you discover what people are saying – wherever they say it – in real-time.

So whether customers are calling to complain, emailing your support address, mentioning you on social platforms, or leaving praise on third-party review sites, you’ll know about it. Importantly, voice and text analytics is able to assign sentiment and meaning to all your otherwise unstructured text data.

2. Remember everything and get context

Contextual data is much more useful than reams of static numbers. XiD can create experience profiles for each customer and employee, connect their profiles to your CRM/HRIS systems, and orchestrate the ideal journey for target groups. With rich data visualization, you’ll be able to see where experience gaps lie and what needs to be fine-tuned.

3. Act with empathy and speed

With proactive suggestions and intelligent insights, you’ll be able to immediately take the appropriate next-right-action based on your customer or employee history and context, at scale.

Learn more about Qualtrics XM here

Why you should use text analytics in customer experience

Having your customer experience management (CXM) platform and text analytics software integrated means that you can use the outputs from your text analytics of customer feedback throughout your program to drive change throughout the organization.

  • Include text visualizations in reports to trend, baseline, and identify key drivers
  • Deeply analyze text data, such as topic and sentiment tags, alongside other quantitative measures from statistical analyses to find clusters and root causes of desired behaviors
  • See trends over time to ensure that proactive action can be taken on areas of concern
  • Automatically deliver role-based dashboards that include relevant text insights in Customer Experience and Employee Experience dashboards
  • Trigger ongoing action items based on topic and sentiment to close the loop with upset and at-risk customers
  • Benchmark topic categories and sentiment ratings to set goals for the future

You will also be able to uncover previously unknown themes lurking out there that you never knew to look for. Text analytics uses sophisticated machine learning models to discover blind spots that are hidden in free text comments, leading you to uncover customer pain points you never knew to look for.

Open-text is a great way to discover pain points you weren’t aware of, provide specific context to why a customer respondent left a negative NPS score, and prepare your customer service teams with the background needed to close the loop with the customer.

Text analysis with Qualtrics’ Text iQ

As outlined, your text analysis software needs to be sophisticated and manageable to accurately parse textual data.

Qualtrics Text iQ automates key processes to help you focus on the actions you need to take, rather than the analysis you need to make. Powered by patented machine learning and natural language processing, this complex but easy-to-use software is always listening and evaluating your customers’ key sentiments.

With the ability to monitor trends over time and analyze both structured and unstructured text, Text iQ can deliver you and your frontline staff the insights they need to understand and win over your target audience.

You can read all about Qualtrics Text iQ software functionality , or book a demo now .

Automatically conduct complicated text analysis with Qualtrics Text iQ

Related resources

Text Analytics

Text Mining 16 min read

Topic modeling 18 min read.

Analysis & Reporting

Data Saturation In Qualitative Research 8 min read

How to determine sample size 12 min read.

Market Segmentation

User Personas 14 min read

Focus Groups

Focus Groups 15 min read

Market intelligence 10 min read, request demo.

Ready to learn more about Qualtrics?

  • Skip to Guides Search
  • Skip to breadcrumb
  • Skip to main content
  • Skip to footer
  • Skip to chat link
  • Report accessibility issues and get help
  • Go to Penn Libraries Home
  • Go to Franklin catalog

Text Analysis

  • Text Analysis Home
  • Google N-gram Viewer
  • Constellate
  • ProQuest TDM Studio
  • Importing Files
  • NLTK Package
  • Scattertext Package
  • SpaCy Package
  • TextBlob Package
  • Text Analysis Using R
  • Topic Modelling

With the explosion of digital information, researchers are faced with the immense challenge of deriving meaningful insights from vast amounts of unstructured data. Text analysis offers a range of techniques that can help analyze large volumes of text data and extract valuable insights. This guide aims to introduce some of the key concepts and terminologies of text analysis that can reveal hidden patterns and relationships within textual data, leading to valuable insights for diverse fields ranging from marketing to social sciences. 

Sentiment Analysis : Sentiment analysis employs natural language processing techniques to identify and extract subjective information from text, such as opinions and emotions expressed in the textual data, and is commonly used to analyze social media posts, customer reviews, and other text data to determine the overall sentiment.

Text Classification : Text classification involves categorizing text data into predefined classes or categories based on the content of the text and is frequently used for tasks such as spam filtering, topic identification, and sentiment classification.

Topic Modeling : Topic modeling is a statistical method used to identify topics or themes that occur in a collection of documents, allowing hidden patterns and relationships within text data to be discovered. It is widely applied in fields such as social sciences and humanities.

Named Entity Recognition : Named Entity Recognition (NER) is the process of identifying and extracting named entities from text, such as names of people, places, and organizations. It is commonly used for information extraction, retrieval, and data analysis.

Text Clustering : Text clustering is the process of grouping similar documents together based on their content, which is frequently used to identify patterns and similarities in large text datasets, particularly in fields such as marketing and customer service.

Text Summarization : Text summarization involves creating a concise summary of a longer text document and can be used to quickly understand the main points and themes of a large document or set of documents.

Text Mining : Text mining involves extracting useful information from unstructured text data using techniques such as natural language processing, machine learning, and information retrieval to discover patterns, relationships, and trends in large text datasets.

Named Entity Disambiguation : Named Entity Disambiguation is the process of disambiguating named entities by distinguishing between entities with similar names or referring to the same real-world entity, thereby reducing ambiguity in text data.

Word Frequencies : Word frequency analysis involves counting the number of times each word appears in a text document or corpus to identify common words or phrases, which can provide insights into the content of the text data.

Visualization : Text visualization involves creating visual representations of text data, such as word clouds, topic models, and graphs, to identify patterns, trends, and relationships in the data and communicate insights to stakeholders in a clear and concise manner.

  • << Previous: Text Analysis Using R
  • Next: Topic Modelling >>
  • Last Updated: Jun 3, 2024 11:42 AM
  • URL: https://guides.library.upenn.edu/penntdm

















 

 

 

 

 

 

 

Accurate and reliable coding (or labeling) using humans and machines is central to both the qualitative research and the text analytics process. For new and experienced researchers, learning advanced project management skills adds hours of struggle. Since 2005, ( ) coders and project managers have delivered large-scale, high value-added, reliable and accurate annotated text data in a timely and expert manner. The QDAP approach to qualitative research grows out of more than a decade of working on projects funded by the National Science Foundation, the National Institutes of Health, DARPA, the Smithsonian and others. Our combination of software, methods, experience, and novel forms of text analytics measurement present a uniquely scientific research support offering with applications across the disciplines.

The (CAT), winner of the 2008 " " award from the section of the , is now available as open source software. Download the or the latest .
 

curated by QDAP Director Stu Shulman.

 

& .

 

. ?
, and APIs


© 2008-2012

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

  • Notifications You must be signed in to change notification settings

Qualitative data analysis for text, images, audio, video. Cross platform. Python 3.10 or newer and PyQt6.

ccbogel/QualCoder

Folders and files.

NameName
4,940 Commits
ISSUE_TEMPLATE ISSUE_TEMPLATE

Repository files navigation

q research software text analysis

QualCoder is a qualitative data analysis application written in Python.

Text files can be typed in manually or loaded from txt, odt, docx, html, htm, md, epub, and PDF files. Images, video, and audio can also be imported for coding. Codes can be assigned to text, images, and a/v selections and grouped into categories in a hierarchical fashion. Various types of reports can be produced including visual coding graphs, coder comparisons, and coding frequencies.

This software has been used on MacOS and various Linux distros. Instructions and other information are available here: https://qualcoder.wordpress.com/ and on the Github Wiki .

It is best to download the Current Release from the Releases page, see the Releases link in the right-hand column on this page.

If you like QualCoder please buy me a coffee ...

Buy Me A Coffee

INSTALLATION

Prerequisites.

Optional: VLC for audio/video coding. Optional: ffmpeg installed for speech-to-text and waveform image see here to install ffmpeg on Windows: https://phoenixnap.com/kb/ffmpeg-windows .

For installing from source you will need to have Python 3.10 or a newer version installed.

Use the exe

Newer releases contain an exe file (created on Windows 10, 64-bit). Double-click to run. Look for the Releases link on the right-hand side of this page. I have had feedback on one instance on Windows where an anti-virus affected the importing and moving of files by QualCoder (AVG). An online virus testing site www.virustotal.com indicated 2 vendors out of many detected a potential problem due to their detection methods (false positives), 5 March 2022. Always check the MD5 checksum on downloading the exe. I have not got the exe Microsoft certified (I am not sure of the processes or cost involved). If you are uncomfortable with these warnings install from the source as detailed next.

Alternatively, install from source:

Seriously consider using a virtual environment (commands in point 6 below). Not using a virtual environment may affect other Python software you may have installed.

  • Download and install the Python programming language. The minimum recommended version for QualCoder is 3.10. Python3 . Download the file (at the bottom of the website) "Windows installer (64-bit)"

IMPORTANT: in the first window of the installation mark the option "Add Python to PATH"

Download the QualCoder software from: https://github.com/ccbogel/QualCoder from the Green Code button. This is the newest, but not yet officially released code (occasionally coding errors creep in). Click the green button "Code", and then "Download ZIP". Alternatively , choose the most recent release zip, see the right-hand side of this page for the link to Releases.

Unzip the folder to a location (e.g. downloads). (Tip, remove the doubled-up folder extraction QualCoder-master\QualCoder-master when asked where to extract. Just QualCoder-master).

Use the Windows command prompt. Type "cmd" in the Windows Start search engine, and click on the black software "cmd.exe" - the command console for Windows. In the console type or paste, using the right-click mouse copy and paste (ctrl+v does not work)

In the command prompt, move (using the cd command) into the QualCoder folder. You should be inside the QualCoder-master folder or if using a release (the Qualcoder-3.5 folder). e.g.

  • Install and activate the virtual environment. This step can be skipped, but I recommend you do not skip it.

The py command uses the most recent installed version of Python the py command does not work on all Windows OS, you may instead replace py with python3 You can use a specific version on your Windows if you have many Python versions installed, e.g. py -3.10 See discussion here: Difference between py and python

We recommend using a virtual environment to install packages. This will ensure that the dependencies for QualCoder are isolated from the rest of your system. On some Windows OS you may need to replace the py command with python3 below:

  • Install python modules. Type the following to upgrade all python modules before importing:

Type the following to install the required modules:

Wait, until all modules are installed.

Note: on some Windows computers, you may have to type python3 instead of py as py may not be recognised.

  • Run QualCoder from the command prompt
  • If running QualCoder in a virtual environment, to exit the virtual environment type:

The command prompt will then remove the (env) wording.

To start QualCoder again

If you are not using a virtual environment, as long as you are in the same drive letter, eg C:

py -m qualcoder

If you are using a virtual environment:

cd to the Qualcoder-master (or Qualcoder release folder), then type:

env\Scripts\activate.bat

Debian/Ubuntu Linux

There is an executable file (double-click to run) for Ubuntu 22 in the 3.5 release. Alternatively, install from source code below. It is best to run QualCoder inside a Python virtual environment so that the system-installed python modules do not clash and cause problems. If you are using the alternative Ubuntu Desktop manager Xfce you may need to run this: sudo apt install libxcb-cursor0

  • Recommend that you install vlc (download from site) or:

sudo apt install vlc

  • Install pip

sudo apt install python3-pip

  • Install venv I am using python3.10 you can choose another recent version if you prefer, and if more recent versions are in the Ubuntu repository.

sudo apt install python3.10-venv

Download and unzip the Qualcoder folder.

Open a terminal and move (cd) into that folder. You should be inside the QualCoder-master folder or if using a release, e.g. the Qualcoder-3.5 folder. Inside the QualCoder-master folder:

python3.10 -m venv qualcoder

Activate venv, this changes the command prompt display using (brackets): (qualcoder) Note: To exit venv type deactivate

source qualcoder/bin/activate

  • Update pip so that it installs the most recent Python packages.

pip install --upgrade pip

  • Install the needed Python modules.

pip install chardet ebooklib ply openpyxl pandas pdfminer pyqt6 pillow pdfminer.six plotly pydub python-vlc rispy six SpeechRecognition

  • You must be in the QualCoder-master folder (Or the main release folder if using a release. e.g. QualCoder-3.5 folder). Install QualCoder, and type the following, the dot is important:

python3 -m pip install .

You may get a warning which can be ignored: WARNING: Building wheel for Qualcoder failed

  • To run type

After all this is done, you can deactivate to exit the virtual environment. At any time to start QualCoder in the virtual environment, cd to the Qualcoder-master (or Qualcoder release folder), then type: source qualcoder/bin/activate Then type qualcoder

Arch/Manjaro Linux

It has not been tested, but please see the above instructions to build QualCoder inside a virtual environment. The below installation instructions may affect system-installed python modules.

  • Install modules from the command line

sudo pacman -S python python-chardet python-openpyxl python-pdfminer python-pandas python-pillow python-ply python-pyqt6 python-pip

  • Install additional python modules

sudo python3 -m pip install ebooklib plotly pydub python-vlc rispy SpeechRecognition

If successful, all requirements are satisfied.

  • Build and install QualCoder, from the downloaded folder type

sudo python setup.py install

  • To run type:

Or install from AUR as follows:

yay -S qualcoder

Fedora/CentOS/RHEL linux

It has not been tested, but please see the above instructions to build QualCoder inside a virtual environment. The below installation instructions may affect system-installed Python modules.

Retrieve the current package code from this repository

  • Open your preferred shell (terminal).
  • Navigate to your preferred code directory.
  • There, run: git clone https://github.com/ccbogel/QualCoder.git and
  • enter the directory with cd QualCoder
  • Make install_fedora.sh executable ( chmod +x install_fedora.sh ) and
  • run the ./install_fedora.sh script from the terminal. The script is for Python version 3.11.

Then start QualCoder as any other app on your system.

Note 1_ This script installs the dependencies using dnf and the ebook libraries with a work-around, specified at #72 (comment) .

Note 2: Fedora uses Wayland which does not work well with the Qt graphical interface (for now). I suggest you also install Xwayland.

The instructions work on Mac Monterey. It is recommended to use a virtual environment, see: https://sourabhbajaj.com/mac-setup/Python/virtualenv.html The below instructions can be used inside a virtual environment folder instead of placed in Applications.

You will need to install developer tools for macOS. See https://www.cnet.com/tech/computing/install-command-line-developer-tools-in-os-x/

Install recent versions of Python3 and VLC .

Download the latest release "Source code" version in ZIP format, from the releases section of the project here on Github: https://github.com/ccbogel/QualCoder/releases/tag/3.5 and extract it into /Applications

Open the Terminal app (or any other command shell)

Install PIP using these commands (if not already installed). Check pip is installed: try typing pip3 --version and hit ENTER)

-> You should now be able to run pip3 as above.

  • Install Python dependency modules using pip :

Be sure that you are in the QualCoder-Master directory before doing Step 6.

To change the directory, enter or copy and run the script below.

cd /Applications/QualCoder-3.5

  • From the QualCoder-Master directory run the setup script:

Assuming you downloaded the 3.5 version. You can now run with:

Alternative commands to run QualCoder (Suggestions):

From any directory:

From the QualCoder-Master directory:

python3 -m qualcoder

python3 qualcoder/__main__.py

You can install QualCoder anywhere you want, so the path above depends on where you extracted the archive.

Another option to run Qualcoder is shown here: https://www.maketecheasier.com/run-python-script-in-mac/ . This means you can right-click on the qualcoder.py file and open with --> python launcher. You can make an alias to the file and place it on your desktop.

Another option to install on Mac:

  • Open the Terminal App and move to the unzipped Qualcoder-Master directory, then run the following commands:
  • Install Python dependency modules using pip3 :

pip3 install chardet ebooklib ffmpeg-python pyqt6 pillow ply pdfminer.six openpyxl pandas plotly pydub python-vlc rispy six SpeechRecognition

pip3 install -U py2app or for a system installation of python sudo pip3 install -U py2app

python3 setup.py py2app

Dependencies

Python 3.8+ version, pyqt6, Pillow, six (Mac OS), ebooklib, ply, chardet, pdfminer.six, openpyxl, pandas, plotly, pydub, python-vlc, rispy, SpeechRecognition

QualCoder is distributed under the MIT LICENSE.

Citation APA style

Curtain, C. (2023) QualCoder 3.5 [Computer software]. Retrieved from https://github.com/ccbogel/QualCoder/releases/tag/3.5

Dr. Colin Curtain BPharm GradDipComp Ph.D. Pharmacy lecturer at the University of Tasmania. I obtained a Graduate Diploma in Computing in 2011. I have developed my Python programming skills from this time onwards. The QualCoder project originated from my use of RQDA during my PhD - Evaluation of clinical decision support provided by medication review software . My original and now completely deprecated PyQDA software on PyPI was my first attempt at creating qualitative software. The reason for creating the software was that during my PhD RQDA did not always install or work well for me, but I did realise that I could use the same SQLite database and access it with Python. The current database is different from the older RQDA version. This is an ongoing hobby project, perhaps a labour of love, which I utilize with some of the Masters's and Ph.D. students I supervise. I do most of my programming on Ubuntu using the PyCharm editor, and I do a small amount of testing on Windows. I do not have a Mac or other operating system to check how well the software works regards installation and usage.

https://www.utas.edu.au/profiles/staff/umore/colin-curtain

https://scholar.google.com/citations?user=KTMRMWoAAAAJ&hl=en

Leave a review

If you like QualCoder and find it useful for your work. Please leave a review on these sites:

https://www.saashub.com/qualcoder-alternatives

https://alternativeto.net/software/qualcoder

Also, if you like Qualcoder a lot and want to advertise interest in its use, please write an article about your experience using QualCoder.

Code of conduct

Releases 19, contributors 30.

  • Python 100.0%

Digital Humanities

  • Getting Started
  • Digital Scholarship Services
  • Mapping and Timelines
  • Storytelling
  • Text Analysis
  • Visualization
  • Static Sites & Minimal Computing
  • Communities

Text Corpora

  • Text Data Mining by Nick Wolf Last Updated Jun 4, 2024 2050 views this year
  • Google N-Grams Viewer Designed by Google which allows a user to plot the use of words over time in approx. 5 million books. You can select sets of curated works: "American English," "British English," "English," "Chinese (simplified)," "English Fiction," "French," "German," "Hebrew," "Russian," and "Spanish."
  • HathiTrust Research Center The HathiTrust Research Center (HTRC) facilitates non-profit and educational uses of the HathiTrust Digital Library by enabling computational analysis of public domain works and (on limited terms) in-copyright works from its collection.
  • Time Magazine Corpus Allows you to search more than 100 million words of text of American English from 1923 to the present, as found in TIME magazine.

Open E-Book Collections

  • Directory of Open Access Books A continuously updated collection of academic peer-reviewed books from over 50 publishers.
  • Google Books Search a large index of the world's books. Find millions of great books you can preview or read for free.
  • Hathi Trust HathiTrust Digital Library is a large-scale collaborative repository of digital content from research libraries including content digitized via Google Books and the Internet Archive digitization initiatives, as well as content digitized locally by libraries. 17+ million digitized items.
  • Internet Archive Internet Archive is a non-profit digital library offering free universal access to books, movies & music, as well as 376 billion archived web pages.
  • Project Gutenberg Project Gutenberg offers over 42,000 free ebooks from books in the public domain: choose among free epub books, free kindle books, download them or read them online.

Text Visualization Tools

  • Bookworm Bookworm is a simple and powerful way to visualize trends in repositories of digitized texts.

You can select sets of curated works: "American English," "British English," "English," "Chinese (simplified)," "English Fiction," "French," "German," "Hebrew," "Russian," and "Spanish."

Text Mining Tools

  • Calibre Calibre is a free and open source e-book library management application developed by users of e-books for users of e-books. It has a cornucopia of features including E-book format conversion.
  • HathiTrust Research Center The HathiTrust Research Center enables computational access for nonprofit and educational users to published works in the public domain and, in the future, on limited terms to works in-copyright from the HathiTrust.
  • Hermeneutica A collection of text mining tools accompanied by sets of text (an online book) describing the how’s and why’s of text mining in modern scholarship. Hermeneuti is a collaborative project by Stéfan Sinclair & Geoffrey Rockwell.
  • Voyant Tools Voyant Tools is a web-based text reading and analysis environment. It is a scholarly project that is designed to facilitate reading and interpretive practices for digital humanities students and scholars as well as for the general public. The site includes example projects. You can add text directly into the Voyant browser app and create interactive panels to embed in your online publications.

Text and Video Annotation

  • Annotation Studio Annotation Studio is a suite of collaborative web-based annotation tools currently under development at MIT.
  • Hypothes.is An open-source software project that aims to collect comments about statements made in any web-accessible content, and filter and rank those comments to assess each statement's credibility. It has been summarized as "a peer review layer for the entire Internet."
  • Manifold The intuitive, collaborative, open-source platform for scholarly publishing that allows for deep reading, annotation, and engagement. You can create books or engage with others published in this platform.
  • Thing Link Seamlessly make your images, videos, and 360 content interactive with text, links, images, videos and over 70 call to actions, creating memorable experiences for audiences.
  • VideoAnt VideoAnt is a web-based video annotation tool for mobile and desktop devices. Use VideoAnt to add annotations, or comments, to web-hosted videos. VideoAnt-annotated videos are called “Ants”.
  • << Previous: Storytelling
  • Next: Visualization >>
  • Last Updated: Aug 7, 2024 2:26 PM
  • URL: https://guides.nyu.edu/digital-humanities

SaaSworthy - Let's find the best software for your business

Home / Text Analysis Software

Updated on: August 16, 2024

Text Analysis Software List (August 2024)

q research software text analysis

Microsoft Text Analytics API

Turn unstructured text into meaningful insights with Text Analytics. Get sentiment analysis, key phrase extraction, and language and entity detection.

IBM Watson Explorer logo

IBM Watson Explorer

IBM Watson Explorer is a new way to explore all your unstructured data. It uses text analytics and machine learning to help you find secrets hiding in your data. A complex claim that took 2 days to process can now be completed in 10 minutes.

SAS Contextual Analysis logo

SAS Contextual Analysis

Uncover insights hidden in massive volumes of textual data with SAS Visual Text Analytics, which combines powerful natural language processing, machine learning and linguistic rules to help you get the most out of unstructured data.

SAS Sentiment Analysis logo

SAS Sentiment Analysis

SAS Visual Text Analytics logo

SAS Visual Text Analytics

Micro Focus IDOL logo

Micro Focus IDOL

Micro Focus IDOL enterprise search & data analytics platform searches & analyzes unstructured data from any source including text, video & speech. Learn more here.

Enago Reports logo

Enago Reports

Enago Reports is the ultimate solution for those seeking to produce professional writing of the highest standards. This service’s comprehensive set of features helps simplify the writing and proofreading process while ensuring compliance and accuracy. When they use Enago Reports, they can be sure of superior language quality in their work. These AI-powered native language editors review, edit, and proofread their documents to check grammar, punctuation, and spelling. These native editors also ensure that their writing is culturally appropriate for their target audience. Enago Reports provides a reliable source of information to eliminate bias in their work. These editors employ a systematic approach to gathering and structuring information to help support their argument. They also use AI technology to identify any potential bias during the review process. The result? Well-researched, unbiased work that’s sure to impress any reader. Enago Reports can help them sidestep the dreaded process of proofreading and journal submission. These automated proofreaders scan each document for mistakes faster than ever before. This submission wizard helps them optimize their article for journal acceptance, so they can put themselves one step ahead of the competition. Lest we forget, Enago Reports can protect them from plagiarism and AI-generated content. This advanced software scans for any copied content and flags it for their review. This helps them avoid any possible legal issues that may arise from copying somebody else’s work. So why settle for less? Choose Enago Reports and produce professional work that is free of bias and errors. Put their trust in us for superior language quality, technical compliance, and a one-stop solution for all their writing needs.

Wootric Text & Sentiment Analytics logo

Wootric Text & Sentiment Analytics

Wootric CXInsight™ uses machine learning to auto-categorize and assignment sentiment to unstructured feedback from surveys, online reviews, social media, support tickets, employee feedback, and more. Machine learning identifies category themes and sentiment in each verbatim comment, instantly analyzing volumes of feedback. No more silos — democratize customer data.

Lumoa logo

Reduce churn and increase revenue with Lumoa's customer experience platform. Understand what drives customer experience in an easy and affordable manner. Collect, analyze, control and take actions on customer feedback.

Undetectable AI logo

Undetectable AI

Undetectable AI understands this struggle and has created a solution that will revolutionize the world of content creation. This powerful AI Detector and AI humanizer were specifically designed with serious writers and content creators in mind. They believe that every piece of content should have its unique voice and perspective, and with Undetectable AI, they can achieve just that. This AI Detector is a game-changer in itself. Unlike other detectors that simply highlight common phrases and sentences, ours goes a step further. It thoroughly scans its content and suggests changes to make it more original and distinctive. With just a click of a button, they can instantly enhance the quality of their writing. But what truly sets us apart from the rest is our AI humanizer. They understand that while AI can help with efficiency, nothing beats the creativity and emotion a human can bring to the table. That's why our humanizer allows them to humanize their AI text with consistency and quality. Their content will have the perfect balance of both AI and human elements, making it stand out from the rest. With Undetectable AI, they can be confident that their content will be top-notch, professional, and above all, undetectable as AI-generated.

Provalis Research QDA Miner logo

Provalis Research QDA Miner

QDA Miner is an easy-to-use quantitative data analysis software package. QDA Miner can analyze small and large collections of documents and images.

Canvs AI logo

Canvs AI is an emotion and behavior insights platform that understands how consumers feel, why they feel that way, and the business impact those feelings and behaviors create for brands through analyzing digital conversations, such as open ended text prompts in surveys, online product reviews, or from social comments on Twitter, Facebook, Instagram and YouTube.

Luminoso Daylight logo

Luminoso Daylight

Luminoso Daylight is a cloud-based Natural Language Understanding platform for sentiment analysis in 15 languages. The software offers tools to analyze and report on the presence of sentiment in text data. Evaluate mixed datasets with correlating text feedback and numeric ratings. Visualize concepts relate to each other and identify themes.

PoolParty logo

PoolParty Semantic Suite is the most complete and advanced semantic middleware platform on the global market. It uses innovative means to help organizations build and manage enterprise knowledge graphs as a basis for their AI strategy. A knowledge graph is a rich representation of a knowledge domain that is capable of deriving more understanding out of your data. It is used to break down data silos by linking data from various departments and organizations.

Quantexa is an integrated Contextual Decision Intelligence (CDI) platform used to connect data points to create context – entity graphs enriched with vital intelligence. Analytics can be powered by context and embedded into operational decision processes. It offers a modular integrated platform with open data interfaces and APIs.

fractal logo

Fractal Analytics helps global Fortune 100 companies power every human decision in the enterprise by bringing analytics and AI to the decision.

Citibeats logo

Citibeats is an artificial intelligence platform used to search and analyze the large amounts of text provided by citizens. The software offers tools to identify social facts and trends that are useful for companies and institutions. Interpret the needs and opinions of users in real-time on a large scale.

Indico Data

Indico Data is a platform used to drive more value across the business. The software offers tools capable of handling even the most unruly, unstructured intake data. Take documents from any source and transform them into insights and competitive advantage. Integrated OCR with built-in quality correction to measure the performance of work-with technologists.

Lang.ai logo

Lang.ai is a no-code service automation platform that empowers customer support teams to build AI models that they can directly control to improve and automate critical support processes. We seamlessly integrate into Zendesk and Salesforce and take the tedious and manual tasks out of agents’ hands so they can focus on what is most important, the customer. Our customers are leveraging Lang for the following use cases and with our plug and play technology they’re up and running in 48 hours with no model training and maintenance required.

Moderation API logo

Moderation API

Modern businesses demand quick, accurate content moderation. With the Moderation API from [Company Name], they can automate text analysis to streamline their workflow. Utilizing state-of-the-art artificial intelligence technology, this API helps professional teams become more efficient and save time without compromising on privacy and accuracy. The Moderation API ensures that the scales are faster and stays ahead of the competition. Working with this revolutionary tool, companies can trust that their content is moderated quickly without sacrificing quality or safety standards. With robust and reliable tools such as these, growing businesses can trust that their content is in good hands. Choose the Moderation API to ensure the business's success.

Text analysis software uses natural language processing (NLP) to extract valuable insights from structured and unstructured text data. This tool enables professionals to understand the sentiment analysis, key phrases, language nuances, recurring themes, and patterns that emerge from the data. Text Analytics software can process data from various channels, including emails, phone call transcripts, surveys, and customer feedback. Additionally, it can integrate with other analytical platforms, such as big data analytics and business intelligence systems. To qualify as text analysis software, a product must import text data from different sources and offer visualization features for text data analysis.

List of Text Analysis Software

PRODUCT NAME AGGREGATED RATINGS
4.4
3.9
4
3.5
3.5
4.1
0
5
0
0
)   Remove All

We understand SaaS better

Saasworthy helps stakeholders choose the right saas platform based on detailed product information, unbiased reviews, sw score and recommendations from the active community..

icon

Looking for the right SaaS

We can help you choose the best SaaS for your specific requirements. Our in-house experts will assist you with their hand-picked recommendations.

icon

Want more customers?

Our experts will research about your product and list it on SaaSworthy for FREE.

  • Open access
  • Published: 14 August 2024

Financial hardship screening among Native American patients with cancer: a qualitative analysis

  • Amber S. Anderson-Buettner 1 ,
  • Amanda E. Janitz 1 ,
  • Mark P. Doescher 2 ,
  • Stefanie D. Madison 3 ,
  • Michaela A. Khoussine 4 ,
  • Keri L. Harjo 5 ,
  • Marvin B. Bear 6 ,
  • Stephnie Dartez 7 ,
  • Sheryl K. Buckner 8 &
  • Dorothy A. Rhoades 9  

BMC Health Services Research volume  24 , Article number:  928 ( 2024 ) Cite this article

Metrics details

Cancer-related financial hardship is an increasingly recognized concern for patients, families, and caregivers. Many Native American (NA) patients are at increased risk for cancer-related financial hardship due to high prevalence of low income, medical comorbidity, and lack of private health insurance. However, financial hardship screening (FHS) implementation for NA patients with cancer has not been reported. The objective of this study is to explore facilitators and barriers to FHS implementation for NA patients.

We conducted key informant interviews with NA patients with cancer and with clinical staff at an academic cancer center. Included patients had a confirmed diagnosis of cancer and were referred to the cancer center through the Indian Health Service, Tribal health program, or Urban Indian health program. Interviews included questions regarding current financial hardship, experiences in discussing financial hardship with the cancer care and primary care teams, and acceptability of completing a financial hardship screening tool at the cancer center. Clinical staff included physicians, advanced practice providers, and social workers. Interviews focused on confidence, comfort, and experience in discussing financial hardship with patients. Recorded interviews were transcribed and thematically analyzed using MAXQDA® software.

We interviewed seven patients and four clinical staff. Themes from the interviews included: 1) existing resources and support services; 2) challenges, gaps in services, and barriers to care; 3) nuances of NA cancer care; and 4) opportunities for improved care and resources. Patients identified financial challenges to receiving cancer care including transportation, lodging, food insecurity, and utility expenses. Patients were willing to complete a FHS tool, but indicated this tool should be short and not intrusive of the patient’s financial information. Clinical staff described discomfort in discussing financial hardship with patients, primarily due to a lack of training and knowledge about resources to support patients. Having designated staff familiar with I/T/U systems was helpful, but perspectives differed regarding who should administer FHS.

Conclusions

We identified facilitators and barriers to implementing FHS for NA patients with cancer at both the patient and clinician levels. Findings suggest clear organizational structures and processes are needed for financial hardship to be addressed effectively.

Peer Review reports

Introduction

Cancer-related financial hardship is an increasingly recognized problem for patients, families, and caregivers [ 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 ]. As a relatively new field of study, financial hardship screening among patients with cancer is being studied in diverse settings with diverse instruments [ 9 ]. In a recent study, researchers looked at both medical and nonmedical financial hardship and sacrifice among patients with cancer in the general, all-races US population. Medical financial hardship included domains such as material (bankruptcy), psychological (stress related to paying bills), and behavioral (delaying care due to cost) whereas nonmedical financial sacrifice contained categories like savings depletion and changes in spending. More than 38% of study participants had to make financial sacrifices and 42% reported medical financial hardship [ 6 ]. Other studies that focused on the association between cancer and financial hardship similarly discovered financial sacrifices resulting from cancer treatment [ 10 , 11 , 12 , 13 ]. Such hardship has been linked to limited care, worse treatment adherence and poor health outcomes [ 7 , 14 , 15 ].

Financial hardship may be worse in some racial minority populations than in the general population [ 6 , 16 , 17 , 18 ], but cancer-related financial hardship for Native American (NA) patients has rarely been reported. Many NA patients, including persons of American Indian (AI) or Alaska Native descent, may be at increased risk for cancer-related financial hardship due to highly prevalent factors, such as low income, medical comorbidity [ 19 ], and lack of private health insurance coverage. Although a few studies of NA patients with cancer included a measure of anxiety or stress due to costs of care [ 20 , 21 ], financial hardship was not the focus of these studies.

Further, the systems of care utilized by many NA patients are unique, have a complex history, and often poorly understood [ 22 ]. NA patients are eligible to receive medical care coverage from an Indian Health Service (IHS) facility, a tribal health facility, or an IHS-supported urban health program, collectively abbreviated as “I/T/U” programs [ 23 , 24 ]. The IHS is a federal program that operates a network of hospitals and clinics in the United States (US). Tribal Health Program hospitals and clinics are operated by individual tribes, which also receive federal (IHS) funding to care for NA patients. Urban Indian Programs are independent organizations that receive IHS funds for NA patients. Marked differences in coverage exist between I/T/U programs, which also differ from other payers. Most I/T/U programs have limited resources and are severely under-resourced for cancer care and must refer patients to outside entities [ 24 ]. Unfamiliarity with the I/T/U systems of care or payment at the outside entities can lead to confusion and delays in oncologic referrals and treatment for many NA patients.

Although studies have evaluated financial hardship among NA patients with cancer, no study of implementing a FHS tool among NA patients with cancer exists in the current literature. NA patients with cancer in one study often reported financial barriers to care [ 20 ] as well as lack of coordination between systems of care as a major barrier to care [ 20 , 25 ]. Male caregivers of patients with cancer on a reservation in the American Southwest more often identified financial burden of caregiving as their leading concern than did female caregivers [ 26 ].

Potential facilitators and challenges to screening for cancer-related financial hardship for I/T/U patients have not been previously studied. Studies of cancer-related financial hardship in other populations may not reflect those faced by patients who depend on I/T/U systems of care for coverage.

The I/T/U system in Oklahoma includes approximately 50 federally operated health centers or hospitals, tribally operated health centers or hospitals, and urban outpatient facilities. Oklahoma has the largest tribal land area in the US. At the time of the study, more than 14% of citizens (~ 483,000) identified as NA either alone or in combination with another race, the highest proportion in the US [ 27 ]. Incidence and mortality rates from high-priority cancers are especially alarming for the state’s NA population, with 1.4 and 1.8 times higher mortality for lung and colorectal cancer, respectively, compared with the non-Hispanic White state population [ 28 ].

To design and implement a pilot FHS program for NA patients referred to a cancer center in Oklahoma, we obtained stakeholder views on facilitators and challenges to FHS. This manuscript includes findings from the first objective of a three-part study. Findings from subsequent parts of the study, including pilot FHS implementation and evaluation, will be presented elsewhere. We hypothesized that themes specific to NA patients referred by I/T/U facilities would emerge that inform the design and implementation of FHS for these patients. We also hypothesized that designated referral coordinators, or navigators, for I/T/U patients at the cancer center would be considered as helpful initiators for FHS. This study reports findings from patient, clinical provider, and clinical staff key informant interviews that identify several facilitators and challenges to consider when designing FHS. The results support FHS that considers facilitators and challenges related to I/T/U systems of care, as well as the role of designated staff familiar with these systems.

This descriptive case study research design uses qualitative methodology and a stakeholder-engaged approach [ 29 , 30 , 31 ] with semi-structured interviews to gather diverse perspectives related to FHS implementation for NA patients. This approach helps identify specific contextual facilitators and barriers to implementation that may not have been identified in other healthcare settings.

The setting for the study is the Stephenson Cancer Center (SCC), affiliated with the University of Oklahoma Health Sciences, opened in 2011. In recognition of the need for expertise in I/T/U health systems SCC created the American Indian Navigation Program (AINP). Of 20,540 patients at SCC in 2019, NA patients accounted for 6.4%, with more than half referred from I/T/U facilities across the state. From 2017–2019, 1,222 new I/T/U patients from multiple federally recognized tribal nations received AINP services. The American Indian Navigators (AINs) see any patient referred from an I/T/U facility, regardless of the type of cancer diagnosis.

The interview guides were developed for this study. In February 2021, the research team met with a 10-person stakeholder advisory board consisting of I/T/U and SCC clinicians, staff, and a NA patient for feedback regarding a semi-structured, open-ended instrument to guide patient (Patient Interview Guide) and provider (Provider Interview Guide) interviews. The Consolidated Framework for Implementation Research (CFIR) [ 32 ] served as the conceptual model for the guide to ensure consideration of factors that may influence financial hardship screening for NA patients with cancer [ 33 ]. The CFIR includes five domains (intervention characteristics, outer setting, inner setting, characteristics of individuals, and process) that may affect the implementation of a new screening process.

Key informants for semi-structured interviews included NA patients as well as clinic providers and staff at SCC. We used purposive sampling to recruit within the identified clinics as both patients and providers. Signed informed consent was obtained for all participants.

Eligibility for patients to participate as key informants included being NA, referred from an I/T/U system, ages 18 years or older, diagnosed with cancer, and currently receiving cancer care at the SCC. Project staff reviewed patient lists of potentially eligible patients and advised on which patients to recruit for this project based on general health status and reason for the visit (e.g., standard follow-up or chemotherapy visit). Patient participants completed a brief questionnaire immediately following the interview to collect demographic and clinical information, including cancer diagnosis and date of diagnosis as well as gender, race, ethnicity, age at diagnosis, highest level of education, and household income.

Eligibility for SCC personnel included being a provider (physician, physician’s associate, nurse practitioner) or clinical or support staff (clinic or nurse manager, case manager, social worker) at an SCC clinic. Providers were recruited through all-staff emails sent to SCC staff. SCC providers and staff completed a brief questionnaire to include demographics (gender, race, ethnicity, and age), years in practice, and their roles at SCC.

All patient and provider/staff participants were individually interviewed either in-person or by teleconference (Zoom) in sessions lasting less than one hour. Interviews were digitally audio-recorded and transcribed. Transcribed interviews were checked for accuracy and uploaded into MAXQDA for analysis [ 34 ]. Two research team members reviewed the transcripts, developed the codebooks, and coded interviews together to ensure cultural nuances were captured. Independent codebooks for provider and patient interviews were developed and revised. To enhance rigor and reproducibility, codes were developed and revised through an iterative process. Integrated approaches of inductive and deductive analysis have been shown to provide a more comprehensive perspective of the phenomenon of interest [ 35 ]. Codes were first developed using the key informant interview guides and subsequently revised as relevant patterns emerged. Through routine discussion, agreement of coding structure and definitions was achieved. Codes were reviewed and patterns were identified in each data set, presented below as themes. Findings from the analysis were shared with the research team and Stakeholder Advisory Board.

As noted, eight patients consented to the interview, and seven (four female, three male) patients completed the interview. Three patients were aged 35–54 years and four were 65 years and older. Four participants had lung cancer, and the three others had different cancers. Four were currently in treatment for their cancer, with two determining the treatment plan and one obtaining follow-up care from initial surgery. Provider and staff participants varied in position and included two Physician Assistants, one Social Worker, and one Surgical Oncologist. Years of practice ranged from four to 16 years.

Themes identified for both providers/staff (herein simplified to “providers”) and patients included: 1) existing resources and support services; 2) challenges, gaps in services, and barriers to care; 3) nuances of American Indian cancer care; and 4) opportunities for improved care and resources.

Existing resources and support services

Providers reported complexities in coordinating care for the NA patients. Interfacing with I/T/U health systems was identified as a challenge, particularly the inability to electronically prescribe medications, requirement to have certain orders completed at the referring facility (e.g., CT scans completed at the I/T/U facility instead of the cancer center) and obtaining pre-authorization for certain procedures to be completed at the cancer center. Providers perceived that the pre-authorization process delays care in some cases. While requiring patients to have orders completed by their referring I/T/U is also perceived as a barrier to some providers, others reported that this requirement benefited some patients by not having to make multiple trips to the cancer center. To reduce patient travel, some providers reported strategically scheduling multiple appointments on the same day.

Patients reported that their respective I/T/U provided support in a variety of capacities, including direct financial support with the amount and availability of support varying by site. Other types of support provided to participants by I/T/Us included gift cards to purchase gasoline for their vehicle, housing and utility assistance, and food assistance. In addition to financial support, some participants reported that their I/T/U provided support with their cancer care coordination by navigating the referral process, explaining the treatment plan, and prioritizing patients with cancer so that services and referrals can be completed in a timely manner. Other participants had to rely on alternative, pre-existing sources of funding and insurance to support themselves during treatment, such as Veterans Affairs, Medicare, Medicaid, or Social Security to make ends meet. Other sources of financial support reported by participants included personal savings accounts, unemployment, and tribal resources.

Patients identified resources to aid in the care coordination process, including navigation assistance at the cancer center, the I/T/U facility, and family support. Some patients relied solely on cancer center navigation services, whereas others received additional care coordination services at their referring I/T/U. One patient had received a medical bill from the cancer center but after sending to their I/T/U representative, the bill was paid in full, and the patient’s burden was alleviated. Regarding family support, most patients reported having a strong support system, relying on spouses, children, grandchildren, and parents to alleviate burdens related to their cancer care. Family support included providing transportation to cancer treatment, advocating for patient needs, coordinating cancer care, and identifying financial resources. Some patients described their family support systems more broadly, whereas others identified specific instances where family had assisted. One patient recounted the support their spouse provided, indicating that their life was saved as a result of them encouraging the patient to schedule an initial appointment. Participants also reported that their families provided financial support, assisting in several capacities ranging from bills to their children’s school uniforms.

Challenges, gaps in services, and barriers to care

All providers reported familiarity with the cancer center’s AINP and described the AINs as helpful for both providers and patients and critical mediators between I/T/Us and the cancer center. Although providers perceived the AINs to be communicative, some expressed a desire to have them become more accessible and embedded into their clinic. Providers felt that the limited number of AINs was a potential barrier to expanding navigation services, including financial hardship screening. Several barriers to treating NA patients were reported by providers. Staffing limitations have resulted in the inability to implement screening tools, embed necessary staff within the clinic, or adequately address financial challenges faced by patients. Providers noted the need for better clinical integration of social workers and financial navigators and suggested that having a point of contact within social work would promote care coordination and relationship building between patients and clinical staff. Challenges related to tribal health referrals were also identified as a barrier to care. Providers described challenges in discussing financial hardship with patients. Given the variety of roles among providers, the degree of comfort in discussing financial hardship varied. Providers who routinely refer patients to resources considered financial-related conversations to be part of their job responsibility, whereas those directly involved in medical care experienced discomfort in having these conversations (Table  1 ).

Providers reported that some NA patients experienced reduced quality of care, compared to other patients, due to the unique health system features as previously described (inability to electronically prescribe certain medications, requirement of imaging and/or lab tests to be completed at the I/T/U facility), and the mandatory pre-authorization process. A major challenge reported by multiple providers was transportation, with resources including transportation services for low-income participants through Medicaid, gas reimbursement using small emergency funds, or charitable donations. Other patient challenges included maintaining employment and few resources for childcare.

Patients identified multiple financial challenges to their cancer treatment, primarily related to logistics, such as transportation, lodging, and food during travel to the cancer center. The AIN team and I/T/Us could provide resources for some patients, including gas cards, hotel vouchers, and support for housing, utilities, and children. However, the I/T/Us primarily provided these resources, which were nevertheless limited, with some participants noting that there were no funds available when they inquired about them. Participants frequently relied on a family member for transportation to the cancer center, adding stress for the participants in arranging consistent transportation. While some transportation options are available through Medicaid, some patients lived outside of the radius for this transportation service.

Multiple patients reported unanticipated challenges during their cancer journey and concerns related to the financial impacts on their families. These included not knowing what to expect of cancer treatment after a recent diagnosis, transportation, and coping with changes to physical appearance after cancer-related surgery. The COVID-19 pandemic was also noted by multiple participants as affecting their ability to attend appointments and being generally disruptive.

Multiple patients reported psychosocial stress, including transportation concerns, stresses in their work environment, food insecurity, having enough funds to care for family and required medical treatment. Resources that patients described included Family Medical Leave Act, family support, and tribal resources (though these were limited). Patients reported that they relied on their faith and their family for support during stressful situations, like cancer diagnosis and treatment. Some patients reported stress associated with a cancer diagnosis, side effects of their cancer treatment, and challenges with pre-existing health conditions. Participants also identified challenges related to the management of taking new and numerous medications, pre-existing mobility, and vision impairments, and maintaining the right state of mind at work.

Nuances of Native American cancer care

Providers reported several nuances in treating and providing services to their NA patients. The logistics in care coordination between I/T/U systems and SCC were described as a challenge by some providers. Other challenges acknowledged by providers included lack of accommodations and travel support for long distances required for patients to travel for cancer care. Some providers had misconceptions of resources available to NA patients, with some believing that NA patients have more resources, whereas others believed they have fewer resources. One participant reported that although some NA patients may have access to unique resources for healthcare support, they face unique system-related challenges, including a complex referral and authorization process, as described further below (Table  2 ).

One patient noted cultural nuances and perspectives of cancer care. This participant emphasized the importance of returning to traditional ways and communal benefit when completing the interview by stating, “You don’t look out for one. You got to look out as a whole.” The value of family and community involvement and support was also described as an important component of care, stating their spouse answered healthcare providers’ questions because they are “just really Indian.” Other patients did not note specific nuances they attributed to NA differences in health care systems.

Opportunities for improved care and resources

Providers described several opportunities for improvement to better address the needs of NA patients. While some providers were less familiar with current financial hardship screening protocols, most participants indicated that screening strategies needed to be enhanced and better integrated into clinical workflow. Providers noted that financial hardship extends beyond the inability to pay medical bills; therefore, screening should also include food insecurity, transportation barriers, and challenges in filling prescriptions. Some participants expressed a desire to be more knowledgeable about anticipated financial hardships and resources for patients, in addition to having increased access to social workers.

Discomfort in discussing financial hardship was reported to be a result of the limited knowledge of resources and lack of adequate training in having financial-related conversations. Providers were not consistent in addressing FHS with all patients but would discuss if the patient asked a question. Providers reported that AINs and social workers were better equipped to aid patients with financial concerns. However, most providers expressed interest in training related to financial hardship screening, including skills for administering screening tools, resources available for patients, and awareness of patient needs (Table  3 ).

Providers had differing perspectives of patients’ comfort in discussing financial hardship, although some providers recognized that each patient is unique in their willingness to discuss their finances. According to some providers, some patients are upfront about their financial needs whereas others are willing to discuss challenges only if inquired. Providers suggested standardizing the FHS process so that patients are screened as a result of protocol, rather than after a patient expresses a concern. Providers recommended FHS that is thorough and specific yet brief enough for patients to complete routine paperwork. Opinions about who should conduct FHS varied among providers. Clinicians stated they rely on other staff (like social work or navigators) to address financial issues, unless the financial issue is obvious (e.g., missed appointments). Providers reported that, ideally, FHS would occur with the patient’s primary care provider, but many patients do not receive consistent primary care.

Participants stated that incorporating FHS into the standard EMR workflow, including automatic referrals and routine screening, would allow FHS to fit well within the current distress/social work referral process. FHS should be done routinely (e.g., every 6 months). Providers consistently stated that the first appointment with the SCC team for a newly diagnosed cancer was not the ideal time to address financial hardship due to the overwhelming nature of a cancer diagnosis. Ideally, AINs, social work, and nurse/financial navigators would be more accessible to address financial concerns as they arise.

Patients reported differing perspectives of financial hardship discussions and screening. While some patients indicated that someone on their healthcare team should be aware of financial challenges patients experience, others stated that inquiring about financial situations could feel like prying if resources were not available to address challenges. Among those who were comfortable with screening, some patients stated no preference of length as long as questions were not repetitive, whereas others preferred a shorter length – one page, 8–20 questions, or 1–15 questions. There were also differing preferences on when and how frequently the screening tool should be implemented. Some patients indicated that the screening tool should be completed at the initial diagnosis so that financial challenges are identified early in the process, whereas others reported that screening should occur after a treatment plan has been established due to the unexpected nature of many emerging financial challenges. Patients reported differing preferences on how the screening was completed, with some preferring in-person at their appointment and others over the phone. Generally, participants reported few financial related topics to avoid in screening for financial hardship, however discussions related to savings, credit standing, and long-term financial situation should be avoided. Some patients also identified a specific member of their healthcare team who they would feel most comfortable having financial related conversations with, including their provider or AIN. Some patients reported that resources are challenging to identify at times and having their healthcare team screen for anticipated challenges would reduce financial distress.

Most patients reported that their healthcare team had not discussed financial related challenges in the past, although resources such as transportation assistance and lodging had been offered. Some patients reported that they had not discussed financial hardship related to their cancer care with their I/T/U or referring provider, however some were informed that their treatment would be paid for by their referring I/T/U or that an AIN at the cancer center would assist with financial challenges that may arise. Patients were inconsistent in their perceptions of their healthcare team serving as a resource. Some patients reported that their healthcare team could provide financial resources, including gift cards, whereas others indicated that financial support services were not available for patients. All patients reported either working with an AIN or being willing to work with one to aid in care coordination processes. Additionally, the interactions with AINs ranged from having no recollection of working with them to routine interactions. Some patients indicated that having an AIN was a helpful resource, citing their role in securing lodging and transportation assistance during their cancer treatment. However, others were unaware of AIN services or confused about their role. Some unfamiliarity with AINs was attributed to family members serving as a liaison between the providers and the patient. Patients who were unaware of AIN services often reported that they would like to be connected.

Limitations included a small sample size due to recruitment challenges which may limit generalizability to larger populations. Another limitation were clinical workflow constraints which may have resulted in inhibited responses. In addition, as the study was conducted at a single cancer center, its findings may not be generalizable to all healthcare settings serving Native American patients.

Using a stakeholder approach for perspectives on implementing FHS for NA patients with cancer at an academic cancer center has revealed challenges and opportunities for both the cancer center and I/T/U systems. FHS was considered by both patients and providers as potentially beneficial to patients, as NA cancer care-related issues generally revolve around the different systems of care between I/T/U and non-I/T/U facilities. Having a designated team of navigators with expertise in the unique requirements of I/T/U patients was considered beneficial to both patients and providers. The need for more AINs was identified, especially if adding FHS to their roles. Structural systems to integrate FHS with both cancer center and I/T/U resources to address potential needs were viewed as potentially improving care by reducing delays in care or access to treatments. By integrating the CFIR framework within the interview guides, we ensured that each domain of CFIR was addressed, which will support development of future intervention strategies that account for each domain and provide the best opportunity for success. We included questions about future implementation of a financial hardship screening tool to assess comfortability and potential acceptability of questions related to financial hardship. By talking with both patients and providers, we were able to assess the outer setting (features of the external environment) and inner setting (organizational factors that may influence the intervention). We evaluated the characteristics of individuals by asking about their knowledge of financial hardship resources and financial needs during cancer treatment. By including questions of both patients and providers about what factors or strategies may influence implementation, we evaluated the process domain.

The reliance upon family was prominent and has emerged in other studies of NA patients with cancer, reporting that many NA families provide more than just emotional support [ 25 ]. While HIPAA presents challenges in the engagement of family, opportunities to expand involvement more systematically when requested by the patient exist.

While generally favorable to FHS, patients and providers differed in opinions regarding implementation. Training providers to undertake FHS and discuss financial concerns would be an opportunity to improve financial communication with patients. Adding FHS to the experience of receiving cancer care, however, may add to an often-overwhelming process. A study of NA patients found that excessive “paperwork” was a frequently noted barrier to cancer care [ 36 ]. The ideal implementation for asking and addressing financial barriers to care is currently being studied across the nation in diverse settings.

This study made identified several factors to consider when implementing FHS for NA patients referred to a cancer center. It employed a stakeholder approach to gather perspectives from both Native American cancer patients and cancer center providers on implementing FHS which allowed identification of facilitators and barriers to FHS implementation at both patient and provider levels. As hypothesized, specific I/T/U-related themes emerged, including both facilitators and challenges regarding coordination and coverage for care as well as varying degrees of cancer center and I/T/U support for indirect costs of care, such as transportation and lodging. Also as hypothesized, having AINs familiar with I/T/U systems of care were overall viewed by both patients and providers as useful in facilitating access to cancer work up and treatment, but additional staffing was identified as needed for implementation of FHS. Importantly, the study gathered culture-specific perspectives on the acceptability of FHS for Native American patients, addressing a gap in the literature. The application of the CFIR ensured a comprehensive assessment of factors influencing potential FHS implementation, providing a structured approach understanding the complex issues involved.

Theoretically, this research highlights the importance of considering cultural nuances and unique health system features when implementing FHS for Native American patients, emphasizing the need for culturally tailored approaches in healthcare interventions. The research demonstrated the utility of using implementation science frameworks like CFIR to guide assessment of FHS implementation factors, showcasing how such frameworks can be applied in real-world healthcare settings. Additionally, the study identified the need to balance FHS with existing screening burdens on cancer patients, pointing to the importance of considering the overall patient experience in healthcare interventions.

From a practical standpoint, the study offers several important insights. It suggests that clear organizational structures and processes are needed for financial hardship to be addressed effectively in healthcare settings. The findings indicate a need for provider training on conducting FHS and discussing financial concerns with patients, highlighting an area for potential improvement in patient care. The study also points to opportunities to better integrate American Indian Navigators and social workers into clinical workflows to address financial hardship. From a patient’s view, it highlights the need to understand the impact that family playsin FHS. Most importantly, this study suggests that FHS should be brief, routine, and integrated into existing clinical processes to be most effective and feasible.

A systematic review estimated that nearly half of individuals with cancer experience financial hardship [ 19 ]. It is associated with delays in cancer care and poor clinical outcomes [ 37 , 38 , 39 ]. The reasons for these adverse effects may reflect the multidimensional nature of financial hardship that encompasses: 1) direct consequences of treatment (e.g., out-of-pocket expenses, debt, and decreased income); 2) psychological distress because of costs; and 3) deleterious coping mechanisms (e.g., delaying or skipping medications or care) [ 40 ]. Because our study begins to address the important question of what financial hardship means for NA patients with cancer and their clinicians, our findings provide insights preparatory to future research to explore the underlying elements of financial hardship in relation to adverse health outcomes in NA patients.

In particular, our findings suggest that research on financial hardship among NA patients with cancer should examine the extent to which tribes and the I/T/U system can help address financial concerns, as the availability of tribal or I/T/U resources appears to be inconsistent. NA patients also emphasized the centrality of family and community in addressing financial issues, indicating that research examining the role of families in addressing financial hardship concerns is warranted.

A growing body of research suggests that implementation of navigation services for patients with cancer may help address financial hardship [ 41 , 42 , 43 ]. While some studies have examined NA-specific navigation programs [ 20 , 43 , 44 , 45 , 46 , 47 ], these studies have not focused on navigation that includes systematic financial hardship screening. Thus far, NA navigation programs have concentrated primarily on helping NA patients coordinate cancer care between oncology clinics and the I/T/U system. Successful NA navigation programs [ 20 , 43 , 44 , 45 , 46 , 47 ] are community-based and focus on patient needs, including barriers to accessing cancer care, cultural concerns, and education about cancer and treatment options [ 48 ]. Two studies have evaluated stress and anxiety due to the cost of cancer care among NA patients [ 20 , 21 ], but none included systematic financial hardship screening. Thus, research is needed to further investigate the potential impact and sustainability of navigated FHS interventions to help address financial concerns and enhance the care experience of NA patients with cancer. In particular, our findings suggest that research examining the implementation of systematic financial hardship screening should explore the questions of who should administer screening (e.g., navigators versus clinicians), how detailed the screening should be, and how frequently it should occur. Our findings also suggest that staffing limitations impede the ability to conduct financial hardship screening. Because the Centers for Medicare and Medicaid Services now provides coverage for cancer navigation services for persons with Medicare coverage [ 49 ], research could be conducted to determine whether this new CMS benefit can meaningfully support navigation services addressing financial hardship for NA patients with cancer.

This study had several limitations. Patient recruitment strategies interrupting clinical workflow and time restraints resulted in a small sample size for this study. We also experienced challenges in recruiting clinicians, likely due to clinical time restraints and unfamiliarity with FHS practices. While a broad range of NA perspectives were captured, tribal and cultural variance could be broadened in future studies. Conducting interviews with patients in a clinical environment may have inhibited their ability to adequately reflect and respond to the question guide. Similarly, provider interviews occurring outside of clinical hours may have allowed for enhanced responses. Future studies should examine culture-specific perspectives on the acceptability of FHS.

In conclusion, our study identified both facilitators and barriers to implementing FHS at a single cancer center among NA patients. Future studies should seek to understand the role of FHS in the context of the myriad of screenings cancer patients receive during treatment, in addition to the timing and frequency in which FHS should occur.

Availability of data and materials

Data are not available to be shared due to confidentiality issues.

Abbreviations

  • Native American

Financial Hardship Screening

Indian Health Service (IHS) facility, a tribal health facility, or an IHS-supported urban health program

Stephenson Cancer Center

American Indian Navigation Program

American Indian Navigators

Consolidated Framework for Implementation Research

United States

Ekwueme DU, Yabroff KR, Guy GP Jr, Banegas MP, de Moor JS, Li C, et al. Medical costs and productivity losses of cancer survivors–United States, 2008–2011. MMWR Morb Mortal Wkly Rep. 2014;63(23):505–10.

PubMed   PubMed Central   Google Scholar  

Ekwueme DU, Zhao J, Rim SH, de Moor JS, Zheng Z, Khushalani JS, et al. Annual out-of-pocket expenditures and financial hardship among cancer survivors aged 18–64 years - United States, 2011–2016. MMWR Morb Mortal Wkly Rep. 2019;68(22):494–9.

Article   PubMed   PubMed Central   Google Scholar  

Davis ME, Fugett S. Financial toxicity: limitations and challenges when caring for older adult patients with cancer. Clin J Oncol Nurs. 2018;22(6):43–8.

PubMed   Google Scholar  

Pisu M, Henrikson NB, Banegas MP, Yabroff KR. Costs of cancer along the care continuum: what we can expect based on recent literature. Cancer. 2018;124(21):4181–91.

Article   PubMed   Google Scholar  

Yabroff KR, Dowling EC, Guy GP Jr, Banegas MP, Davidoff A, Han X, et al. Financial hardship associated with cancer in the United States: findings from a population-based sample of adult cancer survivors. J Clin Oncol. 2016;34(3):259–67.

Han X, Zhao J, Zheng Z, de Moor JS, Virgo KS, Yabroff KR. Medical financial hardship intensity and financial sacrifice associated with cancer in the United States. Cancer Epidemiol Biomarkers Prev. 2020;29(2):308–17.

Zafar SY. Financial Toxicity of Cancer Care: It's Time to Intervene. J Natl Cancer Inst. 2016;108(5).

Carrera PM, Kantarjian HM, Blinder VS. The financial burden and distress of patients with cancer: understanding and stepping-up action on the financial toxicity of cancer treatment. CA Cancer J Clin. 2018;68(2):153–65.

Yabroff KR, Bradley CJ, Shih YT. Improving the process of screening for medical financial hardship in oncology practice. Cancer Epidemiol Biomarkers Prev. 2021;30(4):593–6.

Tevaarwerk AJ, Kwekkeboom K, Buhr KA, Dennee A, Conkright W, Onitilo AA, et al. Results from a prospective longitudinal survey of employment and work outcomes in newly diagnosed cancer patients during and after curative-intent chemotherapy: a Wisconsin Oncology Network study. Cancer. 2021;127(5):801–8.

Smith GL, Lopez-Olivo MA, Advani PG, Ning MS, Geng Y, Giordano SH, et al. Financial burdens of cancer treatment: a systematic review of risk factors and outcomes. J Natl Compr Canc Netw. 2019;17(10):1184–92.

Shankaran V, Unger JM, Darke AK, Suga JM, Wade JL, Kourlas PJ, et al. S1417CD: a prospective multicenter cooperative group-led study of financial hardship in metastatic colorectal cancer patients. J Natl Cancer Inst. 2022;114(3):372–80.

Liang MI, Simons JL, Herbey II, Wall JA, Rucker LR, Ivankova NV, et al. Navigating job and cancer demands during treatment: a qualitative study of ovarian cancer patients. Gynecol Oncol. 2022;166(3):481–6.

Doherty M, Finik J, Blinder V. Work status reduction and cost-related nonadherence during cancer treatment. Health Soc Work. 2022;47(2):123–30.

Hastert TA, Kyko JM, Reed AR, Harper FWK, Beebe-Dimmer JL, Baird TE, et al. Financial hardship and quality of life among African American and white cancer survivors: the role of limiting care due to cost. Cancer Epidemiol Biomarkers Prev. 2019;28(7):1202–11.

Hastert TA, Banegas MP, Hamel LM, Reed AR, Baird T, Beebe-Dimmer JL, et al. Race, financial hardship, and limiting care due to cost in a diverse cohort of cancer survivors. J Cancer Surviv. 2019;13(3):429–37.

Samuel CA, Spencer JC, Rosenstein DL, Reeder-Hayes KE, Manning ML, Sellers JB, et al. Racial differences in employment and cost-management behaviors in patients with metastatic breast cancer. Breast Cancer Res Treat. 2020;179(1):207–15.

Article   CAS   PubMed   Google Scholar  

Pisu M, Kenzik KM, Oster RA, Drentea P, Ashing KT, Fouad M, et al. Economic hardship of minority and non-minority cancer survivors 1 year after diagnosis: another long-term effect of cancer? Cancer. 2015;121(8):1257–64.

Altice CK, Banegas MP, Tucker-Seeley RD, Yabroff KR. Financial hardships experienced by cancer survivors: a systematic review. J Natl Cancer Inst. 2017;109(2):djw205.

Grimes C, Dankovchik J, Cahn M, Warren-Mears V. American Indian and Alaska native cancer patients’ perceptions of a culturally specific patient navigator program. J Prim Prev. 2017;38(1–2):121–35.

Harris R, Van Dyke ER, Ton TG, Nass CA, Buchwald D. Assessing needs for cancer education and support in American Indian and Alaska Native communities in the northwestern United States. Health Promot Pract. 2016;17(6):891–8.

Rhoades ER, Rhoades DA. The public health foundation of health services for American Indians & Alaska Natives. Am J Public Health. 2014;104(Suppl 3):S278–285.

Burnett-Hartman AN, Bensink ME, Berry K, et al. Access to the Indian health service care system is not associated with early enrollment in medicaid for American Indian and Alaska Natives with cancer. Cancer Epidemiol Biomarkers Prev. 2014;23(2):362–4.

Warne D, Kaur J, Perdue D. American Indian/Alaska Native cancer policy: systemic approaches to reducing cancer disparities. J Cancer Educ. 2012;27(1 Suppl):S18–23.

Craft M, Patchell B, Friedman J, Stephens L, Dwyer K. The experience of cancer in American Indians living in Oklahoma. J Transcult Nurs. 2017;28(3):259–68.

Cordova FM, B Harris R, Teufel-Shone NI, L Nisson P, Joshweseoma L, R Brown S, et al. Caregiving on the Hopi Reservation: Findings from the 2012 Hopi Survey of Cancer and Chronic Disease. J Community Health. 2016;41(6):1177–86.

Centers for Disease Control and Prevention. Tribal Population 2009-2013 [Available from: https://www.cdc.gov/tribal/tribes-organizations-health/tribes/state-population.html .

Sambo AB, Rhoades DA, Campbell JE, Hunsucker LA, Pharr SA, Doescher MP. Cancer among the American Indian Population in Oklahoma. 2022.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82:581-629.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. https://doi.org/10.1186/1748-5908-4-50 .

Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011;38(1):4–23. https://doi.org/10.1007/s10488-010-0327-7 .

Keith RE, Crosson JC, O’Malley AS, Cromp D, Taylor EF. Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci. 2017;12(1):15.

CFIR Research Team-Center for Clinical Management Research. The consolidated framework for implementation research. 2019. Available from: https://cfirguide.org .

Kuckartz U, Rädiker S. Analyzing qualitative data with MAXQDA. Cham: Springer International Publishing; 2019. p. 1–290.

Book   Google Scholar  

Blackstone A. Principles of sociological inquiry: qualitative and quantitative methods. Saylor academy open textbooks. 2018.

Goodwin EA, Burhansstipanov L, Dignan M, Jones KL, Kaur JS. The experience of treatment barriers and their influence on quality of life in American Indian/Alaska Native breast cancer survivors. Cancer. 2017;123(5):861–8.

Ramsey SD, et al. Financial insolvency as a risk factor for early mortality among patients with cancer. J Clin Oncol. 2016;34(9):980–6.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Kent EE, et al. Are survivors who report cancer-related financial problems more likely to forgo or delay medical care? Cancer. 2013;119(20):3710–7.

Zheng Z, et al. Do cancer survivors change their prescription drug use for financial reasons? Findings from a nationally representative sample in the United States. Cancer. 2017;123(8):1453–63.

Khera N, Holland JC, Griffin JM. Setting the stage for universal financial distress screening in routine cancer care. Cancer. 2017;123(21):4092–6.

Bell-Brown A, et al. Assessment of financial screening and navigation capabilities at National Cancer Institute community oncology clinics. JNCI Cancer Spectr. 2023;7(5):pkad055.

Guadagnolo BA, et al. A pre-post survey analysis of satisfaction with health care and medical mistrust after patient navigation for American Indian cancer patients. J Health Care Poor Underserved. 2011;22(4):1331–43.

Jean-Pierre P, et al. Do better-rated navigators improve patient satisfaction with cancer-related care? J Cancer Educ. 2013;28(3):527–34.

Burhansstipanov L, et al. Findings from the native navigators and the Cancer Continuum (NNACC) study. J Cancer Educ. 2014;29(3):420–7.

Harjo LD, Burhansstipanov L, Lindstrom D. Rationale for “cultural” native patient navigators in Indian country. J Cancer Educ. 2014;29(3):414–9.

Petereit DG, et al. Addressing cancer disparities among American Indians through innovative technologies and patient navigation: the walking forward experience. Front Oncol. 2011;1: 11.

Warren-Mears V, et al. Impact of patient navigation on cancer diagnostic resolution among Northwest Tribal communities. J Cancer Educ. 2013;28(1):109–18.

Guadagnolo BA, Petereit DG, Coleman CN. Cancer care access and outcomes for American Indian populations in the United States: challenges and models for progress. Semin Radiat Oncol. 2017;27(2):143–9.

Centers for Medicare and Medicaid Services. CMS finalizes physician payment rule that advances health equity. 2023. Available from: https://www.cms.gov/newsroom/press-releases/cms-finalizes-physician-payment-rule-advances-health-equity . May 14, 2024.

Download references

Acknowledgements

Not applicable.

Research reported in this publication was supported by the National Cancer Institute of the National Institutes of Health under Award Number P30CA225520. Partial funding provided by National Institutes of Health, National Institute of General Medical Sciences, grant 1 U54GM104938. The content is solely the responsibility of the authors and does not necessarily represent the official view of the National Institutes of Health.

Author information

Authors and affiliations.

Department of Biostatistics and Epidemiology, University of Oklahoma Health Sciences, Oklahoma City, OK, USA

Amber S. Anderson-Buettner & Amanda E. Janitz

Department of Family Medicine, University of Oklahoma Health Sciences, Oklahoma City, OK, USA

Mark P. Doescher

Oklahoma City Veterans Affairs Health Care System, Oklahoma City, OK, USA

Stefanie D. Madison

Central States Research, Tulsa, OK, USA

Michaela A. Khoussine

Oklahoma City Indian Clinic, Oklahoma City, OK, USA

Keri L. Harjo

Little Axe Health Center, Absentee Shawnee Tribe, Norman, OK, USA

Marvin B. Bear

Stephenson Cancer Center, University of Oklahoma Health Sciences, Oklahoma City, OK, USA

Stephnie Dartez

Fran and Earl Ziegler College of Nursing, University of Oklahoma Health Sciences, Oklahoma City, OK, USA

Sheryl K. Buckner

Department of Medicine, University of Oklahoma Health Sciences, Oklahoma City, OK, USA

Dorothy A. Rhoades

You can also search for this author in PubMed   Google Scholar

Contributions

ASAB analyzed and interpreted the data and drafted the initial manuscript. AEJ analyzed and interpreted the data and drafted the initial manuscript. MPD provided guidance on interpretation of data and contributed to writing the manuscript. SDM provided guidance on interpretation of data and contributed to writing the manuscript. MAK contributed to data collection. KLH contributed to data collection and provided guidance on interpretation of the data. MBB contributed to data collection and provided guidance on interpretation of the data. SD contributed to data collection and provided guidance on interpretation of the data. SKB contributed to writing the manuscript. DAR provided guidance on interpretation of data and contributed to writing the manuscript

Corresponding author

Correspondence to Amber S. Anderson-Buettner .

Ethics declarations

Ethics approval and consent to participate.

The study was approved by the Oklahoma Area Indian Health Service Institutional Review Board (Protocol Number P-21–01-OK) and the University of Oklahoma Health Sciences Institutional Review Board (Protocol Number 12659). All participants completed written or electronically signed informed consent.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1, supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Anderson-Buettner, A.S., Janitz, A.E., Doescher, M.P. et al. Financial hardship screening among Native American patients with cancer: a qualitative analysis. BMC Health Serv Res 24 , 928 (2024). https://doi.org/10.1186/s12913-024-11357-6

Download citation

Received : 30 January 2024

Accepted : 24 July 2024

Published : 14 August 2024

DOI : https://doi.org/10.1186/s12913-024-11357-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Financial hardship

BMC Health Services Research

ISSN: 1472-6963

q research software text analysis

Information

  • Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

  • Active Journals
  • Find a Journal
  • Proceedings Series
  • For Authors
  • For Reviewers
  • For Editors
  • For Librarians
  • For Publishers
  • For Societies
  • For Conference Organizers
  • Open Access Policy
  • Institutional Open Access Program
  • Special Issues Guidelines
  • Editorial Process
  • Research and Publication Ethics
  • Article Processing Charges
  • Testimonials
  • Preprints.org
  • SciProfiles
  • Encyclopedia

electronics-logo

Article Menu

q research software text analysis

  • Subscribe SciFeed
  • Recommended Articles
  • Google Scholar
  • on Google Scholar
  • Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Research on surface processing method of pulse transmission signal of amplitude-modulated drilling fluid in 10,000-m deep wells, 1. introduction.

  • Strong environmental resilience
  • Difficulty in signal recognition

2. Research on Mud Pulse Signal Recognition Algorithm

2.1. establishment of signal recognition model, 2.2. noise analysis, 2.3. signal processing algorithm, 2.3.1. discrete random signal filtering algorithm, 2.3.2. emd signal filtering algorithm.

  • Extraction of extremum points

2.4. Signal Feature Extraction

2.5. classification of feature groups using grey relational analysis algorithm.

  • Establishment of the pattern library matrix

3. Results Analysis and Discussion

3.1. recognition results of simulated signals, 3.2. recognition results of indoor experimental data, 4. conclusions, 5. future directions, author contributions, data availability statement, conflicts of interest, nomenclature.

tDuration
S(t)composite signal waveform
p(t)pulse with a duration of t
A signal to be transmitted
X and X elements of the signal to be transmitted
nnumber of pulses
k = [x]floor function
αwell inclination
scomplex frequency variable
H(s)filter’s frequency domain transfer function
τfilter’s time constant
h(t)unit impulse response of the filter
u(t)unit step function
O(i)filter’s output signal
I(i)filter’s input signal
Ffilter’s coefficient
O(i − 1)filter’s output signal at the previous time step
Tsampling frequency
X = (X , …, X , …, X )set of amplitude sequence patterns
Y = (Y , …, Y , …, Y )set of time-domain sequence encodings
Rpattern library matrix
Oinitialized amplitude sequence matrix
o elements from initialized amplitude sequence matrix
r elements from initialized pattern library matrix
c grey relational coefficient
Cgrey relational coefficient matrix
pintensity of the pulse signal
p initial intensity of the pulse signal
xtransmission distance
Lattenuation factor
ffrequency of the pulse signal
ωangular frequency of the pulse signal
ctransmission rate
Ddiameter of the pipeline
ρdensity of the mud
μviscosity of the mud

Click here to enlarge figure

  • Zhang, J.; Xie, W. Status of Scientific Drilling Technology for Ultra–Deep Well. Acta Geol. Sin. 2010 , 84 , 887–894. [ Google Scholar ]
  • Liu, H.; Liu, J.; Liu, H.; Qiu, J.; Cai, B.; Liu, J.; Yang, Z.; Liu, J. Progress and development direction of production test and reservoir stimulation technologies for ultra–deep oil and gas reservoirs in Tarim Basin. Nat. Gas Ind. 2020 , 40 , 76–88. [ Google Scholar ]
  • NOV. Available online: https://www.nov.com/products/e-totco-electronic-drift-survey-tool (accessed on 30 July 2024).
  • Peng, Q.; Li, H.; Zhang, Q. Application Status of the Technology of Logging While Drilling. Adv. Mater. Res. 2014 , 3384 , 1650–1653. [ Google Scholar ] [ CrossRef ]
  • Tang, Z.; Zhou, J.; Zhao, H.; Li, L.; Shi, B.; Zeng, M. Measurement and control technology while drilling for ultra–deep horizontal wells in Yuanba Gasfield. Oil Drill. Prod. Technol. 2015 , 37 , 54–57. [ Google Scholar ]
  • Rai, P.; Schunnesson, H.; Lindqvist, P.; Kumar, U. Measurement–while–drilling technique and its scope in design and prediction of rock blasting. Int. J. Min. Sci. Technol. 2016 , 26 , 711–719. [ Google Scholar ] [ CrossRef ]
  • Su, Y.; Dou, X.; Gao, K.; Liu, K. Discussion and prospects of the development on measurement while drilling technology in oil and gas wells. Pet. Sci. Bull. 2023 , 8 , 534–535. [ Google Scholar ]
  • Cai, W.; Wang, P.; Zhu, Y.; Chen, G. Design scheme and key techniques for mechanical wireless inclinometer. Acta Pet. Sin. 2006 , 2 , 103–106. [ Google Scholar ]
  • Sun, J.; Liu, T. Improvement and application of mechanical wireless inclinometer while drilling technology. Drill. Prod. Technol. 2012 , 35 , 108–110. [ Google Scholar ]
  • Liu, X. Research on Mechanism of Continuous Wave Signal Generator Controlled by DSP and the Wind Tunnel Simulation Test. Ph.D. Thesis, China University of Petroleum (East China), Qingdao, China, 2009. [ Google Scholar ]
  • Mcdonald, W.J. Four Different Systems Used for MWD. Oil Gas J. 1978 , 76 , 115–124. [ Google Scholar ]
  • Gearhart, M.; Moseley, L.M.; Foste, M. Current State of the Art of MWD and Its Application in Exploration and Development Drilling. In Proceedings of the International Meeting on Petroleum Engineering, Beijing, China, 17–20 March 1986. [ Google Scholar ]
  • Liang, Y.; Ju, X.; Li, A.; Li, C.; Dai, Z.; Ma, L. The Process of High–Data–Rate Mud Pulse Signal in Logging While Drilling System. Math. Probl. Eng. 2020 , 2020 , 3207087. [ Google Scholar ] [ CrossRef ]
  • Tu, B.; Li, S.; Lin, H.; Ji, M. Research on mud pulse signal data processing in MWD. EURASIP J. Adv. Signal Process. 2012 , 2012 , 182. [ Google Scholar ] [ CrossRef ]
  • Mwachaka, M.; Wu, A.; Fu, Q. A review of mud pulse telemetry signal impairments modeling and suppression methods. J. Pet. Explor. Prod. Technol. 2018 , 9 , 779–792. [ Google Scholar ] [ CrossRef ]
  • Pálsson, B.; Hólmgeirsson, S.; Guðmundsson, Á.; Bóasson, H.Á.; Ingason, K.; Sverrisson, H.; Thórhallsson, S. Drilling of the well IDDP–1. Geothermics 2014 , 49 , 23–30. [ Google Scholar ] [ CrossRef ]
  • NOV. Available online: https://www.nov.com/products/teledrift-prodrift-mwd-tool (accessed on 30 July 2024).
  • drillingforgas.com. Available online: https://drillingforgas.com/en/drilling/directional/teledrift-inclination-only-mwd (accessed on 30 July 2024).
  • Li, L.; Deng, H.; Huang, C.; Zhang, J.; Xie, Y.; Wan, F.; Zeng, P.; Jia, L.; He, X.; Duan, M.; et al. Method for Siginal Transmission Based on Mud Pulse Pressure Amplitide. China Patent CN110500087A, 26 November 2019. [ Google Scholar ]
  • Zhang, F.; Ding, K. Research on the Three Algorithms and Limitations of Generalized Detection Filtering Demodulation Analysis. J. Vib. Eng. 2002 , 2 , 123–128. [ Google Scholar ]
  • Li, Y. Research on Demodulation Method of Gear Fault Vibration Amplitude and Frequency Modulation Signal. Master’s Thesis, South China University of Technology, Guangzhou, China, 2021. [ Google Scholar ]
  • Qu, J.; Xue, Q.; Lu, J. Analysis on the change characteristics of waveform during the transmission of continuous wave mud pulse signal. Alex. Eng. J. 2023 , 80 , 594–608. [ Google Scholar ] [ CrossRef ]
  • Chen, G.; Yan, Z.; Gao, T.; Sun, H.; Li, G.; Wang, J. Study on model–based pump noise suppression method of mud pulse signal. J. Pet. Sci. Eng. 2021 , 200 , 108433. [ Google Scholar ] [ CrossRef ]
  • Deng, J. Grey System Theory Tutorial ; Huazhong University of Science & Technology Press: Wuhan, China, 1990; pp. 21–24. [ Google Scholar ]
  • Zhang, J.; Zhang, Q.; Zhang, J. The result greyness problem of the grey relational analysis and its solution. J. Intell. Fuzzy Syst. 2023 , 44 , 6079–6088. [ Google Scholar ] [ CrossRef ]
Type of Encoding SequenceCorresponding Pattern
1 1 3 2 3 2inclination 1°–10°
1 3 2 3 2 1inclination 11°–20°
3 2 3 2 1 1inclination 21°–30°
2 3 2 1 1 3inclination 31°–40°
3 2 1 1 3 2inclination 41°–50°
2 1 1 3 2 3inclination 51°–60°
Index of PositionFeature Value
10.36
20.36
30.42
40.38
50.42
60.37
PatternCorrelation Coefficient
inclination 1°–10°0.931
inclination 11°–20°0.812
inclination 21°–30°0.704
inclination 31°–40°0.728
inclination 41°50°0.705
inclination 51°60°0.767
ParametersParameter Value
Pipeline Length500/10,000 m
Pipeline MaterialAlloy Steel
Pipeline Diameterφ130 mm
Outer Diameter of the Pipelineφ174 mm
Figure Number
(500 m)
Figure Number
(10,000 m)
Identification Results (500 m)Identification Results (10,000 m)
(a)(a)inclination 1°–10°inclination 1°–10°
(b)(b)inclination 11°–20°inclination 11°–20°
(c)(c)inclination 21°–30°inclination 21°–30°
(d)(d)inclination 31°–40°inclination 31°–40°
(e)(e)inclination 41°–50°inclination 41°–50°
(f)(f)inclination 51°–60°inclination 51°–60°
PatternCorrelation Coefficient
inclination 1°–10°0.8494
inclination 11°–20°0.7989
inclination 21°–30°0.7487
inclination 31°–40°0.7682
inclination 41°50°0.7483
inclination 51°60°0.8100
The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

Wang, Q.; Ji, G.; Guo, J.; Wu, K.; Mei, C.; Zeng, L.; Xue, Q. Research on Surface Processing Method of Pulse Transmission Signal of Amplitude-Modulated Drilling Fluid in 10,000-m Deep Wells. Electronics 2024 , 13 , 3231. https://doi.org/10.3390/electronics13163231

Wang Q, Ji G, Guo J, Wu K, Mei C, Zeng L, Xue Q. Research on Surface Processing Method of Pulse Transmission Signal of Amplitude-Modulated Drilling Fluid in 10,000-m Deep Wells. Electronics . 2024; 13(16):3231. https://doi.org/10.3390/electronics13163231

Wang, Qing, Guodong Ji, Jianhua Guo, Ke Wu, Chao Mei, Long Zeng, and Qilong Xue. 2024. "Research on Surface Processing Method of Pulse Transmission Signal of Amplitude-Modulated Drilling Fluid in 10,000-m Deep Wells" Electronics 13, no. 16: 3231. https://doi.org/10.3390/electronics13163231

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

MDPI

Subscribe to receive issue release notifications and newsletters from MDPI journals

  • Automatic Updating
  • New Product Development
  • Tracking Analysis
  • PowerPoint Reporting
  • Customer Feedback
  • Data Visualization
  • Segmentation
  • Brand Analytics
  • Pricing Research
  • Survey Analysis
  • Data Stories
  • Advertising Research
  • Data Cleaning
  • Driver Analysis
  • Statistical testing
  • Factor Analysis
  • Correspondence Analysis
  • Cluster & Latent Class
  • Text Analysis
  • Customer Stories
  • Market Research Guide
  • The Data Story Academy
  • The Data Story Guide
  • Q Help Guide
  • Hire a Researcher

FEATURED READ

q research software text analysis

How to Fit a Structural Equation Model in Q

Get the blog

  • 877-401-1897

On-demand webinars

Text Analysis in Q

Text Analysis in Q

DIY Segmentation

DIY Segmentation

How to Create Variables in Q

How to Create Variables in Q

Mastering Driver Analysis

Mastering Driver Analysis

Discover the Top Six Techniques of Pricing Research

Discover the Top Six Techniques of Pricing Research

Live Q&A on Brand Tracking – Episode #2

Live Q&A on Brand Tracking – Episode #2

Take Back your Time with Brand Tracking Studies that Automatically Update with Displayr

Take Back your Time with Brand Tracking Studies that Automatically Update with Displayr

How to Quickly Analyze Text Data

How to Quickly Analyze Text Data

How to create visualizations to synthesize survey data

How to create visualizations to synthesize survey data

Five Key Data Analysis Frameworks for Building Strong Brand Strategy

Five Key Data Analysis Frameworks for Building Strong Brand Strategy

Making TURF for New Product Development Awesome in Less than 20 Mins

Making TURF for New Product Development Awesome in Less than 20 Mins

DIY Market Segmentation

DIY Market Segmentation

Using Excel for Data Analysis? There’s a Better Way.

Using Excel for Data Analysis? There’s a Better Way.

DIY Data Science Toolkit for Market Researchers

DIY Data Science Toolkit for Market Researchers

Halve your Analysis and Reporting Time!

Halve your Analysis and Reporting Time!

How to do choice modeling (choice-based conjoint; CBC) webinar!

How to do choice modeling (choice-based conjoint; CBC) webinar!

Learn how to Transform your Survey Data Easier and Quicker!

Learn how to Transform your Survey Data Easier and Quicker!

Market research dashboards without the work!

Market research dashboards without the work!

Extreme PowerPoint Makeover for Market Research

Extreme PowerPoint Makeover for Market Research

Goodbye SPSS, Hello Q!

Goodbye SPSS, Hello Q!

Discover New Ways to Visualize Market Research Data

Discover New Ways to Visualize Market Research Data

How to Set Up Data for Fast, Error-Free Analysis

How to Set Up Data for Fast, Error-Free Analysis

DIY Market Research Dashboards – Building 4 in 40 minutes

DIY Market Research Dashboards – Building 4 in 40 minutes

Isn’t it time to ditch your painful market research software?

Isn’t it time to ditch your painful market research software?

DIY Market Mapping Using Correspondence Analysis

DIY Market Mapping Using Correspondence Analysis

8 Ways to massively improve quant researcher productivity

8 Ways to massively improve quant researcher productivity

DIY Max-Diff

DIY Max-Diff

  • First name *
  • Last name *
  • Email address *

Discover how much time you can save by analyzing with Q

Book demo Free trial

Keep updated with the lastest in market research

Want to learn more about q.

Book a free demo to learn about how to halve your analysis time by using Q.

IMAGES

  1. Text Analysis in Q

    q research software text analysis

  2. Market Research Guide: 12. Q Tutorial

    q research software text analysis

  3. Text Analysis

    q research software text analysis

  4. Text Analysis in Q

    q research software text analysis

  5. Q Research Software

    q research software text analysis

  6. The Best Qualitative Data Analysis Software

    q research software text analysis

COMMENTS

  1. Text Analysis in Q

    Discover how much time you can save by analyzing with Q Book demo Free trial

  2. Text Analysis

    Coding Text Data in Q. Coding Text Data in Older Versions of Q. How to Semi-Automatically Code Text Data. How to Automatically Code Simple Text Variables.

  3. Text Analysis

    To create a Setup Text Analysis output, go to Create > Text Analysis > Advanced > Setup Text Analysis. Under Inputs > Text Analysis Options > Text Variable select a Text variable. Make any other selections that you require. Ensure the Automatic box is checked, or click Calculate. The output generated by this function will look like the below.

  4. Text Analysis

    Text Analysis - Sentiment. Related Videos. Chapter within Text Analysis in Q5 (Video) Generate a variable which assigns scores to a set of text responses which attempts to quantify how positive or negative each response is. This can be done based on raw text in the data set, or it can be based on text which has been processed using Text ...

  5. Q Research Software

    Helping market researchers quickly find the story in their data. Q is analysis software designed just for market research.

  6. Category:Text Analysis

    Category:Text Analysis. Text Analysis can be done by selecting Create > Text Analysis. For an overview of text analysis in Q, see our case study. All options in the menu, except the word cloud, use R ( ). The initial stage of most text analyses is to do some pre-processing of the text, which includes breaking the text up into separate words and ...

  7. How to Map at Text Analysis

    Introduction This article describes how to create a 2-dimensional map visualizing the tokens in a Text Analysis output based on their frequency patterns. The area of the markers is proportional to the number of respondents who used the token, whereas the position is determined by the selected dimension-reduction method (e.g., , , ).

  8. Text Analysis in Q5 (Video)

    Text Analysis in Q5 (Video) The following is a webinar presented by Chris Facer on the text analysis features in the beta release of Q5 - October 2016.

  9. Q Research Software

    Q Research Software is analysis and reporting software designed to help market researchers quickly find the story in their survey data. Automates the grunt work - Everything that can be automated is automated, including cleaning & formatting data, statistical testing, generating tables, updating analyses and reproducing reports. DIY advanced ...

  10. Research Guides: Analyzing Text Data: Text Analysis Methods

    Sentiment analysis is a method of analyzing text to determine whether the emotional tone or sentiment expressed in a piece of text is positive, negative, or neutral. Sentiment analysis is commonly used in businesses to gauge customer feedback, social media monitoring, and market research. Software for Sentiment Analysis.

  11. Sage Research Methods

    Qualitative text analysis is ideal for these types of data and this textbook provides a hands-on introduction to the method and its theoretical underpinnings. It offers step-by-step instructions for implementing the three principal types of qualitative text analysis: thematic, evaluative, and type-building. Special attention is paid to how to present your results and use qualitative data ...

  12. Text Analysis: Definition, Benefits & Examples

    Free Account. Text analysis: definition, benefits & examples. 45 min read Find out how text analysis software works, and how you can use it to find breakthrough insights in unstructured data to take your customer, employee, brand, and product experience programs to another level. Written by Rohan Sinha, Senior Principal CX at Qualtrics.

  13. KADE is an open-source application for the analysis of Q methodology

    KenQ Analysis Desktop Edition (KADE) is a desktop application for the analysis of Q methodology data which operates with full functionality on Microsoft Windows, Apple macOS, and Linux operating systems. It has a graphic user interface that makes it easy to use for students or for researchers with a background mainly in qualitative methods. Installation is not required for use; KADE is ...

  14. Text Analysis Archives

    How to Automatically Extract Entities and Sentiment from Text. by Kris Tonthat. Advanced Analysis Techniques | How To | Text Analysis | Using Q.

  15. Text Analysis Methods

    A guide to text mining tools and methods A description of which methods to try, depending on your dataset and research question.

  16. Qualitative Data Analysis Program

    The Qualitative Data Analysis Program (QDAP): Accurate and reliable coding (or labeling) using humans and machines is central to both the qualitative research and the #bigdata text analytics process. For new and experienced researchers, learning advanced project management skills adds hours of struggle. Since 2005, Qualitative Data Analysis ...

  17. GitHub

    Qualitative data analysis for text, images, audio, video. Cross platform. Python 3.10 or newer and PyQt6. - ccbogel/QualCoder

  18. Research Guides: Digital Humanities: Text Analysis

    The Text Analysis Portal for Research (TAPoR) is a gateway to the tools used in sophisticated text analysis and retrieval. Tool categories include analysis, annotating, Natural Language Processing (NLP), and web development among many others. Voyant Tools is a web-based text reading and analysis environment.

  19. Text_Analysis

    Try Q now Discover how much time you can save by analyzing with Q. Book demo Free trial

  20. Text Analysis Software List (August 2024)

    Text analysis software uses natural language processing (NLP) to extract valuable insights from structured and unstructured text data. This tool enables professionals to understand the sentiment analysis, key phrases, language nuances, recurring themes, and patterns that emerge from the data. Text Analytics software can process data from various channels, including emails, phone call ...

  21. Getting Started with Q

    Q makes it easy for market researchers to discover the stories in their data. At Q, we understand market research and the structure of survey data, so Q makes it quick and easy to generate tables, crosstabs, and other analyses without syntax writing, and allows you to export your results directly in Microsoft PowerPoint, Excel, Word or PDF. In this post, I take you through a simple workflow in ...

  22. Financial hardship screening among Native American patients with cancer

    Cancer-related financial hardship is an increasingly recognized problem for patients, families, and caregivers [1,2,3,4,5,6,7,8].As a relatively new field of study, financial hardship screening among patients with cancer is being studied in diverse settings with diverse instruments [].In a recent study, researchers looked at both medical and nonmedical financial hardship and sacrifice among ...

  23. Remote Sensing

    This paper investigates the impact of flood-induced hydrodynamic forces and high discharge on the masonry arch "Hannibal Bridge" (called "Ponte di Annibale" in Italy) using the Hydraulic Engineering Center's River Analysis Simulation (HEC-RAS) v6.5.0. hydraulic numerical method, incorporating Unmanned Aerial Vehicle (UAV) photogrammetry and aerial Light Detection and Ranging (LIDAR ...

  24. Nutrients

    The objective was to ascertain the efficacy and safety of the EAHM with NB-UVB combination therapy and to identify important drugs for further research. In this study, randomized controlled trials (RCTs) were retrieved from ten databases in Korea, China, and Japan. All statistical analyses were conducted using R software version 4.3.0.

  25. Features

    Use the latest machine learning and statistical techniques. Smart systems and automation for popular methods and full support of the R language.

  26. Electronics

    Li, Y. Research on Demodulation Method of Gear Fault Vibration Amplitude and Frequency Modulation Signal. Master's Thesis, South China University of Technology, Guangzhou, China, 2021. [Google Scholar] Qu, J.; Xue, Q.; Lu, J. Analysis on the change characteristics of waveform during the transmission of continuous wave mud pulse signal. Alex. Eng.

  27. Webinars Archive

    Learn how to Transform your Survey Data Easier and Quicker! View webinar & Resources.