If you are using a Textbox and/or Essay / Long Answer question to collect open-ended responses in your survey, you can now leverage Sentiment Analysis in your Standard Report.
Alchemer uses Natural Language Toolkit (NLTK); specifically the Valance Aware Dictionary and sEntiment Reasoner (VADER) Sentiment Analysis Package1 to analyze your respondents' text responses for sentiment. Once the text is analyzed, a sentiment score is stored. You can then add a Sentiment Element to your Standard Report to show the aggregate sentiment for a given compatible question in your survey.
What is Sentiment Analysis?
Sentiment analysis2 is the process of identifying (statistically) whether a piece of text is positive, negative, or neutral. In the case of Alchemer, the text refers to a respondent's answer to one of the compatible question types.
Compatibility
Sentiment Analysis cannot be performed retroactively. It is only performed on responses collected after the release of this feature.
At this time, sentiment analysis is performed on English language surveys only. A sentiment score of 0 (neutral sentiment) will be returned for non-English open-ended responses.
Sentiment Analysis is not performed on responses collected via Offline Mode.
The VADER Sentiment Analysis Package is automatically run on open-ended responses collected via the following question types*:
If either of the above questions is included as a sub-question of a Custom Group question, sentiment analysis will also be performed.
Limitations
- The above question types will only be analyzed for sentiment when validation is NOT applied. If you are not sure what validation is, visit the Validate Textbox Answer Formats article.
- Only the first 2500 characters entered will be analyzed.
Sentiment Scoring
After a respondent's open-ended response is analyzed, a sentiment score is assigned to the response. The score ranges from -1 to 1. Depending on the specific score returned, the sentiment will be categorized as one of the following:
| Negative Sentiment | Neutral Sentiment | Positive Sentiment |
| ≥ -1 and < - 0.3 | ≥ - 0.3 and ≤ 0.3 | > 0.3 and ≤ 1 |
How is Sentiment Calculated?
Visit our Sentiment Analysis documentation to learn the ins and outs.
Sentiment in the Standard Report
If your survey utilizes one of the question types that is compatible with Sentiment Analysis (Textbox, Essay / Long Answer), you will have access to a new Sentiment Question Chart reporting element within the Standard Report.
- The Sentiment element will display the sentiment score for the associated question on a scale of -1 to +1 (in increments of .1).
- The sentiment will be color-coded along a range from red (negative) to gray (neutral) to green (positive).
- Each individual score (x-axis) will display an associated response count (along the y-axis). The taller the bar, the more responses with that specific sentiment score.
Add Sentiment Chart
- To add a Sentiment Chart to your Standard Report, use the Insert tool between any two report elements.
- After clicking Insert, choose the Question Chart option.
- Locate the Textbox or Essay / Long Answer question for which you want to add a sentiment chart. Directly beneath the question, you will see an indented option that reads Sentiment - [Question Title]. Click on this option to select it.
- Click Add Question to add the chart.
Customize Sentiment Chart
After you have added a Sentiment Chart, there are a couple of things that you may want to customize.
- Click the Chart Type link to the right of the chart to update the chart title. By default, the chart will be titled as "Sentiment: [Question Title]".
- Click the Filter tab to apply a filter to the Sentiment Chart. You can filter based on prior question answers as well as by link source, among other options.
Make sure to Save any changes that you made to the Sentiment Element.
Export Sentiment Score
The sentiment score associated with each respondent's open-ended answer can also be included in a CSV / Excel Export of your survey data.
- When you create an Export via Results > Exports, you will see a number of initial display options, including Include Sentiment Score Data.
- Check the box associated with this option to include the sentiment score in your export file.
For sentiment score data to be included in your export you must click "Create Report". Sentiment score data will not be included in your export if you click "Download All Data Now".
If a question is compatible with Sentiment Analysis, a Sentiment column will be included directly after the question, within your Excel file.
Download Sentiment Score via Individual Responses
When you download and individual response to a PDF, you will have the option to include sentiment scores within the PDF.
On the Customize your response PDF screen, check the box associated with the Include Sentiment Scores option.
If a question is compatible with Sentiment Analysis, the sentiment score will be included directly after the question, within your PDF file.
Further Reading
- VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text
- VADER Sentiment
- VADER lexicon file
- VADER Sentiment Analysis Explained
- Using VADER to handle sentiment analysis with social media text
Sources
1. Hutto, C.J. & Gilbert, E.E. (2014). VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Eighth International Conference on Weblogs and Social Media (ICWSM-14). Ann Arbor, MI, June 2014.
2. “Using VADER to handle sentiment analysis with social media text,” Standard error, last modified April 08, 2017, http://t-redactyl.io/blog/2017/04/using-vader-to-handle-sentiment-analysis-with-social-media-text.html.
3. "VADER Sentiment Analysis Explained," DATA meets MEDIA, last modified April 10, 2017, http://datameetsmedia.com/vader-sentiment-analysis-explained/.
