library(tidyverse)
library(tidytext) # functions for doing text analysis
library(wordcloud)15-sentiment
This page contains code from lecture and, at the bottom, the end of class activity
Load the Data
Random (?) sample of 26,882 reviews of coursera courses (Source: Kaggle)
en_coursera_reviews <- read_csv("https://stat220kurtz.github.io/data/en_coursera_sample.csv")
en_coursera_reviews# A tibble: 26,882 × 5
CourseId Review Label cld2 review_id
<chr> <chr> <dbl> <chr> <dbl>
1 nurture-market-strategies It would be better if the … 1 en 1
2 nand2tetris2 Superb course. Great prese… 5 en 2
3 schedule-projects Excellent course! 5 en 3
4 teaching-english-capstone-2 I'd recommend this course … 5 en 4
5 machine-learning This course was so effecti… 5 en 5
6 python-network-data Words cannot describe how … 5 en 6
7 clinical-trials Great course! 5 en 7
8 python-genomics I didn't know anything abo… 3 en 8
9 strategic-management Loved everything about thi… 5 en 9
10 script-writing No significant instruction… 1 en 10
# ℹ 26,872 more rowsLoad Sentiment data
“bing”
bing_sentiments = get_sentiments("bing") %>%
slice_sample(n = 20)“afinn”
library(textdata)
get_sentiments("afinn") %>%
slice_sample(n = 20)# A tibble: 20 × 2
word value
<chr> <dbl>
1 erroneous -2
2 exaggerate -2
3 accuses -2
4 alas -1
5 stop -1
6 inspiring 3
7 endorsement 2
8 convince 1
9 achievable 1
10 agree 1
11 complacent -2
12 mistake -2
13 consent 2
14 comprehensive 2
15 clouded -1
16 falsified -3
17 rich 2
18 derail -2
19 denied -2
20 accidents -2Sentiment of each review
bing_review_scores <- en_coursera_reviews %>%
unnest_tokens(word, Review) %>%
inner_join(bing_sentiments, by = "word") %>%
group_by(review_id) %>%
summarize(
sum = (sum(sentiment == "positive") - sum(sentiment == "negative"))
)
bing_review_scoresSherlock Holmes Activity
This activity was created based on an STM tutorial available at https://juliasilge.com/blog/sherlock-holmes-stm/
The gutenbergr contains data on works collected as a part of Project Gutenberg, a library of over 75,000 free eBooks. You can use the gutenberg_download() function to download a work or works by their Gutenberg ID into a dataset where each row represents a line in the work. You can look up the Gutenberg ID using the gutenberg_works() function.
The Sherlock Holmes short stories are a part of Project Gutenberg, and their ID is 1661. We can download the raw data with the following code.
library(gutenbergr)
sherlock_raw <- gutenberg_download(1661)We can clean the dataset up a bit so that there is a story column that says which of the twelve short stories each line of text belongs to.
sherlock <- sherlock_raw %>%
mutate(story = ifelse(str_detect(text, "ADVENTURE"),
text,
NA)) %>%
fill(story) %>%
filter(story != "THE ADVENTURES OF SHERLOCK HOLMES") %>%
mutate(story = factor(story, levels = unique(story)))
sherlock# A tibble: 6,099 × 3
gutenberg_id text story
<int> <chr> <fct>
1 1661 "VII. THE ADVENTURE OF THE BLUE CARBUNCLE" VII.…
2 1661 "" VII.…
3 1661 "" VII.…
4 1661 "I had called upon my friend Sherlock Holmes upon the sec… VII.…
5 1661 "after Christmas, with the intention of wishing him the c… VII.…
6 1661 "the season. He was lounging upon the sofa in a purple dr… VII.…
7 1661 "pipe-rack within his reach upon the right, and a pile of… VII.…
8 1661 "morning papers, evidently newly studied, near at hand. B… VII.…
9 1661 "was a wooden chair, and on the angle of the back hung a … VII.…
10 1661 "disreputable hard-felt hat, much the worse for wear, and… VII.…
# ℹ 6,089 more rowsFirst, try to make a word cloud of the data overall. You’ll want to use unnest_tokens() to get convert the dataset so that each row is a word instead of a line. Feel free to try it out on your own, using the sample code from lecture, or don’t be afraid to follow the tutorial cited at the beginning of this exercise if you get stuck.
Once you have created a word cloud for all short stories, feel free to make any aesthetic changes, or try out word clouds for individual short stories to see how words used change by story. Consider removing stop words.
The tutorial walks us through the process of finding the words with the highest tf-idf scores in each short story. Follow that code. What sorts of “topics” seem to make up each of the short stories? Do some short stories seem to have common themes with others? Do any stick out?
If you’d like, run some sentiment analysis on the short stories.