This vignette walks through various plot options available in quanteda through the
The frequency of features can be plotted as a wordcloud using
You can also plot a “comparison cloud”, but this can only be done with fewer than eight documents:
Plot will pass through additional arguments to the underlying call to
## Warning: colors is deprecated; use color instead
kwic object produces a lexical dispersion plot which allows us to visualize the occurrences of particular terms throughout the text. We call these “x-ray” plots due to their similarity to the data produced by Amazon’s “x-ray” feature for Kindle books.
You can also pass multiple kwic objects to
plot to compare the dispersion of different terms:
If you’re only plotting a single document, but with multiple keywords, then the keywords are displayed one below the other rather than side-by-side.
## Loading required package: readtext
You might also have noticed that the x-axis scale is the absolute token index for single texts and relative token index when multiple texts are being compared. If you prefer, you can specify that you want an absolute scale:
In this case, the texts may not have the same length, so the tokens that don’t exist in a particular text are shaded in grey.
The object returned is a ggplot object, which can be modified using ggplot:
library("ggplot2") theme_set(theme_bw()) g <- textplot_xray( kwic(data_corpus_inaugural_subset, "american"), kwic(data_corpus_inaugural_subset, "people"), kwic(data_corpus_inaugural_subset, "communist") ) g + aes(color = keyword) + scale_color_manual(values = c("blue", "red", "green")) + theme(legend.position = "none")
You can plot the frequency of the top features in a text using
features_dfm_inaug <- textstat_frequency(dfm_inaug, n = 100) # Sort by reverse frequency order features_dfm_inaug$feature <- with(features_dfm_inaug, reorder(feature, -frequency)) ggplot(features_dfm_inaug, aes(x = feature, y = frequency)) + geom_point() + theme(axis.text.x = element_text(angle = 90, hjust = 1))
If you wanted to compare the frequency of a single term across different texts, you can also use
textstat_frequency, group the frequency by speech and extract the term.
# Get frequency grouped by president freq_grouped <- textstat_frequency(dfm(data_corpus_inaugural_subset), groups = "President") # Filter the term "american" freq_american <- subset(freq_grouped, freq_grouped$feature %in% "american") ggplot(freq_american, aes(x = group, y = frequency)) + geom_point() + scale_y_continuous(limits = c(0, 14), breaks = c(seq(0, 14, 2))) + xlab(NULL) + ylab("Frequency") + theme(axis.text.x = element_text(angle = 90, hjust = 1))
The above plots are raw frequency plots. For relative frequency plots, (word count divided by the length of the chapter) we need to weight the document-frequency matrix first. To obtain expected word frequency per 100 words, we multiply by 100.
## Document-feature matrix of: 6 documents, 4,215 features (85.1% sparse).
rel_freq <- textstat_frequency(dfm_rel_freq, groups = "President") # Filter the term "american" rel_freq_american <- subset(rel_freq, feature %in% "american") ggplot(rel_freq_american, aes(x = group, y = frequency)) + geom_point() + scale_y_continuous(limits = c(0, 0.7), breaks = c(seq(0, 0.7, 0.1))) + xlab(NULL) + ylab("Relative frequency") + theme(axis.text.x = element_text(angle = 90, hjust = 1))
texstat_frequency allows to plot the most frequent words in terms of relative frequency by group.
dfm_weight_pres <- data_corpus_inaugural %>% corpus_subset(Year > 2000) %>% dfm(remove = stopwords("english"), remove_punct = TRUE) %>% dfm_weight(scheme = "prop") # Calculate relative frequency by president freq_weight <- textstat_frequency(dfm_weight_pres, n = 10, groups = "President") ggplot(data = freq_weight, aes(x = nrow(freq_weight):1, y = frequency)) + geom_point() + facet_wrap(~ group, scales = "free") + coord_flip() + scale_x_continuous(breaks = nrow(freq_weight):1, labels = freq_weight$feature) + labs(x = NULL, y = "Relative frequency")
If you want to compare the differential associations of keywords in a target and reference group, you can calculate “keyness” which is based on
textstat_keyness. In this example, we compare the inaugural speech by Donald Trump with the speeches by Barack Obama.
# Only select speeches by Obama and Trump pres_corpus <- corpus_subset(data_corpus_inaugural, President %in% c("Obama", "Trump")) # Create a dfm grouped by president pres_dfm <- dfm(pres_corpus, groups = "President", remove = stopwords("english"), remove_punct = TRUE) # Calculate keyness and determine Trump as target group result_keyness <- textstat_keyness(pres_dfm, target = "Trump") # Plot estimated word keyness textplot_keyness(result_keyness)
You can also plot fitted Wordscores (Laver et al., 2003) or Wordfish scaling models (Proksch and Slapin, 2008).
Wordscores is a scaling procedure for estimating policy positions or scores (Laver et al., 2003). Known scores are assigned to so called reference texts in order to infer the positions of new documents (“virgin texts”). You can plot the position of words (features) against the logged term frequency, or the position of the documents.
# Transform corpus to dfm ie_dfm <- dfm(data_corpus_irishbudget2010, verbose=FALSE) # Set reference scores refscores <- c(rep(NA, 4), 1, -1, rep(NA, 8)) # Predict Wordscores model ws <- textmodel_wordscores(ie_dfm, refscores, smooth = 1) # Plot estimated word positions (highlight words and print them in red) textplot_scale1d(ws, highlighted = c("minister", "have", "our", "budget"), highlighted_color = "red")
# Set document labels for y-axis doclab <- apply(docvars(data_corpus_irishbudget2010, c("name", "party")), 1, paste, collapse = " ") # Get predictions pred <- predict(ws, se.fit = TRUE) # Plot estimated document positions and group by "party" variable textplot_scale1d(pred, margin = "documents", doclabels = doclab, groups = docvars(data_corpus_irishbudget2010, "party"))
Wordfish is a Poisson scaling model that estimates one-dimension document positions using maximum likelihood (Slapin and Proksch, 2008). Both the estimated position of words and the positions of the documents can be plotted.
# Estimate Wordfish model wf <- textmodel_wordfish(dfm(data_corpus_irishbudget2010), dir = c(6,5)) # Plot estimated word positions textplot_scale1d(wf, margin = "features", highlighted = c("government", "global", "children", "bank", "economy", "the", "citizenship", "productivity", "deficit"), highlighted_color = "red")
You can also plot the estimated document positions of a correspodence analysis (Nenadic and Greenacre 2007).
# Transform corpus to dfm ie_dfm <- dfm(data_corpus_irishbudget2010) # Run correspondence analysis on dfm ca <- textmodel_ca(ie_dfm) # Create doclabels for y-axis doclab <- apply(docvars(data_corpus_irishbudget2010, c("name", "party")), 1, paste, collapse = " ") # Plot estimated positions and group by party textplot_scale1d(ca, margin = "documents", doclabels = doclab, groups = docvars(data_corpus_irishbudget2010, "party"))
Laver, Michael, Kenneth Benoit, and John Garry. 2003. “Extracting Policy Positions from Political Texts Using Words as Data.” American Political Science Review 97(2):311-331.
Nenadic, Oleg, and Michael Greenacre. 2007. “Correspondence analysis in R, with two- and three-dimensional graphics: The ca package.” Journal of Statistical Software 20(3): 1-13. URL: http://www.jstatsoft.org/v20/i03/.
Slapin, Jonathan, and Sven-Oliver Proksch. 2008. “A Scaling Model for Estimating Time-Series Party Positions from Texts.” American Journal of Political Science 52(3):705-772.