dfm 之前的搭配和复合

Question

我想用文本列来查找短语，所以我尝试搭配选项：

library(quanteda)

dataset1 <- data.frame( anumber = c(1,2,3), text = c("Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book.","It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum", "Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source."))

    cols <- textstat_collocations(dataset1 $text, size = 2:3, min_count = 30)

之后使用化合物作为他们的 frq 试试这个：

inputforDfm <- tokens_compound(cols)

Error in tokens_compound.default(cols) : tokens_compound() only works on tokens objects.

但它需要代币？如何制作并插入 dfm :

myDfm <- dataset1 %>%
corpus() %>%
tokens(remove_punct = TRUE, remove_numbers = TRUE, remove_symbols = TRUE) %>%
dfm()

Answer 1

您需要对文本进行标记，因为标记复合需要标记对象作为其第一个参数。

library(quanteda)
## Package version: 2.1.1

这里我将其更改为 min_count = 2，否则您 return 在此示例中没有搭配，因为 none 在文本中出现 30 次或更多次！

cols <- textstat_collocations(dataset1$text, size = 2:3, min_count = 2)

复合后，现在我们可以看到代币之间的复合：

toks <- tokens(dataset1$text) %>%
  tokens_compound(cols)

print(toks)
## Tokens consisting of 3 documents.
## text1 :
##  [1] "Lorem_Ipsum_is" "simply"         "dummy_text"     "of_the"        
##  [5] "printing"       "and"            "typesetting"    "industry"      
##  [9] "."              "Lorem_Ipsum"    "has"            "been"          
## [ ... and 28 more ]
## 
## text2 :
##  [1] "It_has"    "survived"  "not"       "only"      "five"      "centuries"
##  [7] ","         "but"       "also"      "the"       "leap"      "into"     
## [ ... and 37 more ]
## 
## text3 :
##  [1] "Contrary"       "to"             "popular"        "belief"        
##  [5] ","              "Lorem_Ipsum_is" "not"            "simply"        
##  [9] "random"         "text"           "."              "It_has"        
## [ ... and 63 more ]

现在以通常的方式创建 dfm，我们可以通过仅选择这些化合物来查看化合物：

dfm(toks) %>%
  dfm_select(pattern = "*_*")
## Document-feature matrix of: 3 documents, 5 features (33.3% sparse).
##        features
## docs    lorem_ipsum_is dummy_text of_the lorem_ipsum it_has
##   text1              1          2      1           1      0
##   text2              0          0      0           2      1
##   text3              1          0      2           1      1

dfm 之前的搭配和复合

Collocation and compound before a dfm

r

quanteda