如何从语料库中提取特定文本？
How to extract specific text from a corpus?
我有一个包含 213 个文档的语料库，这些文档的长度各不相同。我的目标是从每个文档中提取一段特定的文本，它指的是 "fiscal policy"。使我的尝试复杂化的是我想要提取的文本位在文本之间是不一样的。唯一经常出现在开头的关键词是 财政政策 或 财政政策，仅此而已。
举个例子：
df <- data.frame(Text = c("Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. Regarding regards fiscal policies, almost all euro area countries have submitted their updated stability programmes. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. Against the background of current good times, it is essential that sound budgetary positions are reached in countries with fiscal imbalances and that a pro-cyclical loosening is avoided in all member countries. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. Turning to fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. MORE TEXT"))
cp <- corpus (df)
最终目的是得到这样的语料库：
df <- data.frame(Text = c("As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future.", "Regarding regards fiscal policies, almost all euro area countries have submitted their updated stability programmes.", "As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. Against the background of current good times, it is essential that sound budgetary positions are reached in countries with fiscal imbalances and that a pro-cyclical loosening is avoided in all member countries.", "Turning to fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes."))
cp <- corpus(df)
请注意，即使我只是感兴趣的一点加上我不想要的 "MORE TEXT" 我也会很高兴。我可以简单地对其进行子集化。我没有设法到达那里。到目前为止，我已尝试使用 corpus_segment 以及使用数据帧失败。
有人可以帮我吗？
非常感谢！
不需要语料库函数的Base R解决方案：
trimws(grep("fiscal polic.*", unlist(strsplit(df$Text, "[.]")), ignore.case = TRUE, value = TRUE), "both")
回答进一步的问题——找到索引并使用它的子集数据：
# Return vector of sentences containing pattern:
trimws(grep("fiscal polic.*", unlist(strsplit(df$Text, "[.]")), ignore.case = TRUE, value = TRUE), "both")
# Store the matched text as a vector:
matched_text <- trimws(grep("fiscal .*", unlist(strsplit(df$Text, "[.]")), ignore.case = TRUE, value = TRUE), "both")
#Get the index of the dataframe for each element:
matched_text_idx <- sapply(matched_text, function(x){which(grepl(x, df$Text))})
# If you want to subset the dataframe to contain only the elements which contain pattern:
df$Text[(which(grepl("fiscal polic.*", df$Text)))]
数据：
df <- data.frame(Text = c("Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. Regarding regards fiscal policies, almost all euro area countries have submitted their updated stability programmes. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. Against the background of current good times, it is essential that sound budgetary positions are reached in countries with fiscal imbalances and that a pro-cyclical loosening is avoided in all member countries. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. Turning to fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. MORE TEXT"), stringsAsFactors = FALSE)
我有一个包含 213 个文档的语料库，这些文档的长度各不相同。我的目标是从每个文档中提取一段特定的文本，它指的是 "fiscal policy"。使我的尝试复杂化的是我想要提取的文本位在文本之间是不一样的。唯一经常出现在开头的关键词是 财政政策 或 财政政策，仅此而已。
举个例子：
df <- data.frame(Text = c("Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. Regarding regards fiscal policies, almost all euro area countries have submitted their updated stability programmes. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. Against the background of current good times, it is essential that sound budgetary positions are reached in countries with fiscal imbalances and that a pro-cyclical loosening is avoided in all member countries. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. Turning to fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. MORE TEXT"))
cp <- corpus (df)
最终目的是得到这样的语料库：
df <- data.frame(Text = c("As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future.", "Regarding regards fiscal policies, almost all euro area countries have submitted their updated stability programmes.", "As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. Against the background of current good times, it is essential that sound budgetary positions are reached in countries with fiscal imbalances and that a pro-cyclical loosening is avoided in all member countries.", "Turning to fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes."))
cp <- corpus(df)
请注意，即使我只是感兴趣的一点加上我不想要的 "MORE TEXT" 我也会很高兴。我可以简单地对其进行子集化。我没有设法到达那里。到目前为止，我已尝试使用 corpus_segment 以及使用数据帧失败。
有人可以帮我吗？
非常感谢！
不需要语料库函数的Base R解决方案：
trimws(grep("fiscal polic.*", unlist(strsplit(df$Text, "[.]")), ignore.case = TRUE, value = TRUE), "both")
回答进一步的问题——找到索引并使用它的子集数据：
# Return vector of sentences containing pattern:
trimws(grep("fiscal polic.*", unlist(strsplit(df$Text, "[.]")), ignore.case = TRUE, value = TRUE), "both")
# Store the matched text as a vector:
matched_text <- trimws(grep("fiscal .*", unlist(strsplit(df$Text, "[.]")), ignore.case = TRUE, value = TRUE), "both")
#Get the index of the dataframe for each element:
matched_text_idx <- sapply(matched_text, function(x){which(grepl(x, df$Text))})
# If you want to subset the dataframe to contain only the elements which contain pattern:
df$Text[(which(grepl("fiscal polic.*", df$Text)))]
数据：
df <- data.frame(Text = c("Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. Regarding regards fiscal policies, almost all euro area countries have submitted their updated stability programmes. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. As regards fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. These indications are a cause of concern and entail risks for the future. Against the background of current good times, it is essential that sound budgetary positions are reached in countries with fiscal imbalances and that a pro-cyclical loosening is avoided in all member countries. MORE TEXT", "Whosebug is a great place where very skilled people can give you advice on coding. It is so good that I hope they are going to sort this problem out. This problem is really killing me. Turning to fiscal policy, almost all euro area countries have submitted their updated stability programmes. While these programmes generally indicate that governments plan to proceed towards sound budgetary positions, there are also indications that budget targets do not consistently imply sufficient consolidation and that concrete and credible measures have not yet been specified in all programmes. MORE TEXT"), stringsAsFactors = FALSE)