从列表中删除与子字符串匹配的项目的最快方法 - Python

Question

删除列表中与集合中的子字符串匹配的项的最快方法是什么？

例如，

the_list =
['Donald John Trump (born June 14, 1946) is an American businessman, television personality',
 'and since June 2015, a candidate for the Republican nomination for President of the United States in the 2016 election.',
 'He is the chairman and president of The Trump Organization and the founder of Trump Entertainment Resorts.',
 'Trumps career',
 'branding efforts',
 'personal life',
 'and outspoken manner have made him a celebrity.',
 'Trump is a native of New York City and a son of Fred Trump, who inspired him to enter real estate development.',
 'While still attending college he worked for his fathers firm',
 'Elizabeth Trump & Son. Upon graduating in 1968 he joined the company',
 'and in 1971 was given control, renaming the company The Trump Organization.',
 'Since then he has built hotels',
 'casinos',
 'golf courses',
 'and other properties',
 'many of which bear his name. He is a major figure in the American business scene and has received prominent media exposure']

这个列表实际上比这个长很多（数百万个字符串元素），我想删除集合中包含字符串的所有元素，例如，

{"Donald Trump", "Trump Organization","Donald J. Trump", "D.J. Trump", "dump", "dd"}

最快的方法是什么？循环是最快的吗？

Answer 1

如果您的字符串已经在内存中，请使用列表理解：

new = [line for line in the_list if not any(item in line for item in set_of_words)]

如果您没有将它们放在内存中作为内存使用方面更优化的方法，您可以使用生成器表达式：

new = (line for line in the_list if not any(item in line for item in set_of_words))

Answer 2

Aho-Corasick algorithm 就是专门为这个任务设计的。它具有比嵌套循环 O(n*m) 低得多的时间复杂度 O(n+m) 的明显优势，其中 n 是要查找的字符串数，m 是要搜索的字符串数。

有个不错的Python implementation of Aho-Corasick with accompanying explanation. There are also a couple of implementations at the Python Package Index不过没看过

从列表中删除与子字符串匹配的项目的最快方法 - Python

The fastest way to remove items that matches a substring from list - Python

python

list

set