集群中弹性搜索中的停用词分析器

Question

我想知道分析器（在elasticsearch中）应该只在主节点上实现还是必须在所有节点上单独实现？

找不到准确的答案。

请帮忙！

谢谢

{
filter: {
spanish_stemmer: {
type: "stemmer",
language: "spanish"
},
spanish_stop: {
type: "stop",
stopwords: "_spanish_"
}
},
my_analyzer: {
spanish: {
filter: [
"lowercase",
"spanish_stop",
"spanish_stemmer"
],
tokenizer: "standard"
}
},
mappings: {
customparent: {
properties: {
name: {
type: "string",
analyzer: "my_analyzer"
},
title: {
type: "string",
analyzer: "my_analyzer"
},
fullImageAltDescription: {
type: "string",
analyzer: "my_analyzer"
}
}
}
}
}

添加日志：-

我正在使用 Elasticsearch(2.4.1v) Spring 数据传输客户端从 elasticsearch 获取数据时遇到以下错误：-

org.elasticsearch.action.search.SearchPhaseExecutionException: 所有分片失败在 org.elasticsearch.action.search.AbstractSearchAsyncAction.onFirstPhaseResult(AbstractSearchAsyncAction.java:206) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:152) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:46) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:872) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:850) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.transport.TransportService$4.onFailure(TransportService.java:387) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39) ~[elasticsearch-2.4.0.jar:2.4.0] 在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_101] 在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_101] 在 java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_101] 由以下原因引起：org.elasticsearch.index.query.QueryParsingException：未找到 [match] 分析器 [my_analyzer] 在 org.elasticsearch.index.query.MatchQueryParser.parse(MatchQueryParser.java:101) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:250) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:320) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:223) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:218) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.search.SearchService.parseSource(SearchService.java:856) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.search.SearchService.createContext(SearchService.java:667) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:633) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.search.SearchService.executeDfsPhase(SearchService.java:264) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.search.action.SearchServiceTransportAction$SearchDfsTransportHandler.messageReceived(SearchServiceTransportAction.java:360) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.search.action.SearchServiceTransportAction$SearchDfsTransportHandler.messageReceived(SearchServiceTransportAction.java:357) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376) ~[elasticsearch-2.4.0.jar:2.4.0] 在 org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-2.4.0.jar:2.4.0] ... 省略了 3 个常用框架 00:52:06.100 [http-bio-8080-exec-3] DEBUG o.s.s.w.c.HttpSessionSecurityContextRepository - SecurityContext 为空或内容是匿名的 - 上下文不会存储在 HttpSession 中。 00:52:06.100 [http-bio-8080-exec-3] DEBUG o.s.s.w.c.SecurityContextPersistenceFilter - SecurityContextHolder 现在已清除，因为请求处理已完成 2017 年 9 月 5 日 12:52:06 上午 org.apache.catalina.core.StandardWrapperValve 调用严重：Servlet.service() for servlet [appServlet] 在路径 [/api] 上下文中抛出异常 [请求处理失败；嵌套异常是无法执行阶段[dfs]，所有分片都失败了； shardFailures {[tdPbTD9IQhSuvkFMYt1WbA][catalogviews][0]: RemoteTransportException[[Shellshock][my_ip:9300][indices:data/read/search[phase/dfs]]];嵌套：SearchParseException[无法解析搜索源 [{"from":0,"size":10,"query":{"match":{"title":{"query":"Top Coat":"type":"boolean","operator":"OR","analyzer":"my_analyzer","fuzziness":"1","minimum_should_match":"95%"}}}}]];嵌套：QueryParsingException[[匹配] 分析器 [my_analyzer] 未找到]； }{[tdPbTD9IQhSuvkFMYt1WbA][catalogviews][1]: RemoteTransportException[[Shellshock][my_ip:9300][indices:data/read/search[phase/dfs]]];嵌套：SearchParseException[无法解析搜索源 [{"from":0,"size":10,"query":{"match":{"title":{"query":"Top Coat":"type":"boolean","operator":"OR","analyzer":"my_analyzer","fuzziness":"1","minimum_should_match":"95%"}}}}]];嵌套：QueryParsingException[[匹配] 分析器 [my_analyzer] 未找到]； }{[tdPbTD9IQhSuvkFMYt1WbA][catalogviews][2]: RemoteTransportException[[Shellshock][my_ip:9300][indices:data/read/search[phase/dfs]]];嵌套：SearchParseException[无法解析搜索源 [{"from":0,"size":10,"query":{"match":{"title":{"query":"Top Coat":"type":"boolean","operator":"OR","analyzer":"my_analyzer","fuzziness":"1","minimum_should_match":"95%"}}}}]];嵌套：QueryParsingException[[匹配] 分析器 [my_analyzer] 未找到]； }{[tdPbTD9IQhSuvkFMYt1WbA][catalogviews][3]: RemoteTransportException[[Shellshock][my_ip:9300][indices:data/read/search[phase/dfs]]];嵌套：SearchParseException[无法解析搜索源 [{"from":0,"size":10,"query":{"match":{"title":{"query":"Top Coat":"type":"boolean","operator":"OR","analyzer":"my_analyzer","fuzziness":"1","minimum_should_match":"95%"}}}}]];嵌套：QueryParsingException[[匹配] 分析器 [my_analyzer] 未找到]； }{[tdPbTD9IQhSuvkFMYt1WbA][catalogviews][4]: RemoteTransportException[[Shellshock][my_ip:9300][indices:data/read/search[phase/dfs]]];嵌套：SearchParseException[无法解析搜索源 [{"from":0,"size":10,"query":{"match":{"title":{"query":"Top Coat":"type":"boolean","operator":"OR","analyzer":"my_analyzer","fuzziness":"1","minimum_should_match":"95%"}}}}]];嵌套：QueryParsingException[[匹配] 分析器 [my_analyzer] 未找到]； }] 有根本原因 [目录视图] QueryParsingException [[匹配] 分析器 [my_analyzer] 未找到] 在 org.elasticsearch.index.query.MatchQueryParser.parse(MatchQueryParser.java:101) 在 org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:250) 在 org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:320) 在 org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:223) 在 org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:218) 在 org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33) 在 org.elasticsearch.search.SearchService.parseSource(SearchService.java:856) 在 org.elasticsearch.search.SearchService.createContext(SearchService.java:667) 在 org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:633) 在 org.elasticsearch.search.SearchService.executeDfsPhase(SearchService.java:264) 在 org.elasticsearch.search.action.SearchServiceTransportAction$SearchDfsTransportHandler.messageReceived(SearchServiceTransportAction.java:360) 在 org.elasticsearch.search.action.SearchServiceTransportAction$SearchDfsTransportHandler.messageReceived(SearchServiceTransportAction.java:357) 在 org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) 在 org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77) 在 org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376) 在 org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) 在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 在 java.lang.Thread.run(Thread.java:748)

Answer 1

您必须在所有节点中设置。

例如，如果您使用自己的词典设置同义词分析器，则必须将synonym.txt放在所有节点的特定文件夹中。

编辑：看起来你的映射没问题，尝试添加 'type' 自定义到你的分析器。

"analyzer": {
    "spanish": {
        "type": "custom",
        "tokenizer": "standard",
        "filter": [
            "lowercase",
            "spanish_stop",
            "spanish_stemmer"
        ]
    }
}

Answer 2

顺便说一句，问题出在 Ansaldos 提到的分析器映射本身。我不得不把 my_analyzer 放在分析器块中。我在下面添加了正确的映射。

{ "settings":{ "analysis":{ "filter":{ "spanish_stop":{ "type":"stop", "stopwords":"spanish" }, "spanish_stemmer":{ "type":"stemmer", "language":"spanish" } }, "analyzer":{ "my_analyzer":{ "tokenizer":"standard", "filter":[ "lowercase", "spanish_stop", "spanish_stemmer" ] } }, "mappings":{ "catalogviewparent":{ "properties":{ "title":{ "type":"string", "analyzer":"my_analyzer" }, "fullImageAltDescription":{ "type":"string", "analyzer":"my_analyzer" }, "name":{ "type":"string", "analyzer":"my_analyzer" } } } } } } }

集群中弹性搜索中的停用词分析器

Stop words analyzer in elasticsearch in a cluster

spring

elasticsearch

master-slave