使用嵌套字段提升的查询不从 Elasticsearch 返回任何结果

Query with Nest field boosting returning no results from Elasticsearch

我在使用字段提升获取查询以与 Elasticsearch 一起工作时遇到了实际问题。我已经浏览过关于该主题的 Nest 文档,但它们并不是特别有用,所以我的代码实际上是基于这个问题的解决方案:Elastic Search using NEST Field Boosting.

如果我 运行 下面的查询,我得到一个结果,正如预期的那样:

var matches =
    _client.Search<SearchableMerchant>(
        s => s.From((page - 1) * pageSize)
            .Size(pageSize)
            .QueryString("*test*")
            .MinScore(1)
    );

但是,如果我尝试使用字段提升,使用以下内容,我找不到匹配项:

var matches =
            _client.Search<SearchableMerchant>(
                s => s.From((page - 1) * pageSize)
                    .Size(pageSize)
                    .Query(q => q
                        .Boosting(bq => bq
                            .Positive(pq => pq
                                .CustomScore(cbf => cbf
                                    .Query(cbfq => cbfq
                                        .QueryString(
                                            qs => qs
                                            .OnFieldsWithBoost(d => d
                                                .Add("opportunities.acquirerLocationMID", Math.Pow(2, 17))
                                                .Add("opportunities.amexMID", Math.Pow(2, 16))
                                                .Add("opportunities.epayMID", Math.Pow(2, 16))
                                                .Add("v1MerchantId", Math.Pow(2, 16))
                                                .Add("locatorId", Math.Pow(2, 15))
                                                .Add("opportunities.opportunityLocatorId", Math.Pow(2, 14))
                                                .Add("businessName", Math.Pow(2, 13))
                                                .Add("searchablePhone", Math.Pow(2, 12))
                                                .Add("address.postCodeDetails.postCode.postCode", Math.Pow(2, 11))
                                                .Add("contacts.contact.searchableEmailAddress", Math.Pow(2, 11))
                                                .Add("contacts.contact.searchableMainPhone", Math.Pow(2, 10))
                                                .Add("contacts.contact.searchableMobilePhone", Math.Pow(2, 10))
                                                .Add("contacts.contact.fullName", Math.Pow(2, 9))
                                                .Add("contacts.contact.surname", Math.Pow(2, 8))
                                                .Add("contacts.contact.firstName", Math.Pow(2, 7))
                                                .Add("searchableAddress", Math.Pow(2, 6))
                                                .Add("ownershipUser.username", Math.Pow(2, 5))
                                                .Add("ownershipUser.searchableFullName", Math.Pow(2, 4))
                                                .Add("ownershipUser.lastName", Math.Pow(2, 3))
                                                .Add("ownershipUser.firstName", Math.Pow(2, 2))
                                                .Add("opportunities.depositAccount", Math.Pow(2, 1))
                                                .Add("opportunities.depositIban", Math.Pow(2, 1))
                                                .Add("opportunities.feesAccount", Math.Pow(2, 1))
                                                .Add("opportunities.feesIban", Math.Pow(2, 1))
                                                //  TODO: Company registration number - somewhere in legal methinks
                                            )
                                            .Query(
                                                "*test*"
                                            )
                                        )
                                    )
                                )
                            )
                            .Negative(nq => nq
                                .Filtered(nfq => nfq
                                    .Query(qq => qq.MatchAll())
                                    .Filter(f =>
                                        f.Missing("opportunities.acquirerLocationMID")
                                        && f.Missing("opportunities.amexMID")
                                        && f.Missing("opportunities.epayMID")
                                        && f.Missing("v1MerchantId")
                                        && f.Missing("locatorId")
                                        && f.Missing("opportunities.opportunityLocatorId")
                                        && f.Missing("businessName")
                                        && f.Missing("searchablePhone")
                                        && f.Missing("address.postCodeDetails.postCode.postCode")
                                        && f.Missing("contacts.contact.searchableEmailAddress")
                                        && f.Missing("contacts.contact.searchableMainPhone")
                                        && f.Missing("contacts.contact.searchableMobilePhone")
                                        && f.Missing("contacts.contact.fullName")
                                        && f.Missing("contacts.contact.surname")
                                        && f.Missing("contacts.contact.firstName")
                                        && f.Missing("searchableAddress")
                                        && f.Missing("ownershipUser.username")
                                        && f.Missing("ownershipUser.searchableFullName")
                                        && f.Missing("ownershipUser.lastName")
                                        && f.Missing("ownershipUser.firstName")
                                        && f.Missing("opportunities.depositAccount")
                                        && f.Missing("opportunities.depositIban")
                                        && f.Missing("opportunities.feesAccount")
                                        && f.Missing("opportunities.feesIban")
                                    )
                                )
                            )
                            .NegativeBoost(0.01)
                        )
                    )
                    .MinScore(1)
            );

我意识到这段代码的结构可以更好,但现在我只想让字段增强查询正常工作 - 我稍后会整理它。

以下是我尝试过的一些方法:

  1. Nest 文档没有提及是否可以使用带有 属性 名称的 OnFieldsWithBoost 的主题。即,这样可以吗?

    .OnFieldsWithBoost(d => d .Add("businessName", Math.Pow(2, 13))

与此相反?

.OnFieldsWithBoost(d => d
    .Add(m => m.businessName, Math.Pow(2, 13))

我问的原因是我有我想要提升的子属性,它们位于集合中。例如,opportunities.opportunityLocatorId。机会显然是集合,我想匹配该集合中任何对象的 opportunityLocatorId 字段具有匹配值的位置。

这适用于字段 - 您可以使用 lambda 或字符串 - 但它适用于提升吗?

不知道,但我已经尝试了两种方法,精简查询以仅包含对 businessName 的提升,因为这是应该与字符串 'test' 匹配的字段,但是仍然没有返回结果。

我也试过去掉 .Negative 子句,以防万一它匹配了它不应该匹配的东西。它可以降低在 .Positive 子句中列出的任何字段中找不到匹配项的任何查询。仍然没有结果。

我还将 .NegativeBoost 值提高到 1(即没有效果,因此不应将任何结果过滤到低于 1 的分数,而这些分数并非以如此低的分数开始) 但是,同样,没有骰子。

这是我的索引的内容,这样您就可以看到 businessName 字段应该与第二个查询匹配 'test',就像与第一个查询一样:

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "merchantv2",
      "_type" : "searchablemerchant",
      "_id" : "00000000-0000-0000-0000-000000000000",
      "_score" : 1.0,
      "_source":{"merchantGuid":"00000000-0000-0000-0000-000000000000","v1MerchantId":0,"locatorId":"0","address":{"addressGuid":"00000000-0000-0000-0000-000000000000","postCodeDetails":{"postCodeKey":0,"postalDistrict":{"postalDistrictKey":0,"postalDistrict":""},"postalLocation":"0","latitude":0.0,"longitude":0.0,"townName":"None","countyKey":0,"countryKey":0,"postCode":{"postCodeKey":0,"postCode":" 0"}},"county":{"countyKey":0,"countyName":"","countryKey":0,"recStatus":3,"countryKeyValue":0},"countryKey":0,"addressTypeKey":0,"updateDate":"0001-01-01T00:00:00+00:00","createdDate":"2016-01-07T19:46:28.4463+00:00"},"searchableAddress":" 0","searchablePhone":"","searchableFax":"","businessName":"","contacts":[],"opportunities":[{"opportunityGuid":"00000000-0000-0000-0000-000000000000","merchantGuid":"00000000-0000-0000-0000-000000000000","location":{"locationGuid":"00000000-0000-0000-0000-000000000000","tradingAddress":{"verified":false,"addressGuid":"00000000-0000-0000-0000-000000000000","postCodeDetails":{"postCodeKey":0,"postalDistrict":{"postalDistrictKey":0,"postalDistrict":""},"postalLocation":"0","latitude":0.0,"longitude":0.0,"townName":"None","countyKey":0,"countryKey":0,"postCode":{"postCodeKey":0,"postCode":" 0"}},"county":{"countyKey":0,"countyName":"","countryKey":0,"recStatus":3,"countryKeyValue":0},"countryKey":0,"addressTypeKey":0,"updateDate":"0001-01-01T00:00:00+00:00","createdDate":"2016-01-07T19:46:28.4463+00:00"}},"opportunityLocatorId":"000000"}]}
    }, {
      "_index" : "merchantv2",
      "_type" : "searchablemerchant",
      "_id" : "5f55fe61-ca65-e411-93f3-0cc47a07ef4a",
      "_score" : 1.0,
      "_source":{"merchantGuid":"5f55fe61-ca65-e411-93f3-0cc47a07ef4a","locatorId":"PM227Z02","address":{"addressGuid":"5c55fe61-ca65-e411-93f3-0cc47a07ef4a","houseNumber":"242","streetName":"Acklam Road","houseName":"","flatAptSuite":"","townName":"London","postCodeDetails":{"postCodeKey":1,"postalDistrict":{"postalDistrictKey":2782,"postalDistrict":"W10"},"postalLocation":"5JJ","latitude":51.52094651,"longitude":-0.20149990,"townName":"London","countyKey":0,"countryKey":224,"postCode":{"postCodeKey":1,"postCode":"W10 5JJ"}},"county":{"countyKey":626,"countyName":"Kensington And Chelsea","countryKey":224,"recStatus":1,"countryKeyValue":224},"countryKey":224,"addressTypeKey":0,"updateDate":"0001-01-01T00:00:00+00:00","createdDate":"2016-01-07T19:46:28.4653+00:00"},"searchableAddress":"242 Acklam Road, London, Kensington And Chelsea, W10 5JJ","searchablePhone":"+44 2031954484","searchableFax":"","businessName":"Test Merchant","contacts":[],"opportunities":[]}
    } ]
  }
}

我在 Windows 7 上使用 Elasticsearch 1.7.1 和 Nest 1.7.1(是的,我知道,但这是客户端使用的)与 .NET 4.5.1。

我也尝试过捕获我的 Web API 和 elasticsearch 之间的流量,但无济于事。可能是配置问题,但 Fiddler 和 Wireshark/npcap 都无法捕获这两者之间的流量,两者都 运行ning 在本地机器上,所以我看不到发送到 elasticsearch 的实际请求,它我怀疑会有帮助。基本上我想知道 Nest 正在吞咽的 Elasticsearch 是否返回任何错误。

嗯……事实证明直觉是正确的。以下是 elasticsearch 日志文件中出现的示例:

[2016-01-08 10:14:01,534][DEBUG][action.search.type       ] [Rocket Racer] All shards failed for phase: [query]
org.elasticsearch.search.SearchParseException: [user][4]: from[0],size[20]: Parse Failure [Failed to parse source [{
  "from": 0,
  "size": 20,
  "min_score": 1.0,
  "query": {
    "boosting": {
      "positive": {
        "custom_score": {
          "query": {
            "query_string": {
              "query": "*test*",
              "fields": [
                "opportunities.acquirerLocationMID^131072",
                "opportunities.amexMID^65536",
                "opportunities.epayMID^65536",
                "v1MerchantId^65536",
                "locatorId^32768",
                "opportunities.opportunityLocatorId^16384",
                "businessName^8192",
                "searchablePhone^4096",
                "address.postCodeDetails.postCode.postCode^2048",
                "contacts.contact.searchableEmailAddress^2048",
                "contacts.contact.searchableMainPhone^1024",
                "contacts.contact.searchableMobilePhone^1024",
                "contacts.contact.fullName^512",
                "contacts.contact.surname^256",
                "contacts.contact.firstName^128",
                "searchableAddress^64",
                "ownershipUser.username^32",
                "ownershipUser.searchableFullName^16",
                "ownershipUser.lastName^8",
                "ownershipUser.firstName^4",
                "opportunities.depositAccount^2",
                "opportunities.depositIban^2",
                "opportunities.feesAccount^2",
                "opportunities.feesIban^2"
              ]
            }
          }
        }
      },
      "negative": {
        "filtered": {
          "query": {
            "match_all": {}
          },
          "filter": {
            "bool": {
              "must": [
                {
                  "missing": {
                    "field": "opportunities.acquirerLocationMID"
                  }
                },
                {
                  "missing": {
                    "field": "opportunities.amexMID"
                  }
                },
                {
                  "missing": {
                    "field": "opportunities.epayMID"
                  }
                },
                {
                  "missing": {
                    "field": "v1MerchantId"
                  }
                },
                {
                  "missing": {
                    "field": "locatorId"
                  }
                },
                {
                  "missing": {
                    "field": "opportunities.opportunityLocatorId"
                  }
                },
                {
                  "missing": {
                    "field": "businessName"
                  }
                },
                {
                  "missing": {
                    "field": "searchablePhone"
                  }
                },
                {
                  "missing": {
                    "field": "address.postCodeDetails.postCode.postCode"
                  }
                },
                {
                  "missing": {
                    "field": "contacts.contact.searchableEmailAddress"
                  }
                },
                {
                  "missing": {
                    "field": "contacts.contact.searchableMainPhone"
                  }
                },
                {
                  "missing": {
                    "field": "contacts.contact.searchableMobilePhone"
                  }
                },
                {
                  "missing": {
                    "field": "contacts.contact.fullName"
                  }
                },
                {
                  "missing": {
                    "field": "contacts.contact.surname"
                  }
                },
                {
                  "missing": {
                    "field": "contacts.contact.firstName"
                  }
                },
                {
                  "missing": {
                    "field": "searchableAddress"
                  }
                },
                {
                  "missing": {
                    "field": "ownershipUser.username"
                  }
                },
                {
                  "missing": {
                    "field": "ownershipUser.searchableFullName"
                  }
                },
                {
                  "missing": {
                    "field": "ownershipUser.lastName"
                  }
                },
                {
                  "missing": {
                    "field": "ownershipUser.firstName"
                  }
                },
                {
                  "missing": {
                    "field": "opportunities.depositAccount"
                  }
                },
                {
                  "missing": {
                    "field": "opportunities.depositIban"
                  }
                },
                {
                  "missing": {
                    "field": "opportunities.feesAccount"
                  }
                },
                {
                  "missing": {
                    "field": "opportunities.feesIban"
                  }
                }
              ]
            }
          }
        }
      },
      "negative_boost": 0.01
    }
  }
}]]
    at org.elasticsearch.search.SearchService.parseSource(SearchService.java:747)
    at org.elasticsearch.search.SearchService.createContext(SearchService.java:572)
    at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:544)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:306)
    at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:231)
    at org.elasticsearch.search.action.SearchServiceTransportAction.call(SearchServiceTransportAction.java:228)
    at org.elasticsearch.search.action.SearchServiceTransportAction.run(SearchServiceTransportAction.java:559)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: org.elasticsearch.index.query.QueryParsingException: [user] No query registered for [custom_score]
    at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:303)
    at org.elasticsearch.index.query.BoostingQueryParser.parse(BoostingQueryParser.java:63)
    at org.elasticsearch.index.query.QueryParseContext.parseInnerQuery(QueryParseContext.java:305)
    at org.elasticsearch.index.query.IndexQueryParserService.innerParse(IndexQueryParserService.java:382)
    at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:281)
    at org.elasticsearch.index.query.IndexQueryParserService.parse(IndexQueryParserService.java:276)
    at org.elasticsearch.search.query.QueryParseElement.parse(QueryParseElement.java:33)
    at org.elasticsearch.search.SearchService.parseSource(SearchService.java:731)
    ... 9 more

那我做错了什么?有谁知道如何修复 elasticsearch 显然不喜欢的第二个查询?有什么方法可以消除 Nest 中的任何错误吗?我希望有一个例外,但那并没有发生 - 它只是 returns 默默地使用一个空的匹配集合,并且集合上没有 属性 表明出现了问题。

感谢收到的任何帮助。

谢谢!

巴特

自定义分数查询已在 Elasticsearch 0.90.4 中弃用并在 Elasticsearch 1.x 中删除。它保留在 NEST 中以实现向后兼容性。相反,您应该使用 function score query.

然而,

NEST 应该指出通过 IsValid 属性 发生了错误,在这种情况下应该是 false。默认情况下,NEST 1.x 不会抛出 Elasticsearch 异常。您可以通过在 ConnectionSettings.

上设置 ThrowOnElasticsearchServerExceptions() 来启用此行为

旁注:在术语的开头使用通配符(例如 *test)通常是不好的做法,因为它会导致索引中的每个术语都被检查。您可能需要考虑修改映射并改用 nGram 分词器之类的东西。

事实证明,我想做的事情很简单,我只是在错误的兔子洞里消失了一会儿。例如,这里有一个 multi_match 查询,我对其应用了字段提升:

curl -XGET http://localhost:9200/merchantv2/_search -d '
{
    "query": {
        "multi_match": {
            "query": "test",
            "type": "phrase_prefix",
            "fields" : ["businessName^3", "address.streetName"]
        }
    }
}'

在这种情况下,我提升了 businessName 字段,使得其中找到的匹配项的重要性是 address.streetName 中找到的匹配项的三倍。似乎工作正常。

这是相关文档的 link:https://www.elastic.co/guide/en/elasticsearch/reference/1.7/query-dsl-multi-match-query.html (props to Val 为此,他针对不同的问题提出了建议。

感谢指点!