如何使用 java api 在 elasticsearch 中搜索特定日期和时间范围内的日志
How to search log within specific range of date and time in elasticsearch using java api
我是 elasticsearch 及其 java api 的新手。我确实尝试编写 hello world java 程序来搜索一些字符串,在其中我将 matchQuery 函数与 QueryBuilder 一起使用并且它工作正常。代码如下。
代码:
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.aggregations.AggregationBuilders;
public class ElasticSearch {
public static void main(String[] args) {
SearchResponse response1=null;
Client client = new TransportClient()
.addTransportAddress(new InetSocketTransportAddress("192.168.1.142", 9301));
try{
//**** SEARCH *****//
response1 = client.prepareSearch("logstash-2015.03.03")
.setTypes("syslog")
.setSearchType(SearchType.QUERY_THEN_FETCH)
.setQuery(QueryBuilders.matchQuery("log_message", "Scanning directory or file : smb://test\":***@\"localhost/SambaShareIn"))
.setExplain(true)
.execute()
.actionGet();
System.out.println("*****************Hits***************"+response1.getHits().getTotalHits());
SearchHit[] searchHitArray = response1.getHits().getHits();
SearchHit searchHit = searchHitArray[0];
System.out.println("#########"+searchHit.getSourceAsString());
System.out.println("*****************Hits***************"+response1.getHits().getHits());
}catch(Exception e){
e.printStackTrace();
}
client.close();
if (response1.getHits().getTotalHits()>0) {
System.out.println("********Test Case Passed*******");
} else {
System.out.println("********Test Case not Passed*******");
int a=10/0;
}
}
}
输出:
*****************Hits***************104
#########{"message":"TID: [0] [ESB] [2015-02-05 18:06:14,458] DEBUG {org.apache.synapse.transport.vfs.VFSTransportListener} - Scanning directory or file : smb://test\":***@\"localhost/SambaShareIn {org.apache.synapse.transport.vfs.VFSTransportListener}","@version":"1","@timestamp":"2015-03-03T06:34:05.879Z","type":"syslog","host":"ubuntu","path":"/home/abc/Documents/wso2esb-4.8.0/repository/logs/wso2carbon.log","tenant_id":"0","server_type":"ESB","timestamp":"2015-02-05 18:06:14,458","level":"DEBUG","java_class":"org.apache.synapse.transport.vfs.VFSTransportListener","log_message":"Scanning directory or file : smb://test\":***@\"localhost/SambaShareIn {org.apache.synapse.transport.vfs.VFSTransportListener}"}
*****************Hits***************[Lorg.elasticsearch.search.internal.InternalSearchHit;@2eaae131
********Test Case Passed*******
但现在我想查找在特定日期和时间范围内记录的结果。我正在使用时间戳范围,但出现异常。代码和异常如下。
代码:
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.aggregations.AggregationBuilders;
public class ElasticSearch {
public static void main(String[] args) {
SearchResponse response1=null;
Client client = new TransportClient()
.addTransportAddress(new InetSocketTransportAddress("192.168.1.142", 9301));
try{
//**** SEARCH *****//
response1 = client.prepareSearch("logstash-2015.03.03")
.setTypes("syslog")
.setSearchType(SearchType.QUERY_THEN_FETCH)
.setQuery("range : {timestamp : {gt : now-24h}}")
.setExplain(true)
.execute()
.actionGet();
//System.out.println(response1);
System.out.println("*****************Hits***************"+response1.getHits().getTotalHits());
SearchHit[] searchHitArray = response1.getHits().getHits();
SearchHit searchHit = searchHitArray[0];
System.out.println("#########"+searchHit.getSourceAsString());
System.out.println("*****************Hits***************"+response1.getHits().getHits());
}catch(Exception e){
e.printStackTrace();
}
client.close();
if (response1.getHits().getTotalHits()>0) {
System.out.println("********Test Case Passed*******");
} else {
System.out.println("********Test Case not Passed*******");
int a=10/0;
}
}
}
异常:
org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed; shardFailures {[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][0]: SearchParseException[[logstash-2015.03.03][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@6e98e93a; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][1]: SearchParseException[[logstash-2015.03.03][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@5a4f889; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][2]: SearchParseException[[logstash-2015.03.03][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@6e98e93a; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][3]: SearchParseException[[logstash-2015.03.03][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@78f8178f; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][4]: SearchParseException[[logstash-2015.03.03][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@3e11473; line: 1, column: 7]]; }
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFailure(TransportSearchTypeAction.java:179)
at org.elasticsearch.search.action.SearchServiceTransportAction.run(SearchServiceTransportAction.java:565)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "main" java.lang.NullPointerException
at nl.weIntegrtae.Search.ElasticSearch.main(ElasticSearch.java:78)
任何人都可以帮助我找到特定日期和时间范围内的结果。
此致,
查询错误.. 使用 json 查询或纯 java 查询。您可以在日期范围查询的弹性搜索查询 dsl 上看到它。
它在 json 和 java 中都有查询。
要进行 json 查询,您可以使用 sense 插件进行弹性搜索。
像这样
Json查询--
{
"range" : {
"timestamp" : {
"gte": "now-24"
}
}
}
或者在java中创建q querybuilder并将其设置到setQuery方法中
QueryBuilder qb = QueryBuilders
.rangeQuery("timestamp")
.from("now-24")
.to("now");
我是 elasticsearch 及其 java api 的新手。我确实尝试编写 hello world java 程序来搜索一些字符串,在其中我将 matchQuery 函数与 QueryBuilder 一起使用并且它工作正常。代码如下。
代码:
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.aggregations.AggregationBuilders;
public class ElasticSearch {
public static void main(String[] args) {
SearchResponse response1=null;
Client client = new TransportClient()
.addTransportAddress(new InetSocketTransportAddress("192.168.1.142", 9301));
try{
//**** SEARCH *****//
response1 = client.prepareSearch("logstash-2015.03.03")
.setTypes("syslog")
.setSearchType(SearchType.QUERY_THEN_FETCH)
.setQuery(QueryBuilders.matchQuery("log_message", "Scanning directory or file : smb://test\":***@\"localhost/SambaShareIn"))
.setExplain(true)
.execute()
.actionGet();
System.out.println("*****************Hits***************"+response1.getHits().getTotalHits());
SearchHit[] searchHitArray = response1.getHits().getHits();
SearchHit searchHit = searchHitArray[0];
System.out.println("#########"+searchHit.getSourceAsString());
System.out.println("*****************Hits***************"+response1.getHits().getHits());
}catch(Exception e){
e.printStackTrace();
}
client.close();
if (response1.getHits().getTotalHits()>0) {
System.out.println("********Test Case Passed*******");
} else {
System.out.println("********Test Case not Passed*******");
int a=10/0;
}
}
}
输出:
*****************Hits***************104
#########{"message":"TID: [0] [ESB] [2015-02-05 18:06:14,458] DEBUG {org.apache.synapse.transport.vfs.VFSTransportListener} - Scanning directory or file : smb://test\":***@\"localhost/SambaShareIn {org.apache.synapse.transport.vfs.VFSTransportListener}","@version":"1","@timestamp":"2015-03-03T06:34:05.879Z","type":"syslog","host":"ubuntu","path":"/home/abc/Documents/wso2esb-4.8.0/repository/logs/wso2carbon.log","tenant_id":"0","server_type":"ESB","timestamp":"2015-02-05 18:06:14,458","level":"DEBUG","java_class":"org.apache.synapse.transport.vfs.VFSTransportListener","log_message":"Scanning directory or file : smb://test\":***@\"localhost/SambaShareIn {org.apache.synapse.transport.vfs.VFSTransportListener}"}
*****************Hits***************[Lorg.elasticsearch.search.internal.InternalSearchHit;@2eaae131
********Test Case Passed*******
但现在我想查找在特定日期和时间范围内记录的结果。我正在使用时间戳范围,但出现异常。代码和异常如下。
代码:
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.aggregations.AggregationBuilders;
public class ElasticSearch {
public static void main(String[] args) {
SearchResponse response1=null;
Client client = new TransportClient()
.addTransportAddress(new InetSocketTransportAddress("192.168.1.142", 9301));
try{
//**** SEARCH *****//
response1 = client.prepareSearch("logstash-2015.03.03")
.setTypes("syslog")
.setSearchType(SearchType.QUERY_THEN_FETCH)
.setQuery("range : {timestamp : {gt : now-24h}}")
.setExplain(true)
.execute()
.actionGet();
//System.out.println(response1);
System.out.println("*****************Hits***************"+response1.getHits().getTotalHits());
SearchHit[] searchHitArray = response1.getHits().getHits();
SearchHit searchHit = searchHitArray[0];
System.out.println("#########"+searchHit.getSourceAsString());
System.out.println("*****************Hits***************"+response1.getHits().getHits());
}catch(Exception e){
e.printStackTrace();
}
client.close();
if (response1.getHits().getTotalHits()>0) {
System.out.println("********Test Case Passed*******");
} else {
System.out.println("********Test Case not Passed*******");
int a=10/0;
}
}
}
异常:
org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed; shardFailures {[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][0]: SearchParseException[[logstash-2015.03.03][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@6e98e93a; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][1]: SearchParseException[[logstash-2015.03.03][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@5a4f889; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][2]: SearchParseException[[logstash-2015.03.03][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@6e98e93a; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][3]: SearchParseException[[logstash-2015.03.03][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@78f8178f; line: 1, column: 7]]; }{[ewJbD-euTBybuTt1-vgGgQ][logstash-2015.03.03][4]: SearchParseException[[logstash-2015.03.03][4]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query_binary":"cmFuZ2UgOiB7dGltZXN0YW1wIDoge2d0IDogbm93LTI0aH19","explain":true}]]]; nested: QueryParsingException[[logstash-2015.03.03] Failed to parse]; nested: JsonParseException[Unrecognized token 'range': was expecting ('true', 'false' or 'null')
at [Source: [B@3e11473; line: 1, column: 7]]; }
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:233)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFailure(TransportSearchTypeAction.java:179)
at org.elasticsearch.search.action.SearchServiceTransportAction.run(SearchServiceTransportAction.java:565)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Exception in thread "main" java.lang.NullPointerException
at nl.weIntegrtae.Search.ElasticSearch.main(ElasticSearch.java:78)
任何人都可以帮助我找到特定日期和时间范围内的结果。
此致,
查询错误.. 使用 json 查询或纯 java 查询。您可以在日期范围查询的弹性搜索查询 dsl 上看到它。 它在 json 和 java 中都有查询。 要进行 json 查询,您可以使用 sense 插件进行弹性搜索。 像这样 Json查询--
{
"range" : {
"timestamp" : {
"gte": "now-24"
}
}
}
或者在java中创建q querybuilder并将其设置到setQuery方法中
QueryBuilder qb = QueryBuilders
.rangeQuery("timestamp")
.from("now-24")
.to("now");