AWS with .NET - 从 CloudWatch 读取日志 - 没有返回日志数据

Question

我正在尝试使用 .NET 从 CloudWatch 读取使用 SNS 发送的消息的日志数据。

从 CloudWatch 控制台 (CloudWatch \ CloudWatch Logs \ Logs Insights) 我输入：

Date range: custom (2w)
LogGroup: sns/ap...../8...../LogName 
Query: fields @timestamp, @message | sort @timestamp desc | limit 20

它 return 有很多日志记录 (@timestamp | @message)

我正在尝试使用 .net AWS SDK 从 c# 执行相同的操作：

public async Task GetLogs()
{
    string logGroupName = "sns/ap...../8...../LogName";

    AWSOptions options = configuration.GetAWSOptions();
    IAmazonCloudWatchLogs logs = options.CreateServiceClient<IAmazonCloudWatchLogs>();

    StartQueryRequest startQueryRequest = new StartQueryRequest();
    startQueryRequest.LogGroupName = logGroupName;
    startQueryRequest.StartTime = 1577850562; //1 Jan 2020
    TimeSpan t = DateTime.UtcNow - new DateTime(1970, 1, 1); //Epoch time starts on 1/1/1970
    int secondsSinceEpoch = (int)t.TotalSeconds;
    startQueryRequest.EndTime = secondsSinceEpoch;
    startQueryRequest.QueryString = "fields @timestamp, @message | sort @timestamp desc";
    startQueryRequest.Limit = 1000;
    StartQueryResponse response2 = await logs.StartQueryAsync(startQueryRequest);

    Console.WriteLine();
}

无法弄清楚为什么它没有 return 任何记录。

Response2 = 
   ContentLength: 50
   QueryId: "guid..."
   ResponseMetadata:
      Metadata Count = 0
      RequestId = "guid..."

知道我做错了什么吗？谢谢！

Answer 1

您只是开始查询，需要调用GetQueryResults获取数据：https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_GetQueryResults.html

Answer 2

以防对其他人有帮助 - 这是我使用 C# 从 AWS 获取日志的方法

public async Task GetLogs()
{
    string logGroupName = "log group name from CloudWatch Log";

    AWSOptions options = configuration.GetAWSOptions();
    IAmazonCloudWatchLogs logs = options.CreateServiceClient<IAmazonCloudWatchLogs>();

    StartQueryRequest startQueryRequest = new StartQueryRequest();
    startQueryRequest.LogGroupName = logGroupName;
    startQueryRequest.StartTime = 1577850562; //1 Jan 2020
    TimeSpan t = DateTime.UtcNow - new DateTime(1970, 1, 1); //Epoch time starts on 1/1/1970
    int secondsSinceEpoch = (int)t.TotalSeconds;
    startQueryRequest.EndTime = secondsSinceEpoch;
    startQueryRequest.QueryString = "fields @timestamp, @message | sort @timestamp desc";
    startQueryRequest.Limit = 1000;
    StartQueryResponse startQueryResponse = await logs.StartQueryAsync(startQueryRequest);

    GetQueryResultsRequest getQueryRequest = new GetQueryResultsRequest();
    getQueryRequest.QueryId = startQueryResponse.QueryId;

    GetQueryResultsResponse getQueryResults = await logs.GetQueryResultsAsync(getQueryRequest);
    for (var i=0; i<getQueryResults.Results.Count; i++)
    {
        ResultField timestampResult = getQueryResults.Results[i][0];
        ResultField messageResult = getQueryResults.Results[i][1];
        var message = JsonConvert.DeserializeObject(messageResult.Value);
        Console.WriteLine(message);
    }           
    Console.WriteLine();
}

Answer 3

我有一个没有拼写错误的工作示例，如果查询需要一些时间才能完成，它将重试：

            const string LogGroupName = "<your log group name here>";
            const string QueryString = "<your query string here>";

            var epoch = new DateTime( 1970, 1, 1 );
            var s = DateTime.UtcNow.AddMinutes( -15 ) - epoch;
            var e = DateTime.UtcNow - epoch;

            using( var client = new AmazonCloudWatchLogsClient() )
            {
                var startQueryRequest = new StartQueryRequest
                {
                    LogGroupName = LogGroupName,
                    StartTime = ( long ) s.TotalSeconds,
                    EndTime = ( long ) e.TotalSeconds,
                    QueryString = QueryString,
                    Limit = 1000
                };

                var startQueryResponse = await client.StartQueryAsync( startQueryRequest );
                var resultRequest = new GetQueryResultsRequest { QueryId = startQueryResponse.QueryId };
                var result = await client.GetQueryResultsAsync( resultRequest );

                // Retry loop with a timeout (AWS states that queries can run up to 15 minutes)
                for( var i = 0 ; i < 60 && ( result.Status == QueryStatus.Running || result.Status == QueryStatus.Scheduled ) ; i++ )
                {
                    System.Threading.Thread.Sleep( 250 );
                    result = await client.GetQueryResultsAsync( resultRequest );
                }

                // Query failed or timed out
                if( result.Status != QueryStatus.Complete )
                {
                    throw new Exception( "Could not complete query." );
                }

                foreach( var item in result.Results )
                {
                    // Use the results here
                }
            }