来自容器化 WebApp 的日志不会通过 Serilog.Sinks.Elasticsearch 发送到 Elasticsearch

Question

我正在测试 Serilog.Sinks.Elasticsearch 库和 ELK 堆栈（Elasticsearch 和 Kibana）的组合，用于从我的 ASP.NET Core 2.2 应用程序收集日志。 Web 应用程序和 ELK 服务通过 Docker 容器化。

当通过 docker-compose 旋转所有容器时，一切都很好，除非出于某种原因，Elasticsearch Sink 没有将 HomeController.cs 中生成的日志推送到 Elasticsearch。这可以在 http://localhost:9200/_cat/indices?v 上检查，其中没有生成带有 logstash-* 的新索引。

有趣的是，当运行通过 IIS（不在容器中）结合 WebApp 和容器中的 ELK 堆栈时，会生成日志并将其发送到 Elasticsearch。我假设 docker-compose.yml 文件中 WebApp 的端口或网络配置有问题。

我也试过在 appsettings.Development.json 中用 elasticsearch（elasticsearch 容器的实际名称）切换本地主机，但没有效果。

谁能帮我解决这个问题？谢谢。

Docker-compose.yml 文件:

version: '3.4'
services:
  elastic.serilog.web:
    image: ${DOCKER_REGISTRY-}elasticserilogweb
    build:
      context: .
      dockerfile: Elastic.Serilog.Web/Dockerfile

  elasticsearch:
   image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0
   container_name: elasticsearch
   ports:
    - "9200:9200"
   volumes:
    - elasticsearch-data:/usr/share/elasticsearch/data
   environment:
    - discovery.type=single-node
    - xpack.security.enabled=false
    - xpack.monitoring.enabled=true
    - xpack.watcher.enabled=false
    - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
   networks:
    - docker-network

  kibana:
   image: docker.elastic.co/kibana/kibana:7.2.0
   container_name: kibana
   ports:
    - "5601:5601"
   depends_on:
    - elasticsearch
   environment:
    - ELASTICSEARCH_HOSTS="http://elasticsearch:9200"
    - XPACK_MONITORING_ENABLED=true
   networks:
    - docker-network

networks:
  docker-network:
    driver: bridge

volumes:
  elasticsearch-data:

Docker-compose.override.yml 文件:

version: '3.4'
services:
  elastic.serilog.web:
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
    ports:
      - "80:7000"

Docker文件:

FROM microsoft/dotnet:2.1-aspnetcore-runtime AS base
WORKDIR /app
EXPOSE 80

FROM microsoft/dotnet:2.1-sdk AS build
WORKDIR /src
COPY Elastic.Serilog.Web/Elastic.Serilog.Web.csproj Elastic.Serilog.Web/
RUN dotnet restore Elastic.Serilog.Web/Elastic.Serilog.Web.csproj
COPY . .
WORKDIR /src/Elastic.Serilog.Web
RUN dotnet build Elastic.Serilog.Web.csproj -c Release -o /app

FROM build AS publish
RUN dotnet publish Elastic.Serilog.Web.csproj -c Release -o /app

FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "Elastic.Serilog.Web.dll"]

Appsettings.Development.json:

{
  "Logging": {
    "LogLevel": {
        "Default": "Error",
        "System": "Error",
        "Microsoft": "Warning"
    }
  },
  "AllowedHosts": "*",
  "ElasticConfiguration": {
    "Uri": "http://localhost:9200/"
  }
}

Startup.cs:

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Serilog;
using Serilog.Exceptions;
using Serilog.Sinks.Elasticsearch;
using System;

namespace Elastic.Serilog.Web
{
    public class Startup
    {
        public IConfiguration Configuration { get; }

        public Startup(IConfiguration configuration, IHostingEnvironment hostingEnvironment)
        {
            var builder = new ConfigurationBuilder()
                .SetBasePath(hostingEnvironment.ContentRootPath)
                .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
                .AddJsonFile($"appsettings.{hostingEnvironment.EnvironmentName}.json", reloadOnChange: true, optional: true)
                .AddEnvironmentVariables();

            Configuration = builder.Build();

            var elasticUri = Configuration["ElasticConfiguration:Uri"];

            Log.Logger = new LoggerConfiguration()
               .Enrich.FromLogContext()
               .Enrich.WithExceptionDetails()
               .Enrich.WithMachineName()
               .WriteTo.Elasticsearch(new ElasticsearchSinkOptions(new Uri(elasticUri))
               {
                   AutoRegisterTemplate = true,
               })
            .CreateLogger();
        }

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
        }

        public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
             loggerFactory.AddSerilog();

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }
            else
            {
                app.UseExceptionHandler("/Home/Error");
                app.UseHsts();
            }

            // app.UseHttpsRedirection();
            app.UseStaticFiles();
            // app.UseCookiePolicy();

            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=Home}/{action=Index}/{id?}");
            });
        }
    }
}

HomeController.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Elastic.Serilog.Web.Models;
using Microsoft.Extensions.Logging;
using Serilog;

namespace Elastic.Serilog.Web.Controllers
{
    public class HomeController : Controller
    {
        ILogger<HomeController> _logger;
        public HomeController(ILogger<HomeController> logger)
        {
            _logger = logger;
        } 

        public IActionResult Index()
        {
            _logger.LogInformation($"oh hai there! : {DateTime.UtcNow}");

            Log.Error("Test Serilog!");

            try
            {
                throw new Exception("oops. i haz cause error in UR codez.");
            }
            catch (Exception ex)
            {
                _logger.LogCritical("ur app haz critical error", ex);
                _logger.LogError(ex, "ur code iz buggy.");

                Log.Information("Test Serilog!");

            }

            return View();
        }

        public IActionResult About()
        {
            ViewData["Message"] = "Your application description page.";

            return View();
        }

        public IActionResult Contact()
        {
            ViewData["Message"] = "Your contact page.";

            return View();
        }

        public IActionResult Privacy()
        {
            return View();
        }

        [ResponseCache(Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
        public IActionResult Error()
        {
            return View(new ErrorViewModel { RequestId = Activity.Current?.Id ?? HttpContext.TraceIdentifier });
        }
    }
}

Answer 1

解决者：

1.) 确保docker-compose.yml中定义的所有服务都在同一个网络中。在我们的例子中，我们必须将所有服务放在自定义“docker-network”中。（如果我们跳过定义自定义网络 Docker 将使用默认网络，它也是桥接模式 - 所以理论上所有容器仍将使用相同的网络）。

2.) 由于我们是运行容器中的服务 "localhost" 与在 [= 中寻址端口 9200 不再相关32=]。我们需要将 localhost 更改为 Elastic Search 容器的实际名称 - 在我们的例子中 "elasticsearch":

"ElasticConfiguration": { "Uri": "http://elasticsearch:9200/" }

完成这两个步骤后，serilog-sinks-elasticsearch 会将所有日志推送到 Elastic Search，并且默认的 logstash-* 索引将在 Kibana 中可见。

Answer 2

该服务运行在容器下，这就是为什么您应该使用容器名称编写日志的原因，如下所示：

Appsettings.Development.json:

{
  "Logging": {
    "LogLevel": {
        "Default": "Error",
        "System": "Error",
        "Microsoft": "Warning"
    }
  },
  "AllowedHosts": "*",
  "ElasticConfiguration": {
    "Uri": "http://elasticsearch:9200/"
  }
}

您可以在此处找到一个完整示例，展示如何设置 Serilog、Seq、elasticsearch 和 kibana，以便在 Docker 容器解决方案下协同工作。

structure logging with serilog seq and elastic search under docker

来自容器化 WebApp 的日志不会通过 Serilog.Sinks.Elasticsearch 发送到 Elasticsearch

Logs from containerized WebApp are not send to Elasticsearch via Serilog.Sinks.Elasticsearch

elasticsearch

serilog

docker-compose

asp.net-core