如何从运行 Fargate ECS 任务中查看 Python 打印语句？

Question

我有一个 Fargate ECS 容器，用于通过 ECS 中的任务运行一个 Docker 容器。任务启动时，会调用一个sh脚本，runner.sh,

#!/bin/sh
echo "this line will get logged to ECS..."
python3 src/my_python_script.py # however print statements from this Python script are not logged to ECS

这又会启动一个长运行宁Python脚本，my_python_script.py。我知道 Python 脚本运行正常，因为它可以完成它需要做的事情，但我看不到 Python 脚本的输出。

my_python_script.py 里面有几个 print() 语句。在我的 ECS Fargate 任务的 CloudWatch 日志中，我看到了 sh 脚本 ("this line will get logged to ECS...") 的输出，但没有看到 Python 脚本中的 print() 语句的输出。

这是我的任务定义中的日志配置：

{
    "ipcMode": null,
    "executionRoleArn": "myecsTaskExecutionRolearn",
    "containerDefinitions": [
        {
            "dnsSearchDomains": null,
            "environmentFiles": null,
            "logConfiguration": {
                "logDriver": "awslogs",
                "secretOptions": null,
                "options": {
                    "awslogs-group": "/ecs/mylogsgroup",
                    "awslogs-region": "eu-west-1",
                    "awslogs-stream-prefix": "ecs"
                }
            },
            "entryPoint": null,
            "portMappings": [],
            "command": null,
            "linuxParameters": null,
            "cpu": 0,
            "environment": [],
            "resourceRequirements": null,
            "ulimits": null,
            "dnsServers": null,
            "mountPoints": [],
            "workingDirectory": null,
            "secrets": null,
            "dockerSecurityOptions": null,
            "memory": null,
            "memoryReservation": null,
            "volumesFrom": [],
            "stopTimeout": null,
            "image": "1234567.dck.aws.com/mydockerimage",
            "startTimeout": null,
            "firelensConfiguration": null,
            "dependsOn": null,
            "disableNetworking": null,
            "interactive": null,
            "healthCheck": null,
            "essential": true,
            "links": null,
            "hostname": null,
            "extraHosts": null,
            "pseudoTerminal": null,
            "user": null,
            "readonlyRootFilesystem": null,
            "dockerLabels": null,
            "systemControls": null,
            "privileged": null,
            "name": "my-task-definition-name"
        }
    ],
    "memory": "4096",
    "taskRoleArn": "myecsTaskRolearn",
    "family": "my-task-definition-name",
    "pidMode": null,
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "networkMode": "awsvpc",
    "cpu": "2048",
    "inferenceAccelerators": [],
    "proxyConfiguration": null,
    "volumes": [],
    "tags": []
}

Docker文件:


FROM rocker/verse:3.6.0
ENV DEBIAN_FRONTEND noninteractive

RUN install2.r --error \
    jsonlite

RUN echo "deb http://ftp.de.debian.org/debian testing main" >> /etc/apt/sources.list
RUN echo 'APT::Default-Release "stable";' | tee -a /etc/apt/apt.conf.d/00local
RUN apt-get update && apt-get -t testing install -y --force-yes python3.6
RUN apt-get update && apt-get -t testing install -y libmagick++-dev python3-pip python-setuptools 

RUN mkdir /app
WORKDIR /app
COPY ./src /app/src

RUN pip3 install --trusted-host pypi.python.org -r /app/requirements.txt

CMD /app/runner.sh

我想我正在遵循 https://docs.aws.amazon.com/AmazonECS/latest/userguide/using_awslogs.html 的 awslogs 说明，但也许不是？我是否需要做一些明显的事情来确保 Python 脚本中的 print() 语句被捕获到我的 ECS 任务的 CloudWatch 日志中？

Answer 1

在我看来，您可能需要处理一些事情。

首先是 Python 的默认缓冲行为，它可能会阻止输出显示。你需要停止这个。

您可以通过在 CMD 之前插入以下内容来正确设置 PYTHONUNBUFFERED 环境变量：

ENV PYTHONUNBUFFERED=1

其次，引用您链接的 Using the awslogs driver 文档：

The type of information that is logged by the containers in your task depends mostly on their ENTRYPOINT command. By default, the logs that are captured show the command output that you would normally see in an interactive terminal if you ran the container locally, which are the STDOUT and STDERR I/O streams. The awslogs log driver simply passes these logs from Docker to CloudWatch Logs. For more information on how Docker logs are processed, including alternative ways to capture different file data or streams, see View logs for a container or service in the Docker documentation.

因此，我将根据 the Exec form of ENTRYPOINT:

将 CMD 行替换为以下内容

ENTRYPOINT ["/app/runner.sh"]

这应该用于为您的 shell 脚本连接 STDOUT 和 STDERR I/O 流，并希望您的 Python 脚本连接到容器日志记录。

如何从运行 Fargate ECS 任务中查看 Python 打印语句？

How to see Python print statements from running Fargate ECS task?

python

docker

amazon-ecs

aws-fargate

如何从 运行 Fargate ECS 任务中查看 Python 打印语句？

How to see Python print statements from running Fargate ECS task?

python

docker

amazon-ecs

aws-fargate

如何从运行 Fargate ECS 任务中查看 Python 打印语句？