Python 的 SyslogHandler 和 TCP

Question

我试图理解为什么来自 Python 的日志框架 (logging.handlers) 的 SyslogHandler class 没有实现 RFC 6587 描述的任何框架机制：

八位字节计数：它"prepends"系统日志帧的消息长度：
Non-Transparent-Framing：用于分隔消息的结尾字符。这是大多数服务器都知道的。

这个 "problem" 可以通过在消息末尾添加一个 LF 字符轻松解决，但是我希望 SyslogHandler 默认会处理这个问题：

sHandler = logging.handlers.SysLogHandler(address=(address[0], address[1]), socktype = socket.SOCK_STREAM)
sHandler.setFormatter(logging.Formatter(fmt=MSG_SYSLOG_FORMAT, datefmt=DATE_FMT))
self.addHandler(sHandler)

这对 Fluentd 和 rsyslog 都不起作用。正如我所说，我已将此临时添加到 handlers.py 的第 855 行（只是为了测试）：

msg = prio + msg + '\n'

现在正在工作。

我的问题：

Python SyslogHandler class 是否应该提供设置 on/off 八位字节计数或结尾字符的可能性。目前它什么都不做...
程序员的工作是了解服务器的工作原理并覆盖处理程序以解决消息框架问题？

目前，我正在做的是重写 emit() 方法，sub-classing SyslogHandler。

Answer 1

logging 中的 Syslog 支持早于 RFC，在该 RFC 之前，几乎没有标准。

准确地说：当 first added to the Python standard library in 2002 and has remained largely the same since (TCP support was added in 2009, and RFC5424 support was improved in 2011); the original code was based on this syslog module from 1997.

时，SysLogHandler 处理程序是 logging 的一部分

从 other bug reports 可以明显看出，维护者希望在此处的代码中保持最广泛的向后兼容性，因此如果您需要更新的 RFC 中的特定功能，您有两个选择：

扩展 class 并自行实现该功能
提交功能请求和/或补丁以改进 logging 模块中的功能；考虑向后兼容性要求。

Answer 2

鉴于问题被标记为 fluentd，您是否尝试过使用 fluent.handler.FluentHandler 代替 logging.handlers.SysLogHandler - 请参阅 https://github.com/fluent/fluent-logger-python？

Answer 3

感谢@Martijn Pieters♦ 的回答，我的回答扩展了他的回答。

我实现了一个继承自 SyslogHandler class 的 class 并覆盖了 emit 函数。我还为这个问题打开了一个拉取请求： https://github.com/python/cpython/pull/24556

python2:

import socket
import logging.handlers as handlers


class TcpSyslogHandler(handlers.SysLogHandler):
    """
    This class override the python SyslogHandler emit function.
    It is needed to deal with appending of the nul character to the end of the message when using TCP.
    Please see: 
    """
    def __init__(self, message_separator_character, address=('localhost', handlers.SYSLOG_UDP_PORT),
                 facility=handlers.SysLogHandler.LOG_USER,
                 socktype=None):
        """
        The user of this class must specify the value for the messages separator.
        :param message_separator_character: The value to separate between messages.
                                            The recommended value is the "nul character": "[=10=]0".
        :param address: Same as in the super class.
        :param facility: Same as in the super class.
        :param socktype: Same as in the super class.
        """
        super(SfTcpSyslogHandler, self).__init__(address=address, facility=facility, socktype=socktype)

        self.message_separator_character = message_separator_character

    def emit(self, record):
        """
        SFTCP addition:
        To let the user to choose which message_separator_character to use, we override the emit function.
        ####
        Emit a record.

        The record is formatted, and then sent to the syslog server. If
        exception information is present, it is NOT sent to the server.
        """
        try:
            msg = self.format(record) + self.message_separator_character

            """
            We need to convert record level to lowercase, maybe this will
            change in the future.
            """
            prio = '<%d>' % self.encodePriority(self.facility, self.mapPriority(record.levelname))
            # Message is a string. Convert to bytes as required by RFC 5424
            if type(msg) is unicode:
                msg = msg.encode('utf-8')
            msg = prio + msg
            if self.unixsocket:
                try:
                    self.socket.send(msg)
                except socket.error:
                    self.socket.close()  # See issue 17981
                    self._connect_unixsocket(self.address)
                    self.socket.send(msg)
            elif self.socktype == socket.SOCK_DGRAM:
                self.socket.sendto(msg, self.address)
            else:
                self.socket.sendall(msg)
        except (KeyboardInterrupt, SystemExit):
            raise
        except Exception:
            self.handleError(record)

python3:

import socket
import logging.handlers as handlers


class SfTcpSyslogHandler(handlers.SysLogHandler):
    """
    This class override the python SyslogHandler emit function.
    It is needed to deal with appending of the nul character to the end of the message when using TCP.
    Please see: 
    """
    def __init__(self, message_separator_character, address=('localhost', handlers.SYSLOG_UDP_PORT),
                 facility=handlers.SysLogHandler.LOG_USER,
                 socktype=None):
        """
        The user of this class must specify the value for the messages separator.
        :param message_separator_character: The value to separate between messages.
                                            The recommended value is the "nul character": "[=11=]0".
        :param address: Same as in the super class.
        :param facility: Same as in the super class.
        :param socktype: Same as in the super class.
        """
        super(SfTcpSyslogHandler, self).__init__(address=address, facility=facility, socktype=socktype)

        self.message_separator_character = message_separator_character

    def emit(self, record):
        """
        SFTCP addition:
        To let the user to choose which message_separator_character to use, we override the emit function.
        ####
        Emit a record.

        The record is formatted, and then sent to the syslog server. If
        exception information is present, it is NOT sent to the server.
        """
        try:
            msg = self.format(record) + self.message_separator_character
            if self.ident:
                msg = self.ident + msg

            # We need to convert record level to lowercase, maybe this will
            # change in the future.
            prio = '<%d>' % self.encodePriority(self.facility,
                                                self.mapPriority(record.levelname))
            prio = prio.encode('utf-8')
            # Message is a string. Convert to bytes as required by RFC 5424
            msg = msg.encode('utf-8')
            msg = prio + msg
            if self.unixsocket:
                try:
                    self.socket.send(msg)
                except OSError:
                    self.socket.close()
                    self._connect_unixsocket(self.address)
                    self.socket.send(msg)
            elif self.socktype == socket.SOCK_DGRAM:
                self.socket.sendto(msg, self.address)
            else:
                self.socket.sendall(msg)
        except Exception:
            self.handleError(record)

Python 的 SyslogHandler 和 TCP

Python's SyslogHandler and TCP

python

tcp

syslog

fluentd

我的问题：