如何在 Glue python 中延迟加载日志消息？

Question

运行 glueetl，GlueVersion 2.0，python 3，AWS Glue 作业我正在尝试使用

执行推荐的 python3 延迟加载日志消息

    logger.info("Attempting to run python module 1 {entrypoint}".format(entrypoint=entrypoint))
    logger.info("Attempting to run python module 2 %s", entrypoint)

这会在第二行产生错误，但第一行会成功并打印字符串。

2021-06-25 04:12:58,782 INFO [Thread-7] log.GlueLogger (GlueLogger.scala:info(8)): Attempting to run python module 1 main.test_program

2021-06-25 04:12:58,818 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(70)): Error from Python:Traceback (most recent call last):
  File "/tmp/main_etl_script.py", line 103, in <module>
    spark=spark)
  File "/tmp/main_etl_script.py", line 32, in main_handler
    logger.info("Attempting to run python module 2 %s", entrypoint)
  File "/opt/amazon/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
    return f(*a, **kw)
  File "/opt/amazon/spark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 332, in get_return_value
    format(target_id, ".", name, value))
py4j.protocol.Py4JError: An error occurred while calling o85.info. Trace:
py4j.Py4JException: Method info([class java.lang.String, class java.lang.String]) does not exist
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
    at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
    at py4j.Gateway.invoke(Gateway.java:274)
    at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
    at py4j.commands.CallCommand.execute(CallCommand.java:79)
    at py4j.GatewayConnection.run(GatewayConnection.java:238)
    at java.lang.Thread.run(Thread.java:748)

我的记录器设置很简单：

import logging
...
glue_context = GlueContext(SparkContext())
logger = glue_context.get_logger()

为什么会出现这种情况？

Answer 1

它不喜欢 2 %s 之后的逗号":

logger.info("Attempting to run python module 2 %s", entrypoint)

用 f 字符串试试这个：

logger.info(f"Attempting to run python module 2 {entrypoint}")

如何在 Glue python 中延迟加载日志消息？

How to lazy load log messages in Glue python?

apache-spark

pyspark

aws-glue