AWS Glue 作业 运行 失败 - 没有可用的 log4j-web 模块
AWS Glue job run failed - no log4j-web module available
我将下面的脚本写到 运行 一个 Glue 作业：
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
from pyspark.sql.functions import *
from awsglue.dynamicframe import DynamicFrame
args = getResolvedOptions(sys.argv, ['JOB_NAME'])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)
source_data = glueContext.create_dynamic_frame.from_catalog(database = "source_db", table_name = "source_table")
source_data.toDF().createOrReplaceTempView("data")
query = "SELECT id, date_created FROM data"
data_df = spark.sql(query)
data_dynamicframe = DynamicFrame.fromDF(data_df.repartition(1), glueContext, "data_dynamicframe")
target_data = glueContext.write_dynamic_frame.from_catalog(frame = data_dynamicframe, database = "target", table_name = "target_table", transformation_ctx = "target_data")
job.commit()
我在日志中收到了这条消息
Thread-4 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
有没有人遇到过同样的情况？脚本有问题吗？
谢谢！
原来是打错了！
脚本运行良好，我仍然收到以下消息
Executor task launch worker for task 0 INFO Log4j appears to be running
in a Servlet environment, but there's no log4j-web module available. \ If
you want better web container support, please add the log4j-web JAR to
your web archive or server lib directory.
2021-11-19 16:16:27,020 Executor task launch worker for task 0 INFO Log4j
appears to be running in a Servlet environment, but there's no log4j-web
module available. If you want better web container support, please add
the log4j-web JAR to your web archive or server lib directory.
而且我想这在未来值得研究。
我将下面的脚本写到 运行 一个 Glue 作业：
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
from pyspark.sql.functions import *
from awsglue.dynamicframe import DynamicFrame
args = getResolvedOptions(sys.argv, ['JOB_NAME'])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)
source_data = glueContext.create_dynamic_frame.from_catalog(database = "source_db", table_name = "source_table")
source_data.toDF().createOrReplaceTempView("data")
query = "SELECT id, date_created FROM data"
data_df = spark.sql(query)
data_dynamicframe = DynamicFrame.fromDF(data_df.repartition(1), glueContext, "data_dynamicframe")
target_data = glueContext.write_dynamic_frame.from_catalog(frame = data_dynamicframe, database = "target", table_name = "target_table", transformation_ctx = "target_data")
job.commit()
我在日志中收到了这条消息
Thread-4 INFO Log4j appears to be running in a Servlet environment, but there's no log4j-web module available. If you want better web container support, please add the log4j-web JAR to your web archive or server lib directory.
有没有人遇到过同样的情况？脚本有问题吗？ 谢谢！
原来是打错了！
脚本运行良好，我仍然收到以下消息
Executor task launch worker for task 0 INFO Log4j appears to be running
in a Servlet environment, but there's no log4j-web module available. \ If
you want better web container support, please add the log4j-web JAR to
your web archive or server lib directory.
2021-11-19 16:16:27,020 Executor task launch worker for task 0 INFO Log4j
appears to be running in a Servlet environment, but there's no log4j-web
module available. If you want better web container support, please add
the log4j-web JAR to your web archive or server lib directory.
而且我想这在未来值得研究。