Pyspark Dataframe 到 AWS MySql：要求失败：驱动程序无法打开 JDBC 连接

Question

我想将 pyspark 数据帧写入 AWS RDS 中的 MySQL table，但我一直收到错误

pyspark.sql.utils.IllegalArgumentException: requirement failed: The driver could not open a JDBC connection. Check the URL: jdbc:mysql:mtestdb.ch4i3d3jc0yc.eu-central-1.rds.amazonaws.com

我的代码如下所示：

import os
import sys

spark = SparkSession.builder\
            .appName('test-app')\
            .config('spark.jars.packages', 'mysql:mysql-connector-java:8.0.28')\
            .getOrCreate()

properties = {'user':'admin', 'password':'password', 'driver':'com.mysql.cj.jdbc.Driver'}
resultDF.write.jdbc(url='jdbc:mysql:mtestdb.ch4i3d3jc0yc.eu-central-1.rds.amazonaws.com', table='mcm_objects', properties=properties)\
            .mode('append')\
            .save()

我也试过 url 'jdbc:mysql://mtestdb.ch4i3d3jc0yc.eu-central-1.rds.amazonaws.com', 但后来我得到了错误：

java.sql.SQLException: No database selected

不知道我做错了什么。任何帮助将不胜感激

Answer 1

table 应该是 {dbName}.{dbtable}:

resultDF.write.jdbc(url='jdbc:mysql:mtestdb.ch4i3d3jc0yc.eu-central-1.rds.amazonaws.com', table='{dbname}.mcm_objects', properties=properties)\
            .mode('append')\
            .save()

Pyspark Dataframe 到 AWS MySql：要求失败：驱动程序无法打开 JDBC 连接

Pyspark Dataframe to AWS MySql: requirement failed: The driver could not open a JDBC connection

apache-spark

pyspark

amazon-rds

mysql-connector

spark-jdbc