如何使用 SQLAlchemy 在 Python 运行时从 Google Cloud Function 访问非 Google MySQL 服务器数据库（无云 SQL！）
How to access a non-Google MySQL server database (no Cloud SQL!) from Google Cloud Function in Python runtime using SQLAlchemy
我尝试从 Python 运行时中的 Google 云功能连接到不由 Google 云托管的外部 MySQL 服务器数据库。
我的“requirements.txt”：
# Function dependencies, for example:
# package>=version
SQLAlchemy>=1.4.2
PyMySQL==1.0.2
云函数核心代码：
from os import environ
import sqlalchemy
db_user = environ["DB_USER"]
db_pass = environ["DB_PASS"]
db_name = environ["DB_NAME"]
db_host = environ["DB_HOST"]
db_port = environ["DB_PORT"] # if not used, default 3306 anyway
db_address = f"""mysql+pymysql://{db_user}:{db_pass}@{db_host}/{db_name}?charset=utf8&use_unicode=1"""
db_engine = sqlalchemy.create_engine(db_address)
当我在“测试”选项卡中测试 Cloud Function 时，按下 Test the function，出现错误：
for termination reason. Additional troubleshooting documentation can
be found at
https://cloud.google.com/functions/docs/troubleshooting#logging
Details: 500 Internal Server Error: The server encountered an internal
error and was unable to complete your request. Either the server is
overloaded or there is an error in the application. ```
并且在日志中：
或作为可搜索文本：
Debug2022-01-07T14:36:29.221892880ZMYCLOUDFUNCTIONdvaw7xewhjqj
Function execution started
Default2022-01-07T14:36:29.529ZMYCLOUDFUNCTIONdvaw7xewhjqj
OpenBLAS WARNING - could not determine the L2 cache size on this
system, assuming 256k
Debug2022-01-07T14:36:30.964665907ZMYCLOUDFUNCTIONdvaw7xewhjqj
Function execution took 1743 ms, finished with status code: 200
Debug2022-01-07T14:36:50.088620704ZMYCLOUDFUNCTIONdvawxkmbid1w
Function execution started Function execution started
Default2022-01-07T14:36:50.340ZMYCLOUDFUNCTIONdvawxkmbid1w
2022-01-07 14:36:50,267 [ERROR]: Exception on / [POST] 2022-01-07
14:36:50,267 [ERROR]: Exception on / [POST]
Error2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w
Traceback (most recent call last): File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 2447, in wsgi_app response = self.full_dispatch_request() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1952, in full_dispatch_request rv = self.handle_user_exception(e)
File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1821, in handle_user_exception reraise(exc_type, exc_value, tb)
File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py",
line 39, in reraise raise value File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1950, in full_dispatch_request rv = self.dispatch_request() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1936, in dispatch_request return
self.view_functionsrule.endpoint File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/init.py",
line 99, in view_func return function(request._get_current_object())
File "/workspace/main.py", line 139, in get_csv_in_tmp_and_move_to_gs
engine = sqlalchemy.create_engine(db_address) File "", line 2,
in create_engine File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/deprecations.py",
line 309, in warned Traceback (most recent call last): File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 2447, in wsgi_app response = self.full_dispatch_request() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1952, in full_dispatch_request rv = self.handle_user_exception(e)
File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1821, in handle_user_exception reraise(exc_type, exc_value, tb)
File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py",
line 39, in reraise raise value File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1950, in full_dispatch_request rv = self.dispatch_request() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1936, in dispatch_request return
self.view_functionsrule.endpoint File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/init.py",
line 99, in view_func return function(request._get_current_object())
File "/workspace/main.py", line 139, in get_csv_in_tmp_and_move_to_gs
engine = sqlalchemy.create_engine(db_address) File "", line 2,
in create_engine File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/deprecations.py",
line 309, in warned
Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w
return fn(*args, **kwargs) return fn(*args, **kwargs)
Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/create.py",
line 560, in create_engine File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/create.py",
line 560, in create_engine
Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w
dbapi = dialect_cls.dbapi(**dbapi_args) dbapi =
dialect_cls.dbapi(**dbapi_args)
Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/mysqldb.py",
line 163, in dbapi File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/mysqldb.py",
line 163, in dbapi
Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w
return import("MySQLdb") return import("MySQLdb")
Debug2022-01-07T14:36:50.342294068ZMYCLOUDFUNCTIONdvawxkmbid1w
Function execution took 254 ms, finished with status: 'crash' Function
execution took 254 ms, finished with status: 'crash' ```
当我在连接生命周期中使用 with 语句时，我得到的内容略有不同，但问题是一样的，它无法连接到数据库：
Traceback (most recent call last): File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 2447, in wsgi_app response = self.full_dispatch_request() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1952, in full_dispatch_request rv = self.handle_user_exception(e)
File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1821, in handle_user_exception reraise(exc_type, exc_value, tb)
File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py",
line 39, in reraise raise value File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1950, in full_dispatch_request rv = self.dispatch_request() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py",
line 1936, in dispatch_request return
self.view_functionsrule.endpoint File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/init.py",
line 99, in view_func return function(request._get_current_object())
File "/workspace/main.py", line 177, in get_csv_in_tmp_and_move_to_gs
connection = engine.connect() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 3204, in connect return self._connection_cls(self,
close_with_result=close_with_result) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 96, in init else engine.raw_connection() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 3283, in raw_connection return
self._wrap_pool_connect(self.pool.connect, _connection) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 3253, in _wrap_pool_connect
Connection.handle_dbapi_exception_noconnection( File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 2100, in handle_dbapi_exception_noconnection util.raise( File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/compat.py",
line 207, in raise raise exception File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py",
line 3250, in _wrap_pool_connect return fn() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py",
line 310, in connect return _ConnectionFairy._checkout(self) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py",
line 868, in _checkout fairy = _ConnectionRecord.checkout(pool) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py",
line 476, in checkout rec = pool._do_get() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/impl.py",
line 146, in do_get self.dec_overflow() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py",
line 70, in exit compat.raise( File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/compat.py",
line 207, in raise raise exception File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/impl.py",
line 143, in _do_get return self._create_connection() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py",
line 256, in _create_connection return _ConnectionRecord(self) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py",
line 371, in init self.__connect() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py",
line 666, in connect pool.logger.debug("Error on connect(): %s", e)
File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py",
line 70, in exit compat.raise( File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/compat.py",
line 207, in raise raise exception File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py",
line 661, in __connect self.dbapi_connection = connection =
pool._invoke_creator(self) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/create.py",
line 590, in connect return dialect.connect(*cargs, **cparams) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/default.py",
line 597, in connect return self.dbapi.connect(*cargs, **cparams) File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/pymysql/connections.py",
line 353, in init self.connect() File
"/layers/google.python.pip/pip/lib/python3.9/site-packages/pymysql/connections.py",
line 664, in connect raise exc sqlalchemy.exc.OperationalError:
(pymysql.err.OperationalError) (2003, "Can't connect to MySQL server
on 'MY_SERVER_ADDRESS' (timed out)") Default 2022-01-09T23:03:25.322Z
MYCLOUDFUNCTIONearoz5s8jozu (Background on this error at:
https://sqlalche.me/e/14/e3q8) (Background on this error at:
https://sqlalche.me/e/14/e3q8)
我在本地计算机和本地 Docker 容器中测试了相同的代码和设置，两者都可以使用相同的连接字符串 (db_url) 进行连接。
问题
如何连接到非Google MySQL 服务器？
也许还有一些路要走
想法
这是否只是一个权限问题，例如，我应该为我的用户角色添加一些权限吗？
想法
或者可能是 Google Cloud Functions 不允许对 Google 宇宙之外的服务器进行任何访问？
想法
并且如果只有 Cloud SQL 服务器被接受为 Cloud Functions 中的源（可能是这种情况，因为我只找到了这种情况的指南，例如来自 Google 的官方指南：Connecting from Cloud Functions to Cloud SQL or this Example of how to use MySQL in a Google Cloud Function)，我能否以某种方式使外部服务器成为 Google 接受的云 SQL 服务器而无需实际将其上传到那里，或者是否有任何其他解决方法？
想法
connection example from Cloud Functions to Google Cloud SQL (MySQL) 要求您激活云 SQL API 作为第一步，然后在连接字符串中使用套接字：
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
instance_connection_name = os.environ["INSTANCE_CONNECTION_NAME"]
pool = sqlalchemy.create_engine(
# Equivalent URL:
# mysql+pymysql://<db_user>:<db_pass>@/<db_name>?unix_socket=<socket_path>/<cloud_sql_instance_name>
sqlalchemy.engine.url.URL.create(
drivername="mysql+pymysql",
username=db_user, # e.g. "my-database-user"
password=db_pass, # e.g. "my-database-password"
database=db_name, # e.g. "my-database-name"
query={
"unix_socket": "{}/{}".format(
db_socket_dir, # e.g. "/cloudsql"
instance_connection_name) # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
}
),
**db_config
)
如果没有这样的 API 并且没有套接字来保护连接，SQL 查询是否可能无法触发到 MySQL 服务器，而不是 Google云SQL服务器?
想法
要不我改成GoogleCloud Run instead to have the chance to run my own container there? See . It seems that large file sizes, which can certainly arise from SQL query results in csv, are not at all recommended in Cloud Functions, see Streaming binary data from Google Cloud Storage to Cloud Function，它的注释：
... I do not recommend using Cloud Functions to process large files
though. I would rather recommend other serverless options as App
Engine, or Cloud Run. Large files processing may take a while and
Cloud Functions timeout after a certain time.
想法
下一条评论提到了 GCSFS（Python 模块），它可能不提供查询外部数据库的功能：
I did manage to get something working using GCSFS which allows me to
open files in cloud storage and stream binary as if they were local.
解决方案有效
这只是为了通过添加 VPC 连接器来确认接受的答案是否有效：
然后需要将其附加到应该使用它的 Google 云函数。在函数的“详细信息”选项卡下，您应该看到：
然后针对不属于 Google 云的服务器的查询应该有效。
如果数据库在 VM 上，并且在您的 VPC 中，您可以创建一个 VPC connector 并将其附加到您的 Cloud Function 以访问它。
如果部署在其他地方，
- 或者数据库有publicIP，Cloud Functions可以直接访问
- 或者数据库有一个私有IP，你需要在你的VPC和你的数据库的私有外部网络之间创建一个VPN。再次向 Cloud Functions 添加无服务器 VPC 连接器，以允许它通过您的 VPC 和 VPN 访问数据库。
我尝试从 Python 运行时中的 Google 云功能连接到不由 Google 云托管的外部 MySQL 服务器数据库。
我的“requirements.txt”：
# Function dependencies, for example:
# package>=version
SQLAlchemy>=1.4.2
PyMySQL==1.0.2
云函数核心代码：
from os import environ
import sqlalchemy
db_user = environ["DB_USER"]
db_pass = environ["DB_PASS"]
db_name = environ["DB_NAME"]
db_host = environ["DB_HOST"]
db_port = environ["DB_PORT"] # if not used, default 3306 anyway
db_address = f"""mysql+pymysql://{db_user}:{db_pass}@{db_host}/{db_name}?charset=utf8&use_unicode=1"""
db_engine = sqlalchemy.create_engine(db_address)
当我在“测试”选项卡中测试 Cloud Function 时，按下 Test the function，出现错误：
for termination reason. Additional troubleshooting documentation can be found at https://cloud.google.com/functions/docs/troubleshooting#logging Details: 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application. ```
并且在日志中：
或作为可搜索文本：
Debug2022-01-07T14:36:29.221892880ZMYCLOUDFUNCTIONdvaw7xewhjqj Function execution started Default2022-01-07T14:36:29.529ZMYCLOUDFUNCTIONdvaw7xewhjqj OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k Debug2022-01-07T14:36:30.964665907ZMYCLOUDFUNCTIONdvaw7xewhjqj Function execution took 1743 ms, finished with status code: 200 Debug2022-01-07T14:36:50.088620704ZMYCLOUDFUNCTIONdvawxkmbid1w Function execution started Function execution started Default2022-01-07T14:36:50.340ZMYCLOUDFUNCTIONdvawxkmbid1w 2022-01-07 14:36:50,267 [ERROR]: Exception on / [POST] 2022-01-07 14:36:50,267 [ERROR]: Exception on / [POST] Error2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w Traceback (most recent call last): File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise raise value File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/init.py", line 99, in view_func return function(request._get_current_object()) File "/workspace/main.py", line 139, in get_csv_in_tmp_and_move_to_gs engine = sqlalchemy.create_engine(db_address) File "", line 2, in create_engine File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned Traceback (most recent call last): File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise raise value File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/init.py", line 99, in view_func return function(request._get_current_object()) File "/workspace/main.py", line 139, in get_csv_in_tmp_and_move_to_gs engine = sqlalchemy.create_engine(db_address) File "", line 2, in create_engine File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/deprecations.py", line 309, in warned Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w return fn(*args, **kwargs) return fn(*args, **kwargs) Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 560, in create_engine File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 560, in create_engine Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w dbapi = dialect_cls.dbapi(**dbapi_args) dbapi = dialect_cls.dbapi(**dbapi_args) Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 163, in dbapi File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 163, in dbapi Default2022-01-07T14:36:50.341ZMYCLOUDFUNCTIONdvawxkmbid1w return import("MySQLdb") return import("MySQLdb") Debug2022-01-07T14:36:50.342294068ZMYCLOUDFUNCTIONdvawxkmbid1w Function execution took 254 ms, finished with status: 'crash' Function execution took 254 ms, finished with status: 'crash' ```
当我在连接生命周期中使用 with 语句时，我得到的内容略有不同，但问题是一样的，它无法连接到数据库：
Traceback (most recent call last): File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise raise value File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/layers/google.python.pip/pip/lib/python3.9/site-packages/functions_framework/init.py", line 99, in view_func return function(request._get_current_object()) File "/workspace/main.py", line 177, in get_csv_in_tmp_and_move_to_gs connection = engine.connect() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3204, in connect return self._connection_cls(self, close_with_result=close_with_result) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 96, in init else engine.raw_connection() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3283, in raw_connection return self._wrap_pool_connect(self.pool.connect, _connection) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3253, in _wrap_pool_connect Connection.handle_dbapi_exception_noconnection( File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2100, in handle_dbapi_exception_noconnection util.raise( File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3250, in _wrap_pool_connect return fn() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 310, in connect return _ConnectionFairy._checkout(self) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 476, in checkout rec = pool._do_get() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 146, in do_get self.dec_overflow() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get return self._create_connection() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection return _ConnectionRecord(self) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 371, in init self.__connect() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 666, in connect pool.logger.debug("Error on connect(): %s", e) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in exit compat.raise( File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect self.dbapi_connection = connection = pool._invoke_creator(self) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 590, in connect return dialect.connect(*cargs, **cparams) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 597, in connect return self.dbapi.connect(*cargs, **cparams) File "/layers/google.python.pip/pip/lib/python3.9/site-packages/pymysql/connections.py", line 353, in init self.connect() File "/layers/google.python.pip/pip/lib/python3.9/site-packages/pymysql/connections.py", line 664, in connect raise exc sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on 'MY_SERVER_ADDRESS' (timed out)") Default 2022-01-09T23:03:25.322Z MYCLOUDFUNCTIONearoz5s8jozu (Background on this error at: https://sqlalche.me/e/14/e3q8) (Background on this error at: https://sqlalche.me/e/14/e3q8)
我在本地计算机和本地 Docker 容器中测试了相同的代码和设置，两者都可以使用相同的连接字符串 (db_url) 进行连接。
问题
如何连接到非Google MySQL 服务器？
也许还有一些路要走
想法
这是否只是一个权限问题，例如，我应该为我的用户角色添加一些权限吗？
想法
或者可能是 Google Cloud Functions 不允许对 Google 宇宙之外的服务器进行任何访问？
想法
并且如果只有 Cloud SQL 服务器被接受为 Cloud Functions 中的源（可能是这种情况，因为我只找到了这种情况的指南，例如来自 Google 的官方指南：Connecting from Cloud Functions to Cloud SQL or this Example of how to use MySQL in a Google Cloud Function)，我能否以某种方式使外部服务器成为 Google 接受的云 SQL 服务器而无需实际将其上传到那里，或者是否有任何其他解决方法？
想法
connection example from Cloud Functions to Google Cloud SQL (MySQL) 要求您激活云 SQL API 作为第一步，然后在连接字符串中使用套接字：
# Remember - storing secrets in plaintext is potentially unsafe. Consider using
# something like https://cloud.google.com/secret-manager/docs/overview to help keep
# secrets secret.
db_user = os.environ["DB_USER"]
db_pass = os.environ["DB_PASS"]
db_name = os.environ["DB_NAME"]
db_socket_dir = os.environ.get("DB_SOCKET_DIR", "/cloudsql")
instance_connection_name = os.environ["INSTANCE_CONNECTION_NAME"]
pool = sqlalchemy.create_engine(
# Equivalent URL:
# mysql+pymysql://<db_user>:<db_pass>@/<db_name>?unix_socket=<socket_path>/<cloud_sql_instance_name>
sqlalchemy.engine.url.URL.create(
drivername="mysql+pymysql",
username=db_user, # e.g. "my-database-user"
password=db_pass, # e.g. "my-database-password"
database=db_name, # e.g. "my-database-name"
query={
"unix_socket": "{}/{}".format(
db_socket_dir, # e.g. "/cloudsql"
instance_connection_name) # i.e "<PROJECT-NAME>:<INSTANCE-REGION>:<INSTANCE-NAME>"
}
),
**db_config
)
如果没有这样的 API 并且没有套接字来保护连接，SQL 查询是否可能无法触发到 MySQL 服务器，而不是 Google云SQL服务器?
想法
要不我改成GoogleCloud Run instead to have the chance to run my own container there? See
... I do not recommend using Cloud Functions to process large files though. I would rather recommend other serverless options as App Engine, or Cloud Run. Large files processing may take a while and Cloud Functions timeout after a certain time.
想法
下一条评论提到了 GCSFS（Python 模块），它可能不提供查询外部数据库的功能：
I did manage to get something working using GCSFS which allows me to open files in cloud storage and stream binary as if they were local.
解决方案有效
这只是为了通过添加 VPC 连接器来确认接受的答案是否有效：
然后需要将其附加到应该使用它的 Google 云函数。在函数的“详细信息”选项卡下，您应该看到：
然后针对不属于 Google 云的服务器的查询应该有效。
如果数据库在 VM 上，并且在您的 VPC 中，您可以创建一个 VPC connector 并将其附加到您的 Cloud Function 以访问它。
如果部署在其他地方，
- 或者数据库有publicIP，Cloud Functions可以直接访问
- 或者数据库有一个私有IP，你需要在你的VPC和你的数据库的私有外部网络之间创建一个VPN。再次向 Cloud Functions 添加无服务器 VPC 连接器，以允许它通过您的 VPC 和 VPN 访问数据库。