Why BINARY usage in SQLAlchemy with Python3 cause a TypeError: 'string argument without an encoding'

Question

我读了很多类似的问题，但是 none 清楚地回答了我的问题。

我在 mysql table 列上使用 sqlalchemy-utils EncryptedType。
table 创建和插入没问题，但是当我尝试查询时收到：

Traceback (most recent call last):
  File "workspace/bin/test.py", line 127, in <module>
    result = session.query(Tester).all()
  File "workspace\ERP\venv\lib\site-packages\sqlalchemy\orm\query.py", line 3244, in all
    return list(self)
  File "workspace\venv\lib\site-packages\sqlalchemy\orm\loading.py", line 101, in instances
    cursor.close()
  File "workspace\venv\lib\site-packages\sqlalchemy\util\langhelpers.py", line 69, in __exit__
    exc_value, with_traceback=exc_tb,
  File "workspace\venv\lib\site-packages\sqlalchemy\util\compat.py", line 178, in raise_
    raise exception
  File "workspace\venv\lib\site-packages\sqlalchemy\orm\loading.py", line 81, in instances
    rows = [proc(row) for row in fetch]
  File "workspace\venv\lib\site-packages\sqlalchemy\orm\loading.py", line 81, in <listcomp>
    rows = [proc(row) for row in fetch]
  File "workspace\venv\lib\site-packages\sqlalchemy\orm\loading.py", line 642, in _instance
    populators,
  File "workspace\venv\lib\site-packages\sqlalchemy\orm\loading.py", line 779, in _populate_partial
    dict_[key] = getter(row)
  File "workspace\venv\lib\site-packages\sqlalchemy\engine\result.py", line 107, in __getitem__
    return processor(self._row[index])
  File "workspace\venv\lib\site-packages\sqlalchemy\sql\sqltypes.py", line 944, in process
    value = bytes(value)
TypeError: string argument without an encoding

我发现这个错误只发生在使用 python 3，而不是使用 python 2.
而且问题出在 sqlalchemy bynary 类型上，因为我在 Binary、Varbinary 和 Blob 列中遇到了相同的错误。
由于 python3 中的 bytes 需要字符串编码，我将第 944 行 sqlalchemy\sql\sqltypes.py 的代码更改为 value = bytes(value, 'utf-8) 并且效果很好，所以我的问题是：

为什么我需要更改 sqlalchemy 代码？ sqlalchemy 是否完全适用于 python3？更改包的代码是否安全？

这里是一个可以尝试的代码示例：

from sqlalchemy import MetaData, Integer, Column, Table, Binary, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, scoped_session

DB_CONFIG = {
        'user': 'user_test',
        'password': 'PSW_test',
        'host': '127.0.0.1',
        'database': 'db_test',
        }

if __name__ == '__main__':
    Base = declarative_base()
    engine = create_engine("mysql+mysqlconnector://%(user)s:%(password)s@%(host)s/%(database)s" % DB_CONFIG,
                           echo=False)
    Base.metadata.bind = engine
    db_sessionmaker = sessionmaker(bind=engine)
    Session = scoped_session(db_sessionmaker)

    # create the table
    meta = MetaData()
    tests = Table(
        'test', meta,
        Column('id', Integer, primary_key=True),
        Column('attr', Binary)
    )
    meta.create_all(engine)


    class Test(Base):
        __tablename__ = 'test'
        id = Column(Integer, primary_key=True)
        attr = Column(Binary)

    new_test = Test(attr='try'.encode('utf-8'))
    session = Session()
    session.add(new_test)
    session.commit()
    result = session.query(Test).all()
    for a in result:
        print(a, a.id, a.attr)
    Session.remove()

Answer 1

感谢 Ilja Everilä 提供的提示，我得以找到解决方案。也许不是最好的解决方案，但现在正在工作。

我认为根本原因是我的 DB-API 在查询过程中自动将 bytes 转换为 str。所以我只是通过向 create_engine:

添加一个参数来禁用此行为

engine = create_engine("mysql+mysqlconnector://%(user)s:%(password)s@%(host)s/%(database)s" % DB_CONFIG, connect_args={'use_unicode': False})

结果是，如果您有一个 String 列，它将在查询中作为 bytes 而不是 'str' 返回，您必须手动对其进行解码。

肯定有更好的解决方案。

Answer 2

MySQL 连接器似乎确实有问题。只需将 mysql-connector-python 切换为 mysqlclient。我遇到了同样的问题，它帮助了我。

Answer 3

而不是 mysql+mysqlconnector://<user>:<password>@<host>[:<port>]/<dbname> 你会得到 mysql+mysqldb://<user>:<password>@<host>[:<port>]/<dbname>

SQLAchemy 文档建议在 MySQL-Connector¶ 上使用 mysqlclient（MySQL-Python 的分支）¶。

The MySQL Connector/Python DBAPI has had many issues since its release, some of which may remain unresolved, and the mysqlconnector dialect is not tested as part of SQLAlchemy’s continuous integration. The recommended MySQL dialects are mysqlclient and PyMySQL.

Why BINARY usage in SQLAlchemy with Python3 cause a TypeError: 'string argument without an encoding'

Why BINARY usage in SQLAlchemy with Python3 cause a TypeError: 'string argument without an encoding'

python

mysql

sqlalchemy

python-3.x

sqlalchemy-utils