如何用 Marshmallow 序列化 MongoDB ObjectId？

Question

我正在使用 marshmallow 和 mongoengine 在 Flask 之上构建和 API。当我打电话并且应该序列化 ID 时，我收到以下错误：

TypeError: ObjectId('54c117322053049ba3ef31f3') is not JSON serializable

我在其他库中看到了一些方法来覆盖 ObjectId 的处理方式。我还没有用 Marshmallow 弄明白，有人知道怎么做吗？

我的模型是：

class Process(db.Document):
    name = db.StringField(max_length=255, required=True, unique=True)
    created_at = db.DateTimeField(default=datetime.datetime.now, required=True)

我的序列化器：

class ProcessSerializer(Serializer):
    class Meta:
        fields = ("id", "created_at", "name")

和视图：

class ProcessView(Resource):
    def get(self, id):
        process = Process.objects.get_or_404(id)
        return ProcessSerializer(process).data

Answer 1

当您将 Meta.fields 传递给模式时，Marshmallow 会尝试为每个属性选择一个字段类型。因为它不知道 ObjectId 是什么，它只是将它传递给序列化的字典。当您尝试将其转储到 JSON 时，它不知道 ObjectId 是什么并引发错误。要解决这个问题，您需要告诉 Marshmallow 使用哪个字段作为 id。 BSON ObjectId 可以转换为字符串，因此使用 String 字段。

from marshmallow import Schema, fields

class ProcessSchema(Schema):
    id = fields.String()

    class Meta:
        additional =  ('created_at', 'name')

您还可以告诉 Marshmallow 要为 ObjectId 类型使用哪个字段，这样您就不必每次都添加该字段。

from bson import ObjectId
from marshmallow import Schema, fields

Schema.TYPE_MAPPING[ObjectId] = fields.String

Answer 2

marshmallow-mongoengine 这样做：

Marshmallow-Mongoengine is about bringing together a Mongoengine Document with a Marshmallow Schema.

import marshmallow_mongoengine as ma


class ProcessSchema(ma.ModelSchema):
    class Meta:
        model = Process

它有一个 ObjectId 字段 serializes/deserializes ObjectId。

Answer 3

您可以将 fields.Field class 扩展到 create your own field. Here's how marshmallow-mongoengine （在另一个答案中提到）实现了这个：

import bson
from marshmallow import ValidationError, fields, missing

class ObjectId(fields.Field):
    def _deserialize(self, value, attr, data):
        try:
            return bson.ObjectId(value)
        except Exception:
            raise ValidationError("invalid ObjectId `%s`" % value)

    def _serialize(self, value, attr, obj):
        if value is None:
            return missing
        return str(value)

然后：

class MySchema(Schema):
    id = ObjectId()

（我发现这在不使用 MongoEngine 时很有用，只使用 pymongo）

Answer 4

与上面的@dcollien 类似，我扩展了 fields.Field 并使用助手创建了我自己的自定义字段，类似于 Marshmallow 在内部处理字段类型的方式：

from marshmallow import fields, missing
from marshmallow.exceptions import ValidationError
from bson.objectid import ObjectId
from bson.errors import InvalidId
import json

def oid_isval(val: Any) -> bool:
    """
    oid_isval [summary]

    Parameters
    ----------
    val : {Any}
        Value to be assessed if its an ObjectId

    Returns
    ----------
    val : bool
        True if val is an ObjectId, otherwise false
    """    
    if ObjectId.is_valid(val):
        return val

def ensure_objid_type(val: Union[bytes, str, ObjectId]) -> ObjectId:
    """
    Ensures that the value being passed is return as an ObjectId and is a valid ObjectId

    Parameters
    ----------
    val : Union[bytes, str, ObjectId]
        The value to be ensured or converted into an ObjectId and is a valid ObjectId

    Returns
    ----------
    val : ObjectId
        Value of type ObjectId
        
    Raises
    ----------
    ValidationError: Exception
        If it's not an ObjectId or can't be converted into an ObjectId, raise an error.
            
    """
    try:
        # If it's already an ObjectId and it's a valid ObjectId, return it
        if isinstance(val, ObjectId) and oid_isval(val):
            logger.info(f"It's an ObjectId and it's valid! = {val}")
            return val
        
        # Otherwise, if it's a bytes object, decode it and turn it into a string
        elif isinstance(val, bytes):
            val = ObjectId(str(val.decode("utf-8")))
            logger.info(f"Decoded and converted bytes object to ObjectId! = {val}")

        # Otherwise, if it's a string, turn it into an ObjectId and check that it's valid 
        elif isinstance(val, str):
            val = ObjectId(val)
            logger.info(f"Converted str to ObjectId! = {val}")
        
        # Check to see if the converted value is a valid objectId
        if oid_isval(val):
            logger.info(f"It's a valid ObjectId! = {val}")
            return val
    except InvalidId as error:
        logger.error(f"Not a valid ObjectId = {val} | error = {error}")
        raise ValidationError(json.loads(json.dumps(f"{error}")))

class ObjectIdField(fields.Field):
    """Custom field for ObjectIds."""
    # Default error messages
    default_error_messages = {
        "invalid_ObjectId": "Not a valid ObjectId."
    }

    def _serialize(self, value, attr, obj, **kwargs) -> Optional[ObjectId]:
        if value is None:
            return None
        return ensure_objid_type(value)

    def _deserialize(self, value, attr, data, **kwargs):
        if value is None:
            return missing
        if not isinstance(value, (ObjectId, str, bytes)):
            raise self.make_error("_deserialize: Not a invalid ObjectId")
        try:
            return ensure_objid_type(value)
        except UnicodeDecodeError as error:
            raise self.make_error("invalid_utf8") from error
        except (ValueError, AttributeError, TypeError) as error:
            raise ValidationError("ObjectIds must be a 12-byte input or a 24-character hex string") from error

如何用 Marshmallow 序列化 MongoDB ObjectId？

How can I serialize a MongoDB ObjectId with Marshmallow?

python

flask

mongoengine

marshmallow