Python 2.7 和 3.7.2 兼容的 django-redis 序列化程序

Question

我正在尝试编写一个 py2.7 - py3.7 兼容的 django-redis 序列化程序。我将 django-redis==4.8.0 与 django==1.11.22 和 PickleSerializer 一起使用。我在 django-redis 上看到了这个问题 https://github.com/niwinz/django-redis/pull/279 并编写了一个类似于线程中所说的序列化程序。但是我的对象似乎有点复杂？不确定。

我的目标是同时拥有 2 个应用程序运行，一个使用 py2.7，另一个使用 py3.7。它们必须 100% 兼容，我无法克服这一点。

这是序列化程序的代码：

# -*- coding: utf-8 -*-
import six
from django.utils.encoding import force_bytes
from django_redis.serializers.pickle import PickleSerializer

try:
    import cPickle as pickle
except ImportError:
    import pickle


class CompatPickleSerializer(PickleSerializer):

    def loads(self, value):
        if six.PY3:
            return self._loads_py3(value)
        return super(CompatPickleSerializer, self).loads(force_bytes(value))

    def _loads_py3(self, value):
        return pickle.loads(
            force_bytes(value),
            fix_imports=True,
            encoding='bytes'
        )

我尝试序列化的对象示例：

{
    'created_at': datetime.datetime(2019, 7, 30, 20, 0, 29, 244916, tzinfo = < UTC > ),
    'items': [{
        'unit_price': Decimal('3.00'),
        'name': 'my item',
        'id': '12312312',
    }]
    'id': 'b5c6210d-561f-4e4e-a025-e55b39d95418',
    'name': 'cart',
    'customer': None,
}

这个对象比那个大很多，但我假设如果我可以用这个对象做流，我可以用一个更大的对象。

尝试 load python 3.7.2 上的对象后，出现此错误：

Traceback:  

File "my-project/lib/python3.7/site-packages/django_redis/client/default.py" in decode
  313.             value = int(value)


      During handling of the above exception (invalid literal for int() with base 10: b'\x80\x02}q\x01(U\tdiscountsq\x02NU\x10display_order_idq\x03NU\x12shipping_method_idq\x04NU\x0creservationsq\x05}U\ncreated_atq\x06U 2019-07-30T20:00:14.022071+00:00q\x07U\tpromocodeq\x08NU\x11shippi), another exception occurred:



File "my-project/lib/python3.7/site-packages/django/core/handlers/exception.py" in inner
  41.             response = get_response(request)

File "my-project/lib/python3.7/site-packages/django/core/handlers/base.py" in _get_response
  187.                 response = self.process_exception_by_middleware(e, request)

File "my-project/lib/python3.7/site-packages/django/core/handlers/base.py" in _get_response
  185.                 response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "my-project/lib/python3.7/site-packages/django/views/decorators/csrf.py" in wrapped_view
  58.         return view_func(*args, **kwargs)

File "my-project/lib/python3.7/site-packages/django/views/generic/base.py" in view
  68.             return self.dispatch(request, *args, **kwargs)

File "my-project/lib/python3.7/site-packages/rest_framework/views.py" in dispatch
  489.             response = self.handle_exception(exc)

File "my-project/lib/python3.7/site-packages/rest_framework/views.py" in handle_exception
  449.             self.raise_uncaught_exception(exc)

File "my-project/lib/python3.7/site-packages/rest_framework/views.py" in dispatch
  486.             response = handler(request, *args, **kwargs)

File "django/cart/api/cart.py" in get
  98.         cart = Cart.get(cart_id)

File "django/cart/models/cart.py" in get
  1.190.             raise e

File "django/cart/models/cart.py" in get
  1.186.             data = cart_cache.get(id)

File "my-project/lib/python3.7/site-packages/django_redis/cache.py" in _decorator
  33.             return method(self, *args, **kwargs)

File "my-project/lib/python3.7/site-packages/django_redis/cache.py" in get
  82.                                    client=client)

File "my-project/lib/python3.7/site-packages/django_redis/client/default.py" in get
  208.         return self.decode(value)

File "my-project/lib/python3.7/site-packages/django_redis/client/default.py" in decode
  320.             value = self._serializer.loads(value)

File "django/backports/django_redis/serializers.py" in loads
  28.             return self._loads_py3(value)

File "django/backports/django_redis/serializers.py" in _loads_py3
  35.             encoding='bytes'

Exception Type: TypeError at /my-url/
Exception Value: conversion from bytes to Decimal is not supported

对我能做什么有什么想法吗？

Answer 1

如果有人遇到这个问题，我来解决这个问题

# -*- coding: utf-8 -*-
import six
from django_redis.serializers.pickle import PickleSerializer, pickle


class CompatPickleSerializer(PickleSerializer):

    def loads(self, value):
        if six.PY3:
            return self._loads_py3(value)
        return super(CompatPickleSerializer, self).loads(value)

    def _loads_py3(self, value):
        return pickle.loads(
            value,
            fix_imports=True,
            encoding='latin1'
        )

我从 encoding='bytes' 更改为 latin1 并且有效。不需要 force_bytes.

另一个重要的事情是你需要强制PICKLE_VERSION为2，否则你将无法在python2.7上序列化数据[=24] =].

希望对大家有所帮助。

Python 2.7 和 3.7.2 兼容的 django-redis 序列化程序

Python 2.7 and 3.7.2 compatible django-redis serializer

django

pickle

django-redis