Django 按日、周、月、年过滤时间戳数据 GROUP

Question

我有一个 django(DRF) 应用程序，我在其中存储基于 API 响应的周期性时间序列数据。这是我的 model.py

# Model to store the Alexa API Data
class Alexa(models.Model):
    created_at = models.DateTimeField(auto_now_add=True)
    extra = jsonfield.JSONField(null=True)
    rank =  models.PositiveIntegerField(default=0, null=True)

我正在使用 django-filters 根据范围（__lte、__gte）查询数据。喜欢/api/alexa/?created_at__lte=2020-02-14T09:15:52.329641Zreturn之前创建的所有数据2020-02-14T09:15:52.329641Z

[
    {
        "id": 1,
        "created_at": "2020-02-03T19:30:57.868588Z",
        "extra": "{'load_time': 00, 'backlink': 0}",
        "rank": 0
    },
    ...
 ]

有没有一种方法可以根据我传递的查询参数，为 return 按日、周、月和年分组的聚合数据构建端点。例如， /api/alexa/?created_at__lte=2020-02-14T09:15:52.329641Z&group_by=month 会 return

[
    {
        "created_at": "2020-01-01T00:00:00.000000Z",
        "extra": "{'load_time': 00, 'backlink': 0}", <- Aggregated Data 
        "rank": 0                                    <- Aggregated Data
    },
    {
        "created_at": "2020-02-01T00:00:00.000000Z",
        "extra": "{'load_time': 00, 'backlink': 0}", <- Aggregated Data 
        "rank": 0                                    <- Aggregated Data 
    },
 ]

这是我现在的 views.py

class AlexaViewSet(viewsets.ModelViewSet):
    queryset = Alexa.objects.all()
    filter_fields = {'created_at' : ['iexact', 'lte', 'gte']}
    http_method_names = ['get', 'post', 'head']

我看过几个片段进行聚合，但 none 完全满足我的要求，也没有让我对这个主题有一个完整的了解。

我是 Django 的新手，一般来说构建分析仪表板，如果有任何其他方式来表示此类时间序列数据以供在前端图表中使用，我也会感谢您的建议。

编辑：这是我的 serializer.py

class AlexaSerializer(serializers.ModelSerializer):
     class Meta:
         model = Alexa
         fields = '__all__'

Answer 1

首先，class AlexaViewSet 不是序列化器而是ViewSet。您没有在该 ViewSet 上指定序列化程序 class，所以您需要指定它。

另一方面，如果你想在 URL 上传递自定义查询参数，那么你应该覆盖此 ViewSet 的 list 方法并解析在 request 对象检索 group_by 的值，验证它，然后自己执行聚合。

我看到的另一个问题是你还需要定义什么是聚合一个JSON字段，这在SQL中是不支持的，而且它是非常相对的，所以你可能要考虑如果要对其中的字段执行聚合，请重新设计存储此 JSON 字段信息的方式。我建议从 JSON 中提取要聚合的字段（将它们存储在数据库中时）并将它们分别放在 SQL 列中，以便稍后执行聚合。客户端还可以将聚合操作作为查询参数传递，例如 aggregation=sum 或 aggregation=avg。

在简单的情况下，您只需要排名的平均值，这应该可以用作示例（您可以添加 TruncQuarter，等等）：

class AlexaViewSet(viewsets.ModelViewSet):
    serializer_class = AlexaSerializer
    queryset = Alexa.objects.all()
    filter_fields = {'created_at': ['iexact', 'lte', 'gte']}
    http_method_names = ['get', 'post', 'head']

    GROUP_CASTING_MAP = {  # Used for outputing the reset datetime when grouping
        'day': Cast(TruncDate('created_at'), output_field=DateTimeField()),
        'month': Cast(TruncMonth('created_at'), output_field=DateTimeField()),
        'week': Cast(TruncWeek('created_at'), output_field=DateTimeField()),
        'year': Cast(TruncYear('created_at'), output_field=DateTimeField()),
    }

    GROUP_ANNOTATIONS_MAP = {  # Defines the fields used for grouping
        'day': {
            'day': TruncDay('created_at'),
            'month': TruncMonth('created_at'),
            'year': TruncYear('created_at'),
        },
        'week': {
            'week': TruncWeek('created_at')
        },
        'month': {
            'month': TruncMonth('created_at'),
            'year': TruncYear('created_at'),
        },
        'year': {
            'year': TruncYear('created_at'),
        },
    }

    def list(self, request, *args, **kwargs):
        group_by_field = request.GET.get('group_by', None)
        if group_by_field and group_by_field not in self.GROUP_CASTING_MAP.keys():  # validate possible values
            return Response(status=status.HTTP_400_BAD_REQUEST)

        queryset = self.filter_queryset(self.get_queryset())

        if group_by_field:
            queryset = queryset.annotate(**self.GROUP_ANNOTATIONS_MAP[group_by_field]) \
                .values(*self.GROUP_ANNOTATIONS_MAP[group_by_field]) \
                .annotate(rank=Avg('rank'), created_at=self.GROUP_CASTING_MAP[group_by_field]) \
                .values('rank', 'created_at')

        page = self.paginate_queryset(queryset)
        if page is not None:
            serializer = self.get_serializer(page, many=True)
            return self.get_paginated_response(serializer.data)

        serializer = self.get_serializer(queryset, many=True)
        return Response(serializer.data)

对于这些值：

GET /alexa
[
    {
        "id": 1,
        "created_at": "2020-03-16T12:04:59.096098Z",
        "extra": "{}",
        "rank": 2
    },
    {
        "id": 2,
        "created_at": "2020-02-15T12:05:01.907920Z",
        "extra": "{}",
        "rank": 64
    },
    {
        "id": 3,
        "created_at": "2020-02-15T12:05:03.890150Z",
        "extra": "{}",
        "rank": 232
    },
    {
        "id": 4,
        "created_at": "2020-02-15T12:05:06.357748Z",
        "extra": "{}",
        "rank": 12
    }
]

GET /alexa/?group_by=day
[
    {
        "created_at": "2020-02-15T00:00:00Z",
        "extra": null,
        "rank": 102
    },
    {
        "created_at": "2020-03-16T00:00:00Z",
        "extra": null,
        "rank": 2
    }
]

GET /alexa/?group_by=week
[
    {
        "created_at": "2020-02-10T00:00:00Z",
        "extra": null,
        "rank": 102
    },
    {
        "created_at": "2020-03-16T00:00:00Z",
        "extra": null,
        "rank": 2
    }
]

GET /alexa/?group_by=month
[
    {
        "created_at": "2020-02-01T00:00:00Z",
        "extra": null,
        "rank": 102
    },
    {
        "created_at": "2020-03-01T00:00:00Z",
        "extra": null,
        "rank": 2
    }
]

GET /alexa/?group_by=year
[
    {
        "created_at": "2020-01-01T00:00:00Z",
        "extra": null,
        "rank": 77
    }
]

Django 按日、周、月、年过滤时间戳数据 GROUP

Django Filter timestamp data GROUP BY day, week, month, year

python

django

django-rest-framework

django-orm

django-filters