Google 云平台推送订阅发送重复的消息 ID 字段

Question

我们有一个 GCP pubsub 主题，其中推送订阅指向云运行应用程序。推送订阅向我们的云运行端点发送以下 json，具有重复的 messageId/message_id 字段：

{
  "message": {
    "attributes": {
      "bucketId": "...",
      "eventTime": "2021-07-22T15:56:45.913174Z",
      "eventType": "OBJECT_FINALIZE",
      "notificationConfig": "...",
      "objectGeneration": "1626969405908262",
      "objectId": "...",
      "payloadFormat": "JSON_API_V1"
    },
    "data": "...",
    "messageId": "2717017549133308",
    "message_id": "2717017549133308",
    "publishTime": "2021-07-22T15:56:46.081Z",
    "publish_time": "2021-07-22T15:56:46.081Z"
  },
  "subscription": "projects/.../subscriptions/..."
}

当我们尝试将此 json 转换为 ReceivedMessage protobuf 对象时出现问题：

import com.google.pubsub.v1.ReceivedMessage;

ReceivedMessage.Builder receivedMessageBuilder = ReceivedMessage.newBuilder();
JsonFormat.parser().merge(json, receivedMessageBuilder);

我在我的 Cloud 运行应用程序中使用最新版本的 protobuf-java：

    <dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java</artifactId>
      <version>3.17.3</version>
    </dependency>

由于 json 是由 GCP 生成的，我们正在将 json 转换为 Google 定义的 protobuf 对象，我希望它能正常工作。但相反，我们收到以下错误：

Caused by: com.google.protobuf.InvalidProtocolBufferException: Field google.pubsub.v1.PubsubMessage.message_id has already been set.
    at com.google.protobuf.util.JsonFormat$ParserImpl.mergeField(JsonFormat.java:1648)
    at com.google.protobuf.util.JsonFormat$ParserImpl.mergeMessage(JsonFormat.java:1500)
    at com.google.protobuf.util.JsonFormat$ParserImpl.merge(JsonFormat.java:1458)
    at com.google.protobuf.util.JsonFormat$ParserImpl.parseFieldValue(JsonFormat.java:1999)
    at com.google.protobuf.util.JsonFormat$ParserImpl.mergeField(JsonFormat.java:1663)
    at com.google.protobuf.util.JsonFormat$ParserImpl.mergeMessage(JsonFormat.java:1500)
    at com.google.protobuf.util.JsonFormat$ParserImpl.merge(JsonFormat.java:1458)
    at com.google.protobuf.util.JsonFormat$ParserImpl.merge(JsonFormat.java:1340)
    at com.google.protobuf.util.JsonFormat$Parser.merge(JsonFormat.java:476)

我是不是做错了什么？如果我手动删除重复的字段（messageId 和 publishTime），消息将按预期正常转换。但是由于 json 是由 GCP 内部生成的，所以我无法在云环境中对其进行任何控制。有没有人对我如何成功地将此 json 转换为 com.google.pubsub.v1.ReceivedMessage 有任何建议？

Answer 1

这是对 JSON 中字段表示方式所做更改的不幸怪癖。本来是用“message_id”和“publish_time”，但是为了匹配Protocol Buffer JSON standards, they were changed to "publishTime" and "messageId", respectively. In order to ensure backwards compatibility for those who depended on the old way, both are written to the message. Support for this case in the JSON parsers has changed over the years and there is an open issue for protocol buffers才处理的。目前，唯一的选择是在调用 JsonFormat.parser().merge.

之前从收到的 JSON 中删除字段

Google 云平台推送订阅发送重复的消息 ID 字段

Google Cloud Platform push subscription sending duplicate message ID fields

java

json

google-cloud-platform

google-cloud-pubsub

google-cloud-run