Kafka Connect JSON 架构似乎不支持“$ref”标签

Question

我正在将 Kafka Connect 与 JSONSchema 一起使用，我需要在 Kafka Connect 插件中手动转换 JSON 模式（到“Schema”）。我可以从架构注册表中成功检索 JSON 架构，并成功地使用简单的 JSON 架构进行转换，但是我在处理那些复杂且具有引用单个组件中的有效“$ref”标签的架构时遇到了困难JSON 模式定义。

我有几个问题：

JsonConverter.java 似乎无法处理“$ref”。我是正确的，还是它在其他地方以另一种方式处理它？
模式注册表是否处理子定义的引用？如果是，是否有代码显示如何处理解除引用？
在提交给 Schema Registry 之前，是否应该将 JSON Schema 解析为没有引用的字符串（即内联引用），从而消除“$ref”问题？

我正在查看下面的 Kafka 源代码模块JsonConverter.java：

https://github.com/apache/kafka/blob/trunk/connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java#L428

复杂模式的示例（取自 JSON 模式站点）如下所示（注意 "$ref": "#/$defs/veggie" 标记引用稍后的子-定义)

{
  "$id": "https://example.com/arrays.schema.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "description": "A representation of a person, company, organization, or place",
  "title": "complex-schema",
  "type": "object",
  "properties": {
    "fruits": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "vegetables": {
      "type": "array",
      "items": { "$ref": "#/$defs/veggie" }
    }
  },
  "$defs": {
    "veggie": {
      "type": "object",
      "required": [ "veggieName", "veggieLike" ],
      "properties": {
        "veggieName": {
          "type": "string",
          "description": "The name of the vegetable."
        },
        "veggieLike": {
          "type": "boolean",
          "description": "Do I like this vegetable?"
        }
      }
    }
  }
}

下面是模式注册成功后从模式注册表返回的实际模式：

[
  {
    "subject": "complex-schema",
    "version": 1,
    "id": 1,
    "schemaType": "JSON",
    "schema": "{\"$id\":\"https://example.com/arrays.schema.json\",\"$schema\":\"https://json-schema.org/draft/2020-12/schema\",\"description\":\"A representation of a person, company, organization, or place\",\"title\":\"complex-schema\",\"type\":\"object\",\"properties\":{\"fruits\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vegetables\":{\"type\":\"array\",\"items\":{\"$ref\":\"#/$defs/veggie\"}}},\"$defs\":{\"veggie\":{\"type\":\"object\",\"required\":[\"veggieName\",\"veggieLike\"],\"properties\":{\"veggieName\":{\"type\":\"string\",\"description\":\"The name of the vegetable.\"},\"veggieLike\":{\"type\":\"boolean\",\"description\":\"Do I like this vegetable?\"}}}}}"
  }
]

实际模式嵌入在上面返回的字符串中（“模式”字段的内容）并包含 $ref 引用：

{\"$id\":\"https://example.com/arrays.schema.json\",\"$schema\":\"https://json-schema.org/draft/2020-12/schema\",\"description\":\"A representation of a person, company, organization, or place\",\"title\":\"complex-schema\",\"type\":\"object\",\"properties\":{\"fruits\":{\"type\":\"array\",\"items\":{\"type\":\"string\"}},\"vegetables\":{\"type\":\"array\",\"items\":{\"$ref\":\"#/$defs/veggie\"}}},\"$defs\":{\"veggie\":{\"type\":\"object\",\"required\":[\"veggieName\",\"veggieLike\"],\"properties\":{\"veggieName\":{\"type\":\"string\",\"description\":\"The name of the vegetable.\"},\"veggieLike\":{\"type\":\"boolean\",\"description\":\"Do I like this vegetable?\"}}}}}

Answer 1

同样，Apache Kafka 源代码中的 JsonConverter 没有 JSONSchema 的概念，因此，不，$ref 不起作用，它也没有与注册表集成。

您似乎在寻找 io.confluent.connect.json.JsonSchemaConverter class + 逻辑

Kafka Connect JSON 架构似乎不支持“$ref”标签

Kafka Connect JSON Schema does not appear to support "$ref" tags

jsonschema

apache-kafka

apache-kafka-connect

confluent-platform