"Failed to submit supervisor: Request failed with status code 502" 关于向路由器提交摄取规范

Question

当我尝试向德鲁伊 UI（通过路由器）提交摄取规范时，我收到错误 "Failed to submit supervisor: Request failed with status code 502"。摄取规范适用于独立的德鲁伊服务器。

我已经使用 4 台机器设置了集群 - 1 台用于协调器和霸主（master），1 台用于历史和中间管理器（数据），1 台用于代理（查询），1 台用于路由器，有一个单独的动物园管理员的实例。日志中没有错误。

摄取规范如下：

{
  "type": "kafka",
  "dataSchema": {
    "dataSource": "table1",
    "parser": {
      "type": "string",
      "parseSpec": {
        "format": "json",
        "dimensionsSpec": {
          "dimensions": [
           // List of valid dimensions
          ]
        },
        "timestampSpec": {
          "column": "createdOnDate", // In 'YYYY-MM-DD' format
          "format": "iso"
        }
      }
    },
    "granularitySpec": {
      "type": "uniform",
      "segmentGranularity": "MONTH",
      "rollup": false,
      "queryGranularity": "none"
    },
    "metricsSpec": []
  },
  "ioConfig": {
    "type": "kafka",
    "topic": "mongotopic",
    "consumerProperties": {
      "bootstrap.servers": "ip:9092"
    },
    "useEarliestOffset": true
  },
  "tuningConfig": {
    "type": "kafka",
    "forceExtendableShardSpecs": true,
    "maxParseExceptions": 100,
    "maxSavedParseExceptions": 10
  }
}

common.runtime.properties内容如下：

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

#
# Extensions
#

# This is not the full list of Druid extensions, but common ones that people often use. You may need to change this list
# based on your particular setup.
druid.extensions.loadList=["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "druid-s3-extensions","postgresql-metadata-storage"]

# If you have a different version of Hadoop, place your Hadoop client jar files in your hadoop-dependencies directory
# and uncomment the line below to point to your directory.
#druid.extensions.hadoopDependenciesDir=/my/dir/hadoop-dependencies

#
# Logging
#

# Log all runtime properties on startup. Disable to avoid logging properties on startup:
druid.startup.logging.logProperties=true

#
# Zookeeper
#

druid.zk.service.host=druid-ip:2181
druid.zk.paths.base=/druid

#
# Metadata storage
#

# For Derby server on your Druid Coordinator (only viable in a cluster with a single Coordinator, no fail-over):
#druid.metadata.storage.type=derby
#druid.metadata.storage.connector.connectURI=jdbc:derby://metadata.store.ip:1527/var/druid/metadata.db;create=true
#druid.metadata.storage.connector.host=metadata.store.ip
#druid.metadata.storage.connector.port=1527

# For MySQL (make sure to include the MySQL JDBC driver on the classpath):
#druid.metadata.storage.type=mysql
#druid.metadata.storage.connector.connectURI=jdbc:mysql:///druid
#druid.metadata.storage.connector.user=druid
#druid.metadata.storage.connector.password=druid

# For PostgreSQL (make sure to additionally include the Postgres extension):
druid.metadata.storage.type=postgresql
druid.metadata.storage.connector.connectURI=jdbc:postgresql://db.example.com:5432/druid
druid.metadata.storage.connector.user=user
druid.metadata.storage.connector.password=password

#
# Deep storage
#

# For local disk (only viable in a cluster if this is a network mount):
# druid.storage.type=local
# druid.storage.storageDirectory=var/druid/segments

# For HDFS (make sure to include the HDFS extension and that your Hadoop config files in the cp):
#druid.storage.type=hdfs
#druid.storage.storageDirectory=/druid/segments

# For S3:
druid.storage.type=s3
druid.storage.bucket=valid-bucket
druid.storage.baseKey=druid/segments
druid.s3.accessKey=valid-key
druid.s3.secretKey=valid-secret

#
# Indexing service logs
#

# For local disk (only viable in a cluster if this is a network mount):
#druid.indexer.logs.type=file
#druid.indexer.logs.directory=var/druid/indexing-logs

# For HDFS (make sure to include the HDFS extension and that your Hadoop config files in the cp):
#druid.indexer.logs.type=hdfs
#druid.indexer.logs.directory=/druid/indexing-logs

# For S3:
druid.indexer.logs.type=s3
druid.indexer.logs.s3Bucket=druid-test-1
druid.indexer.logs.s3Prefix=druid/indexing-logs

#
# Service discovery
#

druid.selectors.indexing.serviceName=druid/overlord
druid.selectors.coordinator.serviceName=druid/coordinator

#
# Monitoring
#

druid.monitoring.monitors=["org.apache.druid.java.util.metrics.JvmMonitor"]
druid.emitter=logging
druid.emitter.logging.logLevel=info

# Storage type of double columns
# ommiting this will lead to index double as float at the storage layer

druid.indexing.doubleStorage=double

#
# SQL
#
druid.sql.enable=true

通过路由器的 IP (IP:8888/unified-console) 将规范提交到 task/supervisor/Submit Supervisor 选项卡时，我得到 Failed to submit supervisor: Request failed with UI 中的状态代码 502 错误。 Zookeeper、S3 和 Postgres 配置有效。 UI 显示 1 个中层管理人员、0 个历史记录、0 个数据源和 0 个细分市场。

Answer 1

这是因为 druid-kafka-indexing-service 扩展名在 common.runtime.properties 的扩展名列表中丢失了。

"Failed to submit supervisor: Request failed with status code 502" 关于向路由器提交摄取规范

"Failed to submit supervisor: Request failed with status code 502" on submitting ingestion spec to router

postgresql

amazon-s3

druid