Superset 1.0 报告不是运行 (docker)

Question

在超集上向运行获取报告时遇到问题，运行ning 最近来自 /docker

我可以从 ui 设置报告，但之后没有任何反应。

我看到 beat worker 正在运行ning 并且每 5 分钟醒来一次，我也在主要 worker 日志中看到：

[2021-02-07 20:01:00,071: DEBUG/MainProcess] TaskPool: Apply <function _fast_trace_task at 0x7f03c6533830> (args:('email_reports.schedule_hourly', 'f1a887b8-4cb6-4d99-aba2-d571f56fcf1c', {'lang': None, 'task': 'email_reports.schedule_hourly', 'id': 'f1a887b8-4cb6-4d99-aba2-d571f56fcf1c', 'root_id': None, 'parent_id': None, 'group': None, 'meth': None, 'shadow': None, 'eta': None, 'expires': None, 'retries': 0, 'timelimit': [None, None], 'argsrepr': None, 'kwargsrepr': None, 'origin': None, 'reply_to': 'feca0569-95f8-39e2-893e-f2f29b904c06', 'correlation_id': 'f1a887b8-4cb6-4d99-aba2-d571f56fcf1c', 'hostname': 'celery@ebb95abd0849', 'delivery_info': {'exchange': '', 'routing_key': 'celery', 'priority': 0, 'redelivered': None}, 'args': [], 'kwargs': {}}, ([], {}, {'callbacks': None, 'errbacks': None, 'chord': None, 'chain': None}), None, None) kwargs:{})
[2021-02-07 20:01:00,097: DEBUG/MainProcess] Task accepted: email_reports.schedule_hourly[f1a887b8-4cb6-4d99-aba2-d571f56fcf1c] pid:17
[2021-02-07 20:01:00,221: INFO/ForkPoolWorker-1] Task email_reports.schedule_hourly[f1a887b8-4cb6-4d99-aba2-d571f56fcf1c] succeeded in 0.1267093000060413s: None

我的配置：

docker-compose.yaml

已添加：

superset-beat:
  image: *superset-image
  container_name: superset_beat
  command: ["/app/docker/docker-bootstrap.sh", "beat"]
  env_file: docker/.env
  restart: unless-stopped
  depends_on: *superset-depends-on
  user: "root"
  volumes: *superset-volumes

docker-bootstrap.sh

修改为：

if [[ "" == "worker" ]]; then
  echo "Starting Celery worker..."
  celery worker --app=superset.tasks.celery_app:app --pool=prefork -O fair -c 4 -l DEBUG
elif [[ "" == "beat" ]]; then
  echo "Starting Celery beat..."
  celery beat --app=superset.tasks.celery_app:app -l DEBUG
elif [[ "" == "app" ]]; then
  echo "Starting web app..."
  flask run -p 8088 --with-threads --reload --debugger --host=0.0.0.0
fi

superset_config.py

修改为：

RESULTS_BACKEND = RedisCache(
    host=REDIS_HOST, port=REDIS_PORT)
FEATURE_FLAGS = {
    "ALERT_REPORTS": True,
}
class CeleryConfig(object):
    BROKER_URL = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
    CELERY_IMPORTS = (
        "superset.sql_lab",
        "superset.tasks",
    )
    CELERY_RESULT_BACKEND = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
    CELERY_ANNOTATIONS = {
        "tasks.add": {
            "rate_limit": "10/s",
        },
        "sql_lab.get_sql_results": {
            "rate_limit": "100/s",
        },
        "email_reports.send": {
            "rate_limit": "1/s",
            "time_limit": 120,
            "soft_time_limit": 150,
            "ignore_result": True,
        },
    }
    CELERYBEAT_SCHEDULE = {
        "email_reports.schedule_hourly": {
            "task": "email_reports.schedule_hourly",
            "schedule": crontab(minute=1, hour="*"),
        },
    }
    CELERY_TASK_PROTOCOL = 1
CACHE_CONFIG = {
    "CACHE_TYPE": "redis",
    "CACHE_DEFAULT_TIMEOUT": 60 * 60 * 24, # 1 day default (in secs)
    "CACHE_KEY_PREFIX": "superset_results",
    "CACHE_REDIS_URL": f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}",
}
CELERY_CONFIG = CeleryConfig
SQLLAB_CTAS_NO_LIMIT = True
ENABLE_SCHEDULED_EMAIL_REPORTS = True
EMAIL_NOTIFICATIONS = True
SMTP_HOST = "smtp.gmail.com"
SMTP_STARTTLS = True
SMTP_SSL = True
SMTP_USER = "df.team.test@gmail.com"
SMTP_PORT = 465
SMTP_PASSWORD = os.environ.get("SMTP_PASSWORD")
SMTP_MAIL_FROM = "df.team.test@gmail.com"

Answer 1

您应该在 CELERYBEAT_SCHEDULE 中设置 alerts and reports（如果您同时需要警报和报告功能）调度程序。这是一个示例配置：

CELERYBEAT_SCHEDULE = {
    'email_reports.schedule_hourly': {
        'task': 'email_reports.schedule_hourly',
        'schedule': crontab(minute=1, hour='*'),
    },
    'alerts.schedule_check': {
        'task': 'alerts.schedule_check',
        'schedule': crontab(minute='*', hour='*'),
    },
    'reports.scheduler': {
        'task': 'reports.scheduler',
        'schedule': crontab(minute='*', hour='*'),
    },
    'reports.prune_log': {
        'task': 'reports.prune_log',
        'schedule': crontab(minute=0, hour=0),
    },
    'cache-warmup-hourly': {
        'task': 'cache-warmup',
        'schedule': crontab(minute='*/30', hour='*'),
         'kwargs': {
            'strategy_name': 'top_n_dashboards',
            'top_n': 10,
            'since': '7 days ago',
        },
   },
}

Superset 1.0 报告不是运行 (docker)

Superset 1.0 Reports not running (docker)

docker

celery

apache-superset

Superset 1.0 报告不是 运行 (docker)

Superset 1.0 Reports not running (docker)

docker

celery

apache-superset

Superset 1.0 报告不是运行 (docker)